Skip to content
← Back to Community
A Persistent Filesystem Solution for Repl.it
Profile icon
h
has Hacker Plan
MrEconomical

Keep your files persistent

A persistent filesystem on repl.it is something I have wanted to see for a very, very long time. But unfortunately it isn't going to be added, which means that any programatic changes to files you make when you are offline will not be saved. This makes keeping logs or storing data within files difficult on repl.it

The team has introduced repl.it DB, a simple key-value storage built into your repls, to solve this issue. However, repl.it db isn't a perfect solution as it can't replace something like SQL or SQLite databases, and also your code will only ever work on repl.it.

That's why I've made this simple workaround, a script that saves your files into repl.it db. How the repl.it file system works is that any programatic changes will remain on the repl while it is running, but when it restarts from time to time and you aren't on it the changes will be lost. This script detects when files are changed, saves them into repl.it db, and when your repl restarts the script will fetch the changes and then replace your files. Additionally, the state management system using a checksum lets you edit your files as normal when your repl is not running and your manual changes will not be replaced by the old version stored in repl.it db.

I tried to make this script as efficient as possible, completely asynchronous to be non-blocking, and also added string compressing to save space within repl.it db as you only get 50 MB to work with. Every time the repl is run I also delete any files you don't want to keep persistant anymore from the database so you don't have to worry about any data being stuck in there and counting towards the data limit.

Using the script is very simple. All you need to do is include it into your program and run it at the very top. The function takes an array of all the files you want to keep persistent. You can also see the repl attached for an example.

One thing to note is that if you read any files at the beginning that haven't been replaced yet, the old version of the file without changes will still be there so you need to wait until the function is done. (If repl.it ever updates their node version we can use top level await and get rid of the ugly function thing)

Voters
Profile icon
KhaledAhmed18
Profile icon
SLMJ
Profile icon
ubershmekel
Profile icon
Lebyy_Dev
Profile icon
programmeruser
Profile icon
sbtvdgamer
Profile icon
mkhoi
Profile icon
TylerArmstrong2
Profile icon
matthewproskils
Profile icon
lightningrock
Comments
hotnewtop
Profile icon
ironblockhd

Nice! You could make it a node module so it's easier to include.

Profile icon
writeblankspace

I'm currently using python for this. Does anyone know of a python version?

Profile icon
Codemonkey51

but I want files bigger than 5mb https://docs.repl.it/misc/database

CA992191-4758-4B1E-A8CE-2E59B5DA30D5

Profile icon
MrEconomical
Profile icon
Codemonkey51

Maybe like warn the user there file is too big, and tell them to just buy some s3 storage at this point @MrEconomical

Profile icon
AmazingMech2418

@Codemonkey51 It should be possible to divide the file into multiple keys...

Profile icon
Codemonkey51

The script should auto dew it :) makes it ezer.... @AmazingMech2418

Profile icon
AmazingMech2418

@Codemonkey51 Yeah. LOL! You could always modify the script though!

Profile icon
Codemonkey51

Me lazy moment, I mean I have a better idea for storage than repldb that I'm working on rn, but is currently still idea only :/ @AmazingMech2418

Profile icon
AmazingMech2418

@Codemonkey51 What is the idea?

Profile icon
Codemonkey51

Ah, can't give too much details to the public, lol I can dm u if you have discord, sry about that it's 2 reasons, 1 ppl trying to make it on there own bc it prob wont work well and theyll prob break ppls repls, 2 i don't want my idea stolen and have someone else implement it @AmazingMech2418

Profile icon
MrEconomical

@Codemonkey51 I did try to abuse repl.it's api to pretend that a user was on the repl 24/7 and it worked until they put a hidden captcha

Profile icon
AmazingMech2418

@Codemonkey51 Yeah. That makes sense. I don't have Discord, but maybe a multiplayer repl?

Profile icon
Codemonkey51

Lol, nah I have a better non-abuse idea, but it requires 1 I'll ask devs bc it may be annoying on them, 2 it also requires users get permissions, I'll leave it at that @MrEconomical

Profile icon
Codemonkey51

Ok :) you can invite me to one, just reply once you have so I can join @AmazingMech2418

Profile icon
Coder100

ew axios node-fetch better

Profile icon
MrEconomical

@Coder100 http.request gang rise up

Profile icon
Coder100
Profile icon
HackermonDev

cool

Profile icon
foxo72

Good work, could you please add option that you can watch also directories?
Also same scripts for more databases such as MySQL DB, Mongo DB would be appreciated.

Profile icon
[deleted]

Will data save if repl gonna force restart?

Profile icon
MrEconomical

@RocketR the idea is that when the repl restarts, normally the data would be wiped but this script saves it to repldb and loads the data on repl start

Profile icon
[deleted]

@MrEconomical Okay, but if this script won't work?
I tested it on my Discord bot with file database, this didn't save changed data.

Profile icon
MrEconomical

@RocketR it does

Profile icon
[deleted]

@MrEconomical Well, I tested your script on a Discord bot with sqlite3 database which has guild settings (language, prefix) etc. Always-on option of course.

Before using this script, I had several entries in my database. When the bot with the script was on, I added entry (custom prefix) and left it overnight. It so happened that Repl.it automatically restarted the bot during the night and unfortunately did not save the entry I added yesterday.

How can I solve it?

Profile icon
MrEconomical

@RocketR i cant say

Profile icon
[deleted]
Profile icon
AmazingMech2418

Does this work with binary files?

Profile icon
MrEconomical

@AmazingMech2418 I don't see why it shouldn't

Profile icon
AmazingMech2418

@MrEconomical Sometimes, buffers don't convert to strings well...

Profile icon
MrEconomical

@AmazingMech2418 made it less shit now it writes as buffer

Profile icon
AmazingMech2418

@MrEconomical Nice! LOL! (careful about language though...)

Profile icon
ruby78027

wow!

Profile icon
zplusfour

Very cool!

Profile icon
rediar

very hot

Profile icon
DynamicSquid

Ayyyyy, Mr. E's back!!!

Profile icon
Leroy01010