Share your repls and programming experiences

← Back to all posts
A Persistent Filesystem Solution for Repl.it
h
MrEconomical

Keep your files persistent

A persistent filesystem on repl.it is something I have wanted to see for a very, very long time. But unfortunately it isn't going to be added, which means that any programatic changes to files you make when you are offline will not be saved. This makes keeping logs or storing data within files difficult on repl.it

The team has introduced repl.it DB, a simple key-value storage built into your repls, to solve this issue. However, repl.it db isn't a perfect solution as it can't replace something like SQL or SQLite databases, and also your code will only ever work on repl.it.

That's why I've made this simple workaround, a script that saves your files into repl.it db. How the repl.it file system works is that any programatic changes will remain on the repl while it is running, but when it restarts from time to time and you aren't on it the changes will be lost. This script detects when files are changed, saves them into repl.it db, and when your repl restarts the script will fetch the changes and then replace your files. Additionally, the state management system using a checksum lets you edit your files as normal when your repl is not running and your manual changes will not be replaced by the old version stored in repl.it db.

I tried to make this script as efficient as possible, completely asynchronous to be non-blocking, and also added string compressing to save space within repl.it db as you only get 50 MB to work with. Every time the repl is run I also delete any files you don't want to keep persistant anymore from the database so you don't have to worry about any data being stuck in there and counting towards the data limit.

Using the script is very simple. All you need to do is include it into your program and run it at the very top. The function takes an array of all the files you want to keep persistent. You can also see the repl attached for an example.

One thing to note is that if you read any files at the beginning that haven't been replaced yet, the old version of the file without changes will still be there so you need to wait until the function is done. (If repl.it ever updates their node version we can use top level await and get rid of the ugly function thing)

Voters
programmeruser
sbtvdgamer
mkhoi
TylerArmstrong2
matthewproskils
lightningrock
CodeSalvageON
Nanowrimoijk
MatthewWesolows
AmazingMech2418
Comments
hotnewtop
jeanenchanted

I'm currently using python for this. Does anyone know of a python version?

[deleted]

Will data save if repl gonna force restart?

MrEconomical

@RocketR the idea is that when the repl restarts, normally the data would be wiped but this script saves it to repldb and loads the data on repl start

[deleted]

@MrEconomical Okay, but if this script won't work?
I tested it on my Discord bot with file database, this didn't save changed data.

MrEconomical

@RocketR it does

[deleted]

@MrEconomical Well, I tested your script on a Discord bot with sqlite3 database which has guild settings (language, prefix) etc. Always-on option of course.

Before using this script, I had several entries in my database. When the bot with the script was on, I added entry (custom prefix) and left it overnight. It so happened that Repl.it automatically restarted the bot during the night and unfortunately did not save the entry I added yesterday.

How can I solve it?

MrEconomical

@RocketR i cant say

Codemonkey51

but I want files bigger than 5mb https://docs.repl.it/misc/database

MrEconomical
Codemonkey51

Maybe like warn the user there file is too big, and tell them to just buy some s3 storage at this point @MrEconomical

AmazingMech2418

@Codemonkey51 It should be possible to divide the file into multiple keys...

Codemonkey51

The script should auto dew it :) makes it ezer.... @AmazingMech2418

AmazingMech2418

@Codemonkey51 Yeah. LOL! You could always modify the script though!

Codemonkey51

Me lazy moment, I mean I have a better idea for storage than repldb that I'm working on rn, but is currently still idea only :/ @AmazingMech2418

AmazingMech2418

@Codemonkey51 What is the idea?

Codemonkey51

Ah, can't give too much details to the public, lol I can dm u if you have discord, sry about that it's 2 reasons, 1 ppl trying to make it on there own bc it prob wont work well and theyll prob break ppls repls, 2 i don't want my idea stolen and have someone else implement it @AmazingMech2418

MrEconomical

@Codemonkey51 I did try to abuse repl.it's api to pretend that a user was on the repl 24/7 and it worked until they put a hidden captcha

AmazingMech2418

@Codemonkey51 Yeah. That makes sense. I don't have Discord, but maybe a multiplayer repl?

Codemonkey51

Lol, nah I have a better non-abuse idea, but it requires 1 I'll ask devs bc it may be annoying on them, 2 it also requires users get permissions, I'll leave it at that @MrEconomical

Codemonkey51

Ok :) you can invite me to one, just reply once you have so I can join @AmazingMech2418

Coder100

ew axios node-fetch better

MrEconomical

@Coder100 http.request gang rise up

Coder100
AmazingMech2418

Does this work with binary files?

MrEconomical

@AmazingMech2418 I don't see why it shouldn't

AmazingMech2418

@MrEconomical Sometimes, buffers don't convert to strings well...

MrEconomical

@AmazingMech2418 made it less shit now it writes as buffer

AmazingMech2418

@MrEconomical Nice! LOL! (careful about language though...)

ruby78027

wow!

zplusfour

Very cool!

HackermonDev

cool

rediar

very hot

ironblockhd

Nice! You could make it a node module so it's easier to include.

DynamicSquid

Ayyyyy, Mr. E's back!!!

Leroy01010