Skip to content
Sign upLog in
← Back to Community
Why Do Node.JS Server Repls Take Up So Much Storage?
Profile icon
AmazingMech2418

I was just looking around my list of repls and noticed that some of them are very large, the largest ranging from 5MB to 10MB which is a lot with only a 100MB limit. However, all of these large repls are Node.JS repls with a server. The other ones are all under 1MB, the maximum being around 50KB. I then decided to download my 5.3MB repl and, using the Windows Command Line DIR command, noticed that the code is at max 1MB, not 5.3. So, where are those other 4.3MB coming from? Is there a way to reduce this size? Due to the usefulness of the Node.JS server repls, I want a majority of my repls to use Node.JS Express servers, but that is difficult to do if I can only do 10 to 20 of them.

Answered by Coder100 [earned 5 cycles]
View Answer
Voters
Profile icon
coderash
Profile icon
Codemonkey51
Profile icon
AmazingMech2418
Comments
hotnewtop
Profile icon
Coder100

repl.it takes account of all the npm packages you use.
So, if you use express, express uses a lot of files and takes up a lot of storage, and that's how it adds up.

Solution

Start using vanilla.js, meaning using only what you have such as http instead of express.
Hopes this helps!

Profile icon
AmazingMech2418

@Coder100
Do you know how to parse the request body? I've been working on creating a replacement for Express, but can't seem to parse the request body properly and all other NPM packages I found are either for a specific framework like Express or are asynchronous instead of synchronous.

Profile icon
Coder100

Try body-parser, or just looking at the headers.
If you are using the native http module, just do

console.log(JSON.stringify(req.headers, null, 2));

@AmazingMech2418

Profile icon
AmazingMech2418

@Coder100
This is what I got... {"host":"expressreplacement--amazingmech2418.repl.co","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","referer":"https://repl.it/@AmazingMech2418/ExpressReplacement","sec-fetch-dest":"iframe","sec-fetch-mode":"navigate","sec-fetch-site":"cross-site","upgrade-insecure-requests":"1","x-forwarded-for":"130.211.1.217","x-replit-user-id":"","x-replit-user-name":"","x-replit-user-roles":""}I already tried body-parser and it didn't work since it only works with Express I think.

Profile icon
AmazingMech2418

@Coder100
With the post request, I got {"host":"expressreplacement.amazingmech2418.repl.co","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36","content-length":"59","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","content-type":"application/x-www-form-urlencoded; charset=UTF-8","origin":"https://icom.amazingmech2418.repl.co","referer":"https://icom.amazingmech2418.repl.co/home","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-site","x-forwarded-for":"130.211.1.198","x-replit-user-id":"","x-replit-user-name":"","x-replit-user-roles":""}

Profile icon
AmazingMech2418

@Coder100
Update: I was able to use the qs package and create a full Express replacement in 6 KB (or at least enough to use it for what I used it for).

Profile icon
Coder100

Nice work :)
I looked at what body-parser did a little more and I think HTTP might already support it

@AmazingMech2418

Profile icon
AmazingMech2418

@Coder100
Thank you! I was looking at the documentation for body-parser and saw that it used the package qs, so I looked at its documentation and realized that it would work for URL encoded request bodies and that I could just easily add in JSON, text, and buffers to create a much lighter version of body-parser.

Profile icon
Coder100

Nice anyways, here's the repl: https://repl.it/@Coder100/HTTP-SERVER-MORE-VANILLA
So, now you just have to decode the buffer :)

@AmazingMech2418

Profile icon
AmazingMech2418
Profile icon
Coder100
Profile icon
AmazingMech2418

@Coder100
Thank you!

Profile icon
AmazingMech2418

@Coder100
The 5.3 MB repl is now down to 995 KB! It's crazy what making your own node module can do! However, I noticed that a lot of it was actually Socket.io, not Express. I didn't need Socket.io on that repl, so I just removed it.

Profile icon
Coder100

lol yeah, coding on your native desktop will really allow you to see the files that are downloaded in node_modules (repl.it hides that folder by default, probably because of the sheer size some projects take up)

@AmazingMech2418

Profile icon
AmazingMech2418

@Coder100
For me, NPM by default installs packages globally, so I can't actually see the node_modules folder. I don't know why, but that's what it does.

Profile icon
Coder100

hmmm
The only way for that to happen is

npm install -g

@AmazingMech2418

Profile icon
AmazingMech2418

@Coder100
Well, I just did npm start and I didn't see a node_modules folder appear.

Profile icon
Coder100

on your desktop?
Weird
well anyways, in the directory in question, just do

npm list --depth=0

and for global packages,

npm list -g --depth=0

@AmazingMech2418

Profile icon
VIPItsukaAllanB

.PHONY: all install compile run

all: compile

TARGET = main.cpp
TARGET_NAME = $(TARGET:%.cpp=%)

IMAGE = libjpeg-dev libwebp-dev libtiff5-dev libsdl2-image-dev libsdl2-image-2.0-0 libsdl-image1.2-dev

MIXER = libmikmod-dev libfishsound1-dev libsmpeg-dev liboggz2-dev libflac-dev libfluidsynth-dev libsdl2-mixer-dev libsdl2-mixer-2.0-0

TRUE_TYPE_FONTS = libfreetype6-dev libsdl2-ttf-dev libsdl2-ttf-2.0-0

FLAGS = sdl2-config --cflags --libs -lSDL2 -lSDL2_mixer -lSDL2_image -lSDL2_ttf

compile:
g++ $(TARGET) -o $(TARGET_NAME) $(FLAGS)

install:
#install sdl2
install-pkg libsdl2-dev libsdl2-2.0-0

#install sdl image - if you want to display images install-pkg $(IMAGE) #install sdl mixer - if you want sound install-pkg $(MIXER) #install sdl true type fonts - if you want to use text install-pkg $(TRUE_TYPE_FONTS)

run:
./$(TARGET_NAME)

Profile icon
[deleted]

@AmazingMech2418
So, I mentioned using Bash to fix this problem, if you would like to see it, I can link you to the Repl, I'm making right now.

Profile icon
AmazingMech2418

@JadenGarcia
Do you know how to put multiple servers there though? I don't even know if Repl.it can support that right now. However, one thing that would be great for Repl.it is user-defined node modules instead of repl-defined node modules so that modules carry into every repl a user makes.

Profile icon
staticvoidliam7

does'nt really matter there is no limit rn lol
Screenshot 2020-04-07 at 18.28.33

Profile icon
[deleted]

Stop using express! Use require(http) and you'll save massive amounts of data.

Profile icon
AmazingMech2418

@JadenGarcia
But I need post requests and only know how to do get requests without Express and its body-parser.

Profile icon
[deleted]

@AmazingMech2418
What's the body parser do?

Profile icon
[deleted]

@AmazingMech2418
Also, just learn how to do them.

Profile icon
AmazingMech2418

@JadenGarcia
It allows me to read the JSON data sent through the post request.

Profile icon
[deleted]

@AmazingMech2418
You can do that with JSON.parse(...).

Profile icon
AmazingMech2418

@JadenGarcia
However, based on past experience http servers normally don't support post requests anyways. I've tried before and it didn't work.

Profile icon
[deleted]

@AmazingMech2418
Use require("https") then?

Profile icon
AmazingMech2418

@JadenGarcia
Tried that too. Didn't work. I even tried doing fake post requests with get requests with a query string and it didn't work.

Profile icon
AmazingMech2418

@JadenGarcia
I learned how to use post requests in an http server, but have no idea on how to parse the URL encoded request body, given that objects and arrays are used (I can parse if there are first-level objects and/or arrays, but nothing more). Any ideas?

Profile icon
SixBeeps

Wait, how is 5-10MB > 100MB?

Profile icon
AmazingMech2418

@SixBeeps
It isn't, but it is only a twentieth to a tenth of the size and I want to make more than 10-20 server repls.

Profile icon
SixBeeps

@AmazingMech2418
Oh, my bad. I read "only" as "over"