Learn to Code via Tutorials on Repl.it!

← Back to all posts
How to make a package manager (for a custom language!) — Part 3: The (exciting?) conclusion
fuzzyastrocat

This is part 3 of a 3-part series on how to build a package manager. If you haven't gone through part 1 or part 2, check them out here and here.

Part 1 now contains all three parts, so go there.

The task

For this tutorial, we'll be wrapping up the loose ends. We'll do the following:

  • Finish our command line tool: give it update and remove functionality
  • Add to our CDN: Allow users to publish packages to it
  • Add accounts

Setup

The example repls for this section can be found here and here if you'd like to follow along. (They're new repls since we'll be changing both aspects of our package manager.) There's not much setup this time since we're just expanding our existing work.

In contrast to the previous tutorials, we'll be adding lots of stuff in different places. So, I would highly recommend cloning both example repls, and then just following-along with this tutorial. (Use this like a code-guide of how I got to the finished project)

Adding functionality to the command-line tool

First, let's allow our users to do something else than install:

(This replaces the original install check.) Now we need to handle each of these options. We can actually handle all three cases "at the same time". Under packagename := os.Args[2] we add the following:

You can probably reason out what this does: if the package's folder doesn't exist and we're trying to update or remove it, we say that it isn't installed and exit. If it does, and we're trying to install it, we say that it's already installed and exit. There's probably a more elegant way to do this, but it works.

Note: I'm using nested if's for a reason! If the inner if was combined with the outer if, the else if clause would behave differently.

Now let's handle remove's functionality (recall that our install behaved like an update or install, so we've already taken care of the functionality for those two). Under the code we just added, add the following:

Easy! We've now completed task 1: test out ./packager update ..., ./packager install ..., and ./packager remove ... to make sure the changes work. It should give you an error if you try to do something invalid, ie update a package which isn't installed or install a package which is already installed.

Adding to the CDN: Publishing functionality

Okay, now head on over to your CDN. First, we define a function updatePackages so that we can update the package list once a new package is published. If you didn't sync with the example repl, I'd suggest doing that now. updatePackages replaces some code around it, which could be confusing to update and lead to wrong code.

We need to respond to a new kind of request — an HTTP POST request. POST requests have the url and a "body" which contains the important data. We handle our request like so:

This is placed after app.get('/', (req, res) => { ... });. The code here responds to POST requests on /publish/<some_package_name>. It looks at the body, which should be JSON, makes the necessary directory for the package (./packagedata/<some_package_name>) and makes a folder with the new version (./packagedata/<some_package_name>/{data.version}). Finally, we loop through all the keys in the JSON object (which are not the version key) and make a new file for each with the contents of the file equal to the value of that key. We use file.slice(1) since each filename will be prefixed by a _, to differentiate a file named "version" from the actual version number.

To parse the body as JSON, we need to add a new dependency to our project. Below app = express(), we write the following:

This does what you probably think it does — allows the app to parse JSON in a POST request body.

Command-line updates

Now let's make updates on the command-line side so that we can actually publish things. We add || os.Args[1] != "publish" to our initial if-statment check, add "bytes" to our imports, and add the following after packagename := os.Args[2] (check the completed code if you are lost):

So, if you use ./packager publish test 1.0.0, it will look in the directory ./test, take all the files from there and publish them to the CDN (with version 1.0.0). Let's make a directory, ./test, and put a few files inside. Then try ./packager publish test 1.0.0! It should say "Package published successfully", and when you go to your-cdn.repl.co/package/test, it should give you ["1.0.0"] instead of "This package does not exist". Great!

Finishing up

There's one problem with our system so far: anyone can update a package, regardless of who first made it! Let's fix that.

Inside ./test, we add a file .user. Inside that file we put me or whatever name you like. This will be your "user" that gets published with the package to the CDN. Now, on the CDN, we add logic. We replace our original app.post handler with the following:

Now, we need to handle PACKAGE_EXISTS from the command line. After if respstr == "VERSION_EXISTS" { ... }, we put the following:

Great! If you still haven't, you probably want to sync-up with the example repl since there's been so many changes (some of which might not have been covered here).

Now, try this: publish your package "test" again (with the new .user file). Make sure that works without saying "That package already exists!". If it says that the version exists, just change the version (you've already published that version). Once you've successfully published your package with the .user, try changing the .user file and the version and uploading again. This should say "That package already exists!", since you are "a different user".

As always, check the example repls for complete code.

Next steps

That's the end of this tutorial series!

But don't stop here! There's a lot you could do to improve it. The first thing would be to add passwords to your user system. This is a fairly easy addition that will add more (but not complete) security to your packager. Next, I'd add friendlier errors. For instance, try ./packager with no arguments. (Oh no, a Go panic!) This is another easy addition that will make your packager much more refined. Then, there are a few more complex additions you could try:

  • Add a login system through your packager tool itself. Make a command, ./packager login [username] [password] that would store your user data somewhere. Then, rather than having to create a .user file every time you want to make a package, you could just run ./packager init and it would read from the stored user data and put that in a new .user file.
  • Make sure all versions are correct. To do this, you could take npm's route: instead of ./packager publish test 1.5.2, you just do ./packager publish test [major or minor or patch] and it takes care of generating the version for you.
  • Allow folders in a package. Right now we only take the files in the current directory, add a function to recursively traverse the directories too and send that through the JSON.
  • Add an unpublish function. Remove the directory, then call updatePackages().

There's so much more to add — the only limit is your imagination!

Conclusion

I hope you've enjoyed these tutorials (I sure have), and I hope they've helped. If you have any questions, please ask!

Voters
Highwayman
DynamicSquid
fuzzyastrocat
Comments
hotnewtop
CodeLongAndPros

Btw I would Cat the tutorials together for easy reading.

fuzzyastrocat

@CodeLongAndPros Hm ok, I'll do that. Should I just combine them all into the original post and delete these?

CodeLongAndPros

@fuzzyastrocat Perhaps, or just replace these with a link to the big 'un.