Skip to content
Sign upLog in
← Back to Community

Python 3: Saving data in an uneditable, unreadable file!

Profile icon
Muffinlavania

Introducing Pickle! This module allows you to store data in a .dat file. It saves it in binary so people cant snoop around in your files, but you can load up the file anytime you want, and it will read that binary and convert it back to english! This also means that of course its not 'unreadable', computers can translate it back. Pickle can be useful for games where you want to save your data, but also want an efficient password system! All you have to do is import pickle, then you can save, load , and add data to binary files!

Saving data:

with open(your_variable+".dat",'wb') as save: pickle.dump(any_variable, save, protocol = 2)

The pickle.dump part can be repeated as many times as you want, each one for a different variable you want to put in your newly created file!

Loading data:

global (All variables you will be loading) with open(your_variable+".dat",'rb') as save: any_variable = pickle.load(save)

The pickle.load part can again be repeated multiple times for many different variables

Adding Data!

with open(your_variable+".dat",'ab') as save: pickle.dump(any_variable, save, protocol = 2)

Of course you can do this with many variables, and 'ab' means add binary. You need to already have 'your_variable.dat' saved or the code will error

This is a little bit hard to understand, and a bit hard to remember, but it is probably the most useful module i know!

There are more notes on this in the code itself, and thanks to

@squidcoder
for first showing me this, check out his projects for many examples of pickle.

And thanks to

@medcho
for the idea of making a tutorial on this!

Edit:
If you look at the files, like HI.dat, for some stupid reason repl saves everything BUT the sensitive information in binary. I guess you can still encrypt stuff, but its weird. One good thing tho, is that i can not see any of you guy's files.(I think you guys made some right?) So like

@CodeLongAndPros
said, we are all on different servers, so no one can look at another person's file, which still makes pickle viable for passwords and other things

Voters
Profile icon
NarekGermirlia1
Profile icon
FinnDouglas
Profile icon
jajamama
Profile icon
SleepyBehemoth
Profile icon
vahinsahu
Profile icon
AnshAgarwal1
Profile icon
TsunamiOrSumth
Profile icon
dabs364
Profile icon
R16
Profile icon
wjustin
Comments
hotnewtop
Profile icon
jajamama

Awesome! THANKS!

Profile icon
Muffinlavania

@jajamama
yessir

Profile icon
wjustin

@Muffinlavania
how do you overwrite the file???

Profile icon
Muffinlavania

@wjustin
what do yo mean overwrite it? If you load the variable from the file using the load function, then save that same variable back to it, it is replacing it, aka overwriting it. If you load a variable and APPEND something else to it, it will just add it. Also thanks for all the compliments, idk why so many people like this

Profile icon
wjustin

@Muffinlavania
oh thx for noting

Profile icon
Muffinlavania

@wjustin
what do you mean

Profile icon
wjustin

@Muffinlavania
um i mean thx for telling me that i forgot

Profile icon
Muffinlavania

@wjustin
oh ok yaaaaaaaa

Profile icon
dillonjoshua68

Oh ya... I just learned that it’s not good to unpickle files from untrusted sources.... and this is the least secure way to do deserialization, they said JSON is more secure.... so don’t unpickle anything from untrusted sources.... or the malicious code will enter ur terminal and the cracker will get control . But nice tutorial :)

@Muffinlavania

Profile icon
dillonjoshua68

Also you can do this for pickle.dumps....

import pickle as pikl pikl.dumps(obj, file, protocol = 4,*,fix_imports = True) # obj and file are pretty much the files ur gonna write and save to # protocols available are 1, 2, 3, 4, 5 # 5 is the latest # idk why there’s an asterisk (*) # also it’s better to use the format with fix_imports = True ...in many cases # but check.... ps: fix_imports = True fixes some bugs # check highest and default protocol: print(pikl.HIGHEST_PROTOCOL) print(pikl.DEFAULT_PROTOCOL) # should output numbers

@Muffinlavania

Profile icon
Muffinlavania

@dillonjoshua68
what do you mean unpickle? like load up a file? well yea idk sure, but i guess ill have to trust repl :D

Profile icon
dillonjoshua68

The terms pickle and unpickle are used to make it easier to understand, and unpickling is like deserializing

@Muffinlavania

Profile icon
Muffinlavania

@dillonjoshua68
oh ok, well umm im not going to change it (Because its easier to understand) and it still works, but thanks for telling me about it!

Profile icon
Muffinlavania

@dillonjoshua68
oh and also by the way, I've seen that you use .isdigit() in some of your projects, like the one were it store 30 students information. You can enter Rob1 and it wont error, because .isdigit() only works on a single character. So i would do something like this

x=input("ONLY TYPE LETTERS") #Method 1 for i in x: if i.isalpha():#Excludes symbols and numbers continue else: print("...... i thought i said only letters..") break #I think maybe .isalpha can work on an entire string, i would test it out like this print('Number1'.isalpha()) #If it prints out True it means it works with whole strings so it might be easier then isdigit #Method 2 for i in x: if i.isdigit():#Excludes only digits print("*bruh moment*") else: continue
Profile icon
dillonjoshua68

Thx

@Muffinlavania

Profile icon
dillonjoshua68

Btw, for the markdown use this....
( `) 3 times for long code and write the code in between it or just check the markdown post

@Muffinlavania

Profile icon
Muffinlavania

@dillonjoshua68
yea ik....
lol i might have used the ' instead of ` once or twice, whatever

Profile icon
dillonjoshua68

Btw, for the .isdigit() it checks the whole string or thing you put

@Muffinlavania
same for .isalpha()

Profile icon
Muffinlavania

@dillonjoshua68
oh really? ok...

Profile icon
dillonjoshua68

Yep :)

@Muffinlavania

Profile icon
dillonjoshua68

Oh ya... btw... plz just check to confirm... when you don’t globalize name, and password, and rand if it’ll [email protected] cuz it worked for me...

Profile icon
Muffinlavania

@dillonjoshua68
oh really? cause if you put the save and loading functions in well, a function, you have to globalize it, to edit it... i think

Profile icon
dillonjoshua68

Nvm ur right

@Muffinlavania

Profile icon
dillonjoshua68

The thing is that it won’t cause an error, but it’ll allow you to log in even if the password or username isn’t valid

@Muffinlavania

Profile icon
dillonjoshua68

Also.... we can use lists [ ] instead of the strings " " [email protected]
Cuz u use strings for ur repl.... but I wondered if lists would be fine?

Profile icon
dillonjoshua68

Oh btw... I forked ur repl (ok :) )... and I edited it a bit just to familiarize myself with how it works, and also for good practice with pickle.

@Muffinlavania
...BTW, make sure to try marshal module or json for better security

Profile icon
dillonjoshua68

Also... Data types which can be pickled:
Bool, Int, Float, Complex numbers, (Normal and Unicode) str, Tuples, Lists, Sets, Dicts that obtain, picklable objects

Example script:
import pickle

dogs_dict = {‘ozzy’: 3, ‘roger’: 8, ‘luna’: 5}

to pickle this, you need to specify the name of the file you’ll write it to (file doesn’t have an extension)....

arguments: 1. Name of file, 2. wb (we are writing..w.. in binary mode..b..) or even w+b.

like this:

filename = ‘dogs’
outfile = open(filename, ‘wb’)

use pickle.dump() which takes 2 arguments: 1. The object to pickle, 2. File to save it in.

like this:

pickle.dump(dogs_dict, outfile)
outfile.close() # closing the file

Unpickling files:
using the same open() function.... we pass the second argument as ‘rb’ which is read mode + binary mode. Next use pickle.load() following the same argument format as pickle.dump(). But use new variables

Example:
infile = open(filename, ‘rb’)
new_dict = pickle.load(infile)
infile.close()

to make sure you unpickle it successfully do this:

print(new_dict)
print(new_dict == dogs_dict)
print(type(new_dict))

Compressing pickle files:
If you’re saving a large dataset and your pickled file takes up a lot of space... you may want to compress it. This can be done using bzip2 or gzip. They both compress files. bzip2 is slower, and gzip produces files about 2* larger that bzip2.

Example:
import bz2

bzip2

import pickle

smaller_file = bz2.BZ2File(‘sfile’, ‘w’)
pickle.dump(dogs_dict, smaller_file)

To unpickle objects that were written in python 2.... to python 3 do this:

Example:
infile = open(file_name, ‘rb’)
new_dict = pickle.load(infile, encoding = ‘latin1’)

pass an encoding argument and set equal to latin1.

if it contains numpy arrays use encoding = ‘bytes’

@Muffinlavania

Profile icon
DollarSign

omfg it works

Profile icon
VulcanWM

Thanks for the mention!

Profile icon
FinnDouglas

Where is the .dat file stored? Is it local? I use a chromebook so can it still save?

Profile icon
Muffinlavania

@FinnDouglas
lol im actually replying on a chromebook, it just makes a file in the repl itself, not on your computer or anything.
(Also the saving doesnt actually work with like other people only the owner of the repl)

Profile icon
FinnDouglas

@Muffinlavania
ok thank you!

Profile icon
jajamama

Awesome!Thanks!

Profile icon
JordanDixon1

Screenshot 2020-09-14 at 3.04.44 PM


you can still see the info.

Profile icon
Muffinlavania

@JordanDixon1
yea i think repl just does this for no reason, it works on some other coding websites, i really dont know

Profile icon
Leroy01010

error in greeter

Profile icon
Muffinlavania

@Leroy01010
im there

Profile icon
wjustin

owww mmyy gaddddd it works excellent job

@Muffinlavania
!!!!

Profile icon
wjustin

who else is looking at this repl this is legend!!!

Profile icon
Muffinlavania

@wjustin
:D

Profile icon
DollarSign

I needed this a year ago now

Profile icon
Muffinlavania

@DollarSign
lol thanks you like it?

Profile icon
DollarSign

yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy (my hands hurt now)

@Muffinlavania

Profile icon
Muffinlavania

@DollarSign
woah... is that yes spammed or is that why spammed

Profile icon
DollarSign

i typed y and kept tapping y, my hadns stil rut


@Muffinlavania

Profile icon
ChezCoder

pretty sick but kinda confusing

Profile icon
Muffinlavania

@ChezCoder
yea.... do you think i just explained it weird or it is confusing in general

Profile icon
AmazingMech2418

This actually seems pretty easy to read. In the default file,

�X���HIq�.�X���hiyaq�.�X ���i like cheeseq�.

it is clear that the name is "HI", the password is "hiya", and the random thing is "i like cheese".

Profile icon
Muffinlavania

@AmazingMech2418
yea idk it used to save everything in the symbols... idk

Profile icon
wjustin

@AmazingMech2418
buahhhahahahahahahahahhahahahaa

Profile icon
wjustin

@Muffinlavania
lol

Profile icon
dillonjoshua68

Use json or marshal

@AmazingMech2418

Profile icon
CodeLongAndPros

I don’t think this is “unreadable”, any dev with a hex editor and/or python can read it. I would change it to ununderstanable.

Profile icon
Muffinlavania

@CodeLongAndPros
lol ok i will i guess... that just sounds weird which is why i said unreadable, you cant read it. Of course computers can be ill change it whatever

Profile icon
CodeLongAndPros

@Muffinlavania
Or you could say that it is noneditable.

Profile icon
Muffinlavania