Skip to content
← Back to Community
Machine learning neural network
Profile icon
finlay44111

A machine learning neural network i have been working on

Voters
Profile icon
ClementEc
Profile icon
programmeruser
Profile icon
xxpertHacker
Profile icon
Shaggyfullpower
Profile icon
MichaelWang4
Profile icon
ChungusFam
Profile icon
Foster_Bryant
Profile icon
gungurbuz
Profile icon
generationXcode
Profile icon
hazelpy
Comments
hotnewtop
Profile icon
mewant_taco

Dude, very cool!
What is it doing exactly tho?

Profile icon
finlay44111

@mewant_taco the data it is training on is the first 1000 images in the MNIST dataset, it is then being tested in the next 100 images after that. the images are hand written digits. and it is learning to recognise the digits.

Profile icon
mewant_taco

@finlay44111 oh that's pretty cool

Profile icon
staticvoidliam7

would you mind if I recreated this in C#?

Profile icon
finlay44111

@LiamDonohue sure, i would recomend you watch 3blue1brown's series, https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi which shows you the basics and bits of the underlying mathematics surounding it.

Profile icon
finlay44111

@LiamDonohue also i'd like to add the multithreading doesn't seem to work, or well it does, but it runs about 10 time slower than a single thread when use it on my laptop with 8 threads.

Profile icon
staticvoidliam7

this is cool

the brain

Profile icon
generationXcode

Hi, I couldn't go through all the code but did you just train the NN using a genetic algorithm?wow

Profile icon
finlay44111

@generationXcode i used backpropagation, it works by basically finding the derivative of the network at a certain point.

Profile icon
generationXcode

@finlay44111 ohh backprop... I thought u used genetic algorithms

Profile icon
Andi_Chin

nice machine learning big data science neural network evolution program!

Profile icon
ChezCoder

I really wanted to make one of these in python but i kinda just gave up because of the complexity

Profile icon
finlay44111

@ChezCoder i would recommend 3blue1brown's series on it, if you have decent knowledge of high school/college math then you can probably understand and implement the machine learning algorithm used. also python is not a good language, since it is very inefficient.

3blue1brown's video: https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

Profile icon
ChezCoder

@finlay44111 thanks :)

Profile icon
finlay44111
Profile icon
generationXcode

@finlay44111 # python isn't good for ML???
what??!!! no! NO!NONONO! Maybe C is faster, but thats because its compiled... Python is better for machine learning because its data handling capacity is great, its faster and easier to write ML programs because of libraries that have been built for it and its simple to use. It can work on any platform easily. I would go on to say that python is the best for ML. Its also gaining lots of popularity so more tools will be coming up as well for ML. Also there will be more help online if you're facing issues on anything regarding ML with python. Yes, eventually you will have to load your model onto C but dont make the mistake of making the model and experimenting with it on C...
so my advice here is Use C only in production, but build your models in python
But anyways I must admit your project is awesome

Also if you want performance, use Julia programming language

Profile icon
finlay44111

@generationXcode ahh k, so if you want to use libraries (that are written in c or c++) then python is for you. But if you want to do it all from scratch like I did then you need to be using c or c++, you can't write all of the logic in python, it's just too slow.

Profile icon
generationXcode

@finlay44111 not really... The running of the program might be slower than C, but that doesn't mean it isn't fast... I mean for self drive level ML programs you need the speed of C... Writing a neural net from scratch in python is much easier than in C. What do you mean python loic is slow? And I think you may be referring Tensorflow library when you say its written in C, but in reality there are many more machine learning libraries in python that aren't available in C. What I'm saying is that its better to build the model in python and run it in C , if you really want lots of speed...
What do you mean by logic? the math or the AND, OR kind of statements?

Its also suggested by the experts in this field to use python... I mean why doesn't kaggle have a section to write C kernels? Obviously because python is the best language to develop your models in. In C if you haven't properly made your model structure and you want to make some changes to it so that (for example) it stops overfitting on the data, you will have to recompile all your code and wait till it changes to machine code. I mean its much easier to do all of that in python, right? But I agree that for production you should load your model on to C for greater speed. But if you want to develop a model just like that and don't want to reduce its loss then I don't really know...

And again python isn't that slow... My first CNN trained in like a minute and had a really nice accuracy(and it was in python).

Profile icon
finlay44111

@generationXcode it is suggested by the experts to use python, since libraries like tensorflow are available (which is written in c++ and CUDA), if you want to write all of the logic from yourself using basic mathematical functions, in order to actually gain an understanding of what is happening, then c++ is the only way to get the performance required to train these programs.

If you only want to use the APIs these libraries give you and claim that you have build a neural network, sure, go code it all in python and import tensorflow. but if you want to see how it works, in order to be able to expand on it, you need to use a lower level, more efficient language.

I had a look at your repl, https://repl.it/@generationXcode/negative, and it seems like you are just importing tensorflow, a c++ library to do all of the logic, and you are just using the API. this is probably why it didn't take too long to train.

Profile icon
generationXcode

@finlay44111 I don't just use API to build neural nets and even with them, the neural nets formed do exactly what yours does, there is no difference in the math.
And how many times do I have to say that python isn't that slow, its quite fast.Maybe its not as fast as C but that doesn't mean its slow.
Its also very wise to write a NN from scratch in python because the language is much more simple. The math needed can be easily expressed unlike in C.
And I don't really do ML in repl.it. I have notebooks in which I do them(they are TPU accelerated in the cloud)

And please dont forget to read this point - you should develop your models in python and run them in C If you want to know why please read through all of my comments here.
Also, you built your model from scratch, yes, but you also used a library for vectors.
And another thing - I've seen the python tensorflow library and even contributed to it - its written in python.

Profile icon
generationXcode

@finlay44111 This will take forever and ever to be honest, why dont we ask an expert in this field like francois chollet, jeremy howard, rachel thomas?

Profile icon
finlay44111

@generationXcode https://www.wired.com/2015/11/google-open-sources-its-artificial-intelligence-engine/

Google built the underlying TensorFlow software with the C++ programming language.

Profile icon
generationXcode

@finlay44111 I meant the keras functional api within tensorflow.Its written in python. If you don't beleive me, check the github repo for tensorflow for python.
I beleive TF 2.0 is mainly keras now.

Profile icon
finlay44111

@generationXcode yes that is exactly right the API is written in python. all of the logic for the ai and machine language is in c++.

Profile icon
[deleted]

pretty cool. might try to make this in python.

Profile icon
finlay44111

@Thecrowbar1234 i wouldn't recommend python, this requires a lot of computing power and python is not efficient enough to run it well.

Profile icon
[deleted]
Profile icon
generationXcode

@finlay44111 training the MNIST dataset doesnt require that much computing power though...

Profile icon
finlay44111

@generationXcode nope, but with the limited power that you get in repl, you can train it pretty fast if you use an efficient language.

Profile icon
generationXcode

@finlay44111 But you have to wait till its compiled...

Profile icon
finlay44111

@generationXcode which is exacly the reason its faster than python, it takes about a second to compile, and then it is already in machine instructions. with python it is interpreted on the go, so it has to expend extra cpu cycles in translating from the python into the machine language.

which means that with c++ you only go over each line once to generate machine instructions, but with python you will have to interpret each individual instruction however many times that instruction is run.

Profile icon
generationXcode

@finlay44111 Ok I agree with you on this one, you're right - machine code runs faster

Profile icon
Giothecoder

Nice!