🕵️♂️ A scavenger hunt with image classification 🖼️
hey hey hey hey hey
I'm back with something I think y'all will find interesting! I built a game based on image classification - you have to get objects from around your house - whatever the app tells you to get, you fetch!
Check out this demo video I made 🎥 - I go over using the app, and the tools I used to make it!
How does this work?
ml5.js lets us run machine learning models, and train them as well, right in our browser. But the reason it exists is to make ML projects, like this mindblowingly simple 🤯
The whole of my game comes down to me writing about
150 lines of JS - that's it. ml5 allows me to pass the webcam's video stream as a parameter, and keeps telling me what it sees. I'm using the mobile-net model for this, which uses the image-net dataset - around 15 million labelled images. Besed on that huge chunk of data, this model predicts what it sees!
Image Net has 1000 different labels through which objects are classified - some of them include things like
Great White Sharks, and
Ostriches - you know, things that can't be found in most homes 😛
So, I manually went through loads of the labels, trying to label things most of us could find.
Clearly, I was a little sleep deprived - and I probably made some mistakes. Let me know if you find some weird labels - I'm @jajoosam on the Discord 💬
Easy machine learning with JS
Machine Learning is pretty complex, but that doesn't mean making projects with it has to be too! I have gained a lot of appreciation for ml5.js - it's an amazing way to integrate ML into your projects.
I learnt all the tech behind making this project from Dan Shiffman's videos on ml5 - and I highly recommend you check them out if you found this interesting! He's a great, super energetic teacher 👨🏫
Super excited to hear what y'all think about this!