🕵️♂️ A scavenger hunt with image classification 🖼️
hey hey hey hey hey
I'm back with something I think y'all will find interesting! I built a game based on image classification - you have to get objects from around your house - whatever the app tells you to get, you fetch!
Check out this demo video I made 🎥 - I go over using the app, and the tools I used to make it!
This file cannot be displayed: https://storage.googleapis.com/replit/images/1548248878277_61c6e20eadf9da4da8e8967685a61f08.gi
ml5.js lets us run machine learning models, and train them as well, right in our browser. But the reason it exists is to make ML projects, like this mindblowingly simple 🤯
The whole of my game comes down to me writing about
150 lines of JS - that's it. ml5 allows me to pass the webcam's video stream as a parameter, and keeps telling me what it sees. I'm using the mobile-net model for this, which uses the image-net dataset - around 15 million labelled images. Besed on that huge chunk of data, this model predicts what it sees!
Image Net has 1000 different labels through which objects are classified - some of them include things like
Great White Sharks, and
Ostriches - you know, things that can't be found in most homes 😛
So, I manually went through loads of the labels, trying to label things most of us could find.
This file cannot be displayed: https://storage.googleapis.com/replit/images/1548250013223_d4e633164513cafa87953e8e9e755ae1.pn
Clearly, I was a little sleep deprived - and I probably made some mistakes. Let me know if you find some weird labels - I'm @jajoosam on the Discord 💬
Machine Learning is pretty complex, but that doesn't mean making projects with it has to be too! I have gained a lot of appreciation for ml5.js - it's an amazing way to integrate ML into your projects.
I learnt all the tech behind making this project from Dan Shiffman's videos on ml5 - and I highly recommend you check them out if you found this interesting! He's a great, super energetic teacher 👨🏫
This file cannot be displayed: https://storage.googleapis.com/replit/images/1548250591304_1f59b29dfc20b7f8c93d6323da1a4c72.gi
Super excited to hear what y'all think about this!