Using MobileNet a light Javascript machine learning library for image classification I was able to classify the live video feed from the web cam in real-time. The script will look for an image in Flicker that matches the description. We are able to see some hats, mugs, cups and color that were displayed to the camera.
Using ML5 and its image classification model and approach, I implemented this little experiment.
The sketch is recognizing and labeling images from the live input camera in your laptop. At the same time is sending queries with these labels to an API that responses with URL if multiple images. You can clearly see how the background image change accordingly with the objects you out in front of the camera.
Let’s imagine this feature implemented in a museum where you are able to make your own questions visually. You are looking for a painting with a pose, or a particular object or a particular feature and you are able to request this with images.
