At @Shopify, we've been doing some research into how phones and AR glasses can recognize products at a glance.

This lets you bring up relevant information like price, description, stock quantities, and reviews.

[1/6]

Building a machine learning model to recognize a product requires hundreds of images. We needed a shortcut.

We discovered we could use a realistic 3D model of the product to generate thousands of images from different angles and lighting conditions.

[2/6]

The identification is done by taking a photo or video and sending it to our ML server. By offloading the processing to the cloud, we can power sophisticated ML models on less powerful devices (like AR glasses). The first example in this thread is running in a web browser!

[3/6]

We can even recognize what part of a product you're looking at. This could bring up contextual information about that part or play audio to guide you through a product demonstration.

[4/6]

Why is all this useful? Using computer vision to recognize products allows buyers to get information quickly and enables retail staff to surface meaningful information even if it's their first day on the job.

[5/6]

As more merchants start getting 3D models, they'll be able to unlock new and powerful opportunities like this. From interactive web content, to AR, to ML models, to CG photography, the possibilities are endless. https://www.shopify.ca/blog/3d-models-video

[6/6]