front end dev, back end dev, creative technology
NRF 2019
Building a retail experience for the National Retail Federation's annual show

The Future of Retail Ops

Working with Google's retail industry team to build an experience for their larger presence at the National Retail Federation's 2019 show in New York City, we prototyped and built a machine learning-powered smart mirror to demonstrate the ease of use for large retailers and brands to incorporate Google's industry-leading machine learning and data analysis solutions. Starting with knowing the need to show how the experience can benefit both the customer and the employee, we built a smart mirror system that delivers real-time product reviews and recommendations while an employee dashboard used the same information to display up-to-date analytics about the products interacted with and traffic visualizations.

Google at NRF 2019 by Marcus Guttenplan for Sparks

Initial prototypes were built with a mixture of tfjs and AutoML, where we used custom models to track emotion, extract faces, classify objects, and capture poses. As the concept was developed further, we switched to prototyping custom models using AutoML, creating a tool that allowed us to classify specific products. Due to latency concerns, a local copy of the data is stored in a SQL database, while also being sent to the cloud to run through an analytics pipeline using BigQuery and Dataflow to determine and send recommendations.

Moving into production, we came up with a method to ensure correct personalization across users by using facial detection to create "sessions." Because the mirror itself sets up a 1:1 interaction, we were able to avoid the development overhead of tracking personalization for many users at once.

Google at NRF 2019 by Marcus Guttenplan for Sparks Google at NRF 2019 by Marcus Guttenplan for Sparks
Client
Google

Skills
js
tfjs
python
ML

Team
Thomas Ryun
Ryan Barnish
Brian Murphy