Skip to main content

Amazon's Fire Phone recognizes everything around you with 'Firefly'

Amazon's Fire Phone recognizes everything around you with 'Firefly'

Share this story

Amazon's Fire Phone is a showcase for the company's various services, but it's also being positioned as the best possible way to buy products and digital media from Amazon. Jeff Bezos just demoed a new technology called "Firefly" that's capable of recognizing both objects in the real world and content like music and TV shows. Bezos said Firefly can identify up to 100 million items in all. It's triggered by holding the same button that serves as a shortcut to the Fire Phone's camera.

During Bezos' on-stage demonstration, the Fire Phone almost identified household goods like a container of Nutella at near-instantaneous speeds. Firefly lets users easily keep track of anything and everything they've ever scanned and quickly buy those products from Amazon.com. And just like Shazam, the feature can also listen to music and TV shows playing around you and identify them — even down to individual episodes for TV content. In terms of user interface, Firefly seems to be an evolution of the "Flow" feature found in Amazon's existing mobile apps on iOS and Android.

Bezos talked up Firefly's other recognition capabilities, as well. It can intelligently scan real-world items like signs and business cards for phone numbers and other useful information, all while ignoring irrelevant details. And Amazon is allowing third-party developers to tap into the service through an all-new SDK. For example, MyFitnessPal plans to provide nutritional data for food that Firefly recognizes, and Vino will identify and describe wine to help customers choose the right bottle. But make no mistake: Firefly is meant to be Amazon's ultimate tool for making you buy even more from its store.