Skip to main content

Amazon’s Echo Look is a minefield of AI and privacy concerns

Amazon’s Echo Look is a minefield of AI and privacy concerns


What does Amazon want to learn from pictures of its customers? The company won’t say

Share this story


Computer scientist Andrew Ng once described the power of contemporary AI as the ability to automate any mental task that takes a human “less than one second of thought.” It’s a rule of thumb that’s worth remembering when you think about Amazon’s new Echo Look — a smart camera with a built-in AI assistant. Amazon says the Echo Look will help users dress and give them fashion advice, but what other judgements could it make?

Think about it. If you pass a stranger in the street, how much information do you get with a second’s glance? You can probably make some decent estimates about their height, weight, age, race, and gender. If they were far enough along in pregnancy, you’d know. And you could take a stab at other, more contentious questions, like: are they rich or poor? Friendly or closed-off? Are they having a good day? If it takes a second for you to answer these questions, then Amazon’s AI could take a stab at it as well. And, given enough data, it can offer far, far more accurate answers.

As academic and sociologist Zeynep Tufekci put it on Twitter: “Machine learning algorithms can do so much with regular full length pictures of you. They can infer private things you did not disclose [...] All this to sell you more clothes. We are selling out to surveillance capitalism that can quickly evolve into authoritarianism for so cheap.” (The whole thread from Tufecki is definitely worth a read.)

This might seem overly speculative or alarmist to some, but Amazon isn’t offering much reassurance about what they plan to do with data gathered from the Echo Look. representative for the company told The Verge, that at this point, the Look will only use machine learning to analyze users’ fashion choices, but when asked if this might change in the future, they said they “can’t speculate” on the topic. The rep stressed that users can delete videos and photos taken by the Look at any time, but until they do, it seems this content will be stored indefinitely on Amazon’s servers.

This non-denial means the Echo Look could, in the future, provide Amazon with the resource every AI company craves: data. And full-length photos of people taken regularly in the same location would be a particularly valuable dataset — even more so if you combine this information with everything else Amazon knows about its customers (their shopping habits, for one). But when asked whether the company would ever combine these two datasets, an Amazon rep only gave the same, canned answer: “Can’t speculate.”

The Echo Look’s primary function is to give users style advice, and let them compare looks.
The Echo Look’s primary function is to give users style advice, and let them compare looks.
Image: Amazon

The company did, though, say it wouldn’t share any personal information gleaned from the Echo Look to “advertisers or to third-party sites that display our interest-based ads.” That means Amazon could still use data from the Look to target ads at you itself, but at least third parties won’t.

Right now, the Echo Look is halfway between prototype and full-on product. As is often the case with Amazon’s hardware efforts, the company seems most interested in just getting a product out there and gauging public reaction, rather than finessing every detail. The company is giving no indication of when the Echo Look will actually be available, and it’s currently only being sold “by invitation only.” All this means that Amazon itself probably isn’t yet sure what exactly it will do with the data the device collects. But, if the company refuses to give any more detail, it’s understandable to fear the worst.

Update April 28th, 12:10AM ET: The article has been updated to clarify that at this point in time, machine learning analysis will be used by the Echo Look only on clothing and questions of style.