Skip to main content

Amazon’s Echo Look is a minefield of AI and privacy concerns

Amazon’s Echo Look is a minefield of AI and privacy concerns


What does Amazon want to learn from pictures of its customers? The company won’t say

Share this story


Computer scientist Andrew Ng once described the power of contemporary AI as the ability to automate any mental task that takes a human “less than one second of thought.” It’s a rule of thumb that’s worth remembering when you think about Amazon’s new Echo Look — a smart camera with a built-in AI assistant. Amazon says the Echo Look will help users dress and give them fashion advice, but what other judgements could it make?

Think about it. If you pass a stranger in the street, how much information do you get with a second’s glance? You can probably make some decent estimates about their height, weight, age, race, and gender. If they were far enough along in pregnancy, you’d know. And you could take a stab at other, more contentious questions, like: are they rich or poor? Friendly or closed-off? Are they having a good day? If it takes a second for you to answer these questions, then Amazon’s AI could take a stab at it as well. And, given enough data, it can offer far, far more accurate answers.

As academic and sociologist Zeynep Tufekci put it on Twitter: “Machine learning algorithms can do so much with regular full length pictures of you. They can infer private things you did not disclose [...] All this to sell you more clothes. We are selling out to surveillance capitalism that can quickly evolve into authoritarianism for so cheap.” (The whole thread from Tufecki is definitely worth a read.)

This might seem overly speculative or alarmist to some, but Amazon isn’t offering much reassurance about what they plan to do with data gathered from the Echo Look. representative for the company told The Verge, that at this point, the Look will only use machine learning to analyze users’ fashion choices, but when asked if this might change in the future, they said they “can’t speculate” on the topic. The rep stressed that users can delete videos and photos taken by the Look at any time, but until they do, it seems this content will be stored indefinitely on Amazon’s servers.

This non-denial means the Echo Look could, in the future, provide Amazon with the resource every AI company craves: data. And full-length photos of people taken regularly in the same location would be a particularly valuable dataset — even more so if you combine this information with everything else Amazon knows about its customers (their shopping habits, for one). But when asked whether the company would ever combine these two datasets, an Amazon rep only gave the same, canned answer: “Can’t speculate.”

The Echo Look’s primary function is to give users style advice, and let them compare looks.
The Echo Look’s primary function is to give users style advice, and let them compare looks.
Image: Amazon

The company did, though, say it wouldn’t share any personal information gleaned from the Echo Look to “advertisers or to third-party sites that display our interest-based ads.” That means Amazon could still use data from the Look to target ads at you itself, but at least third parties won’t.

Right now, the Echo Look is halfway between prototype and full-on product. As is often the case with Amazon’s hardware efforts, the company seems most interested in just getting a product out there and gauging public reaction, rather than finessing every detail. The company is giving no indication of when the Echo Look will actually be available, and it’s currently only being sold “by invitation only.” All this means that Amazon itself probably isn’t yet sure what exactly it will do with the data the device collects. But, if the company refuses to give any more detail, it’s understandable to fear the worst.

Update April 28th, 12:10AM ET: The article has been updated to clarify that at this point in time, machine learning analysis will be used by the Echo Look only on clothing and questions of style.

Today’s Storystream

Feed refreshed Sep 24 Striking out

External Link
Emma RothSep 24
California Governor Gavin Newsom vetoes the state’s “BitLicense” law.

The bill, called the Digital Financial Assets Law, would establish a regulatory framework for companies that transact with cryptocurrency in the state, similar to New York’s BitLicense system. In a statement, Newsom says it’s “premature to lock a licensing structure” and that implementing such a program is a “costly undertaking:”

A more flexible approach is needed to ensure regulatory oversight can keep up with rapidly evolving technology and use cases, and is tailored with the proper tools to address trends and mitigate consumer harm.

Andrew WebsterSep 24
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.

The Verge
Andrew WebsterSep 24
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.

Andrew WebsterSep 24
Looking for something to do this weekend?

Why not hang out on the couch playing video games and watching TV. It’s a good time for it, with intriguing recent releases like Return to Monkey Island, Session: Skate Sim, and the Star Wars spinoff Andor. Or you could check out some of the new anime on Netflix, including Thermae Romae Novae (pictured below), which is my personal favorite time-traveling story about bathing.

A screenshot from the Netflix anime Thermae Romae Novae.
Thermae Romae Novae.
Image: Netflix
Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.

Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.

External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.

External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.