Skip to main content

Facebook’s image outage reveals how the company’s AI tags your photos

Facebook’s image outage reveals how the company’s AI tags your photos

/

‘Oh wow, the AI just tagged my profile picture as basic’

Share this story

Everyone knows the bit in The Matrix when Neo achieves digital messiah status and suddenly sees “reality” for what it really is: lines of trailing green code.

Well, thanks to an outage currently affecting Facebook, users of the social network have been given a similar peek behind the digital curtain, with many images on the site now replaced with the tags they’ve been assigned by the company’s machine vision systems.

So if you browse through your uploaded photos, instead of seeing holiday snaps or pictures of food and friends, you’ll be shown text saying things like “image may contain: people smiling, people dancing, wedding and indoor” or just “image may contain: cat.”

In short: this is how your life looks to a computer. This is how Facebook’s AI is judging you. Do you feel ashamed before the all-seeing digital eye?!

The same image tags are showing up on Instagram, and as well as detailing general scene and object descriptions, they also suggest who is in a photo based on Facebook’s facial recognition. (The company has been doing this for photos you’re not tagged in since 2017.)

Facebook has been using machine learning to “read” images in this way since at least April 2016, and the project is a big part of the company’s accessibility efforts. Such tags are used to describe photos and videos to users with sight impairments.

There’s a lot of personal data contained in your photos that AI can extract

What’s not clear is whether Facebook also uses this information to target ads. There’s a lot of data about users’ lives that they might otherwise shield from Facebook contained in these images: whether you’ve got a pet, what your hobbies are, where you like going on holiday, or if you’re really into vintage cars, or swords, or whatever.

Back in 2017, one programmer was motivated by these questions to make a Chrome extension that revealed these tags. As they wrote at the time: “I think a lot of internet users don’t realize the amount of information that is now routinely extracted from photographs.”

Some of the tags are less incredible. “It’s chilling how smart AI systems have become.” — Russell
Some of the tags are less incredible. “It’s chilling how smart AI systems have become.” — Russell

Judging by the reactions on Twitter to this outage, this is certainly new information to a lot of people. We’ve reached out to Facebook to confirm whether they use this data for ad targeting and will update this story accordingly.

Regardless of how this information is being used, though, it’s a fascinating peek behind the scenes at one of the world’s biggest data gathering operations. It also shows to what degree the visual world has become machine readable.

Improvements in deep learning in recent years have truly revolutionized the world of machine vision, and visual content online is often as legible to machines as text. But once something is legible, of course, it becomes easy to store, analyze, and extract data from. It’s only when the system breaks down, like today, that we realize it’s happening at all.

Today’s Storystream

Feed refreshed 44 minutes ago 10 minutes in the clouds

J
Twitter
Jay Peters44 minutes ago
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.


T
Twitter
Tom WarrenTwo hours ago
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.


A
External Link
Andrew J. HawkinsTwo hours ago
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.


Welcome to the new Verge

Revolutionizing the media with blog posts

Nilay PatelSep 13
A
External Link
Andrew J. Hawkins6:11 PM UTC
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.


E
TikTok
Elizabeth Lopatto5:52 PM UTC
Spain’s Transports Urbans de Sabadell has La Bussí.

Once again, the US has fallen behind in transportation — call it the Bussí gap. A hole in our infrastructure, if you will.


J
External Link
Jay Peters4:28 PM UTC
Doing more with less (extravagant holiday parties).

Sundar Pichai addressed employees’ questions about Google’s spending changes at an all-hands this week, according to CNBC.

“Maybe you were planning on hiring six more people but maybe you are going to have to do with four and how are you going to make that happen?” Pichai sent a memo to workers in July about a hiring slowdown.

In the all-hands, Google’s head of finance also asked staff to try not to go “over the top” for holiday parties.


E
External Link
Elizabeth Lopatto4:21 PM UTC
Insiders made the most money off of Helium’s “People’s Network.”

Remember Helium, which was touted by The New York Times in an article entitled “Maybe There’s a Use for Crypto After All?” Not only was the company misleading people about who used it — Salesforce and Lime weren’t using it, despite what Helium said on its site — Helium disproportionately enriched insiders, Forbes reports.


J
Youtube
James Vincent2:50 PM UTC
Nvidia’s latest AI model generates endless 3D models.

Need to fill your video game, VR world, or project render with 3D chaff? Nvidia’s latest AI model could help. Trained on 2D images, it can churn out customizable 3D objects ready to import and tweak.

The model seems rudimentary (the renders aren’t amazing quality and seem limited in their variety), but generative AI models like this are only going to improve, speeding up work for all sorts of creative types.


R
Richard Lawler1:02 PM UTC
Green light.

This week Friday brings the debut of Apple’s other new hardware. We’ve reviewed both the new AirPods Pro and this chonky Apple Watch Ultra, and now you’ll decide if you’re picking them up, or not.

Otherwise, we’re preparing for Netflix’s Tudum event this weekend and slapping Dynamic Island onto Android phones.


The Apple Watch Ultra on a woman’s wrist
Photo by Amelia Holowaty Krales / The Verge
J
External Link
Jess Weatherbed12:31 PM UTC
Japan will fully reopen to tourists in October following two and a half years of travel restrictions.

Good news for folks who have been waiting to book their dream Tokyo vacation: Japan will finally relax Covid border control measures for visa-free travel and individual travelers on October 11th.

Tourists will still need to be vaccinated three times or submit a negative COVID-19 test result ahead of their trip, but can take advantage of the weak yen and a ‘national travel discount’ launching on the same date. Sugoi!


T
External Link
Thomas Ricker11:00 AM UTC
Sony starts selling the Xperia 1 IV with continuous zoom lens.

What does it cost to buy a smartphone that does something no smartphone from Apple, Google, Samsung can? $1,599.99 is Sony’s answer: for a camera lens that can shift its focal length anywhere between 85mm and 125mm.

Here’s Allison’s take on Sony’s continuous-zoom lens when she tested a prototype Xperia 1 IV back in May: 

Sony put a good point-and-shoot zoom in a smartphone. That’s an impressive feat. In practical use, it’s a bit less impressive. It’s essentially two lenses that serve the same function: portrait photography. The fact that there’s optical zoom connecting them doesn’t make them much more versatile.

Still, it is a Sony, and like.no.other.


C
External Link
Corin Faife10:44 AM UTC
If God sees everything, so do these apps.

Some Churches are asking congregants to install so-called “accountability apps” to prevent sinful behavior. A Wired investigation found that they monitor almost everything a user does on their phone, including taking regular screenshots and flagging LGBT search terms.


J
External Link
James Vincent8:41 AM UTC
Shutterstock punts on AI-generated content.

Earlier this week, Getty Images banned the sale of AI-generated content, citing legal concerns about copyright. Now, its biggest rival, Shutterstock, has responded by doing ... absolutely nothing. In a blog post, Shutterstock’s CEO Paul Hennessy says there are “open questions on the copyright, licensing, rights, and ownership of synthetic content and AI-generated art,” but doesn’t announce any policy changes. So, you can keep on selling AI art on Shutterstock, I guess.