Skip to main content

Facebook is convinced this man is his mother

Facebook is convinced this man is his mother


Even the best facial recognition systems can be fooled by family members. Can that bug become a feature?

Share this story

Fred Benenson got an email from Facebook recently, informing him that it had detected his face in a number of photos and asking if he would like to be tagged in them. When he clicked through the link, he was amused to find eleven black and white headshots of his mother from her college days. The social network’s DeepFace algorithms detect and identify faces using a neural network with more than 120 million parameters, an ever-improving machine learning process the company says is closely approaching human-level performance. Facebook says DeepFace can determine if two photographs are of the same person 97.25 percent of the time, but there are still plenty of images that can trip up their most powerful machines, especially when it’s a family affair.

"We all have genetic traits that are translated into how we look. If a person closely resembles one of their parents, a photo of the young mother or father could correlate and confuse the algorithm," says David Tunnell, the chief technology officer at NXT-ID, which specializes in three-dimensional facial recognition. In Facebook's case, confusing two family members was a bug, but one that points toward other potential uses for facial recognition, as the process also works in reverse. "We have done research that shows you can use facial recognition to identify a person’s ethnicity, region of origin, and family affiliation," Tunnell says. With good data from my parents, siblings, or cousins, he says it might be possible to identify me even if the system had no actual images of me to work from.

fred benenson and mom

Fred and his mother Donna

Is that creepy, or cool? Mostly it depends on the context. "I did find it oddly compelling," Benenson says in an email to The Verge. "Here was a real life example of a false positive (when an algorithm incorrectly predicts an observation as being something its not — in this case it thought my mom was me) with personal implications." Benenson, who is a data scientist at Kickstarter, says he generally isn’t too troubled by the idea that photographs of our faces are constantly being scanned and analyzed for personal identification. "In general I think people tend to overstate the nefarious things that Facebook is doing with this kind of technology, but it certainly opens the door to larger and more difficult questions."

When it comes to more serious applications, for example the way the Chicago Police are now using facial recognition, Benenson sees the potential for trouble. "What about the cases where this algorithm isn't used for fun photo tagging?" he asked rhetorically. "What if another false positive leads to someone being implicated for something they didn't do? Facebook is a publicly traded company that uses petabytes of our personal data as their business model — data that we offer to them, but at what cost?"

deepface stallone

Facebook's DeepFace system analyzing images of Sylvester Stallone

Tunnell says that the ability to identify people based on family members, even as they age, is frightening, but that the possibility to use this capability for good is equally exciting. "I did a study with my own daughter, where I scanned her entire head every day over the course of several years. What we learned is that, given the right data set, these technologies can identify someone two, four, six years down the road. So that could be critical in helping to locate someone who hasn’t been seen by their own relatives, and who has through substantial physical changes."

In a 2012 US Senate hearing on privacy, Al Franken asked Facebook point blank if they would ever sell their database of facial identities to other companies. Facebook’s Rob Sherman said there was no telling how the company might use the data in the future. Users can always remove the ability to be tagged in a photo, but not the ability to be scanned. "It’s valuable to think hard about the fairness of the tradeoffs we're making with our personal data," says Benenson. "This isn't a calculus we've had enough time to fully understand from a societal perspective. It may be years before we actually know the consequences of feeding so much personal content to such a powerful company."

For the time being, however, Fred and his mother Donna simply found the misidentification to be touching, especially since Fred works in data science. "I can't get over the fact that, for the first time in 30 years that I'm told that I look like Fred, it happens by way of an algorithm," she wrote in an email to The Verge. "What could be more wild or fitting!"

Today’s Storystream

Feed refreshed 48 minutes ago Not just you

External Link
Emma Roth48 minutes ago
We might not get another Apple event this year.

While Apple was initially expected to hold an event to launch its rumored M2-equipped Macs and iPads in October, Bloomberg’s Mark Gurman predicts Apple will announce its new devices in a series of press releases, website updates, and media briefings instead.

I know that it probably takes a lot of work to put these polished events together, but if Apple does pass on it this year, I will kind of miss vibing to the livestream’s music and seeing all the new products get presented.

External Link
Emma RothSep 24
California Governor Gavin Newsom vetoes the state’s “BitLicense” law.

The bill, called the Digital Financial Assets Law, would establish a regulatory framework for companies that transact with cryptocurrency in the state, similar to New York’s BitLicense system. In a statement, Newsom says it’s “premature to lock a licensing structure” and that implementing such a program is a “costly undertaking:”

A more flexible approach is needed to ensure regulatory oversight can keep up with rapidly evolving technology and use cases, and is tailored with the proper tools to address trends and mitigate consumer harm.

The Verge
Andrew WebsterSep 24
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.

Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.

Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.

External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.

External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.