Facial recognition software has become increasingly common in recent years. Facebook uses it to tag your photos; the FBI has a massive facial recognition database spanning hundreds of millions of images; and in New York, there are even plans to add smart, facial recognition surveillance cameras to every bridge and tunnel. But while these systems seem inescapable, the technology that underpins them is far from infallible. In fact, it can be beat with a pair of psychedelic-looking glasses that cost just $0.22.
Researchers from Carnegie Mellon University have shown that specially designed spectacle frames can fool even state-of-the-art facial recognition software. Not only can the glasses make the wearer essentially disappear to such automated systems, it can even trick them into thinking you’re someone else. By tweaking the patterns printed on the glasses, scientists were able to assume one another’s identities or make the software think they were looking at celebrities. (In the image at the top of the article, you can see the researchers wearing the glasses in the top row of pictures, and the identity they copied in the bottom row.)
It takes a lot less to trick machines than trick people
The glasses work because they exploit the way machines understand faces. Facial recognition software is often powered by deep learning; systems that crunch through large amounts of data to sift out recurring patterns. In terms of recognizing faces, this could mean measuring the distance between an individual’s pupils, for example, or looking at the slant of their eyebrows or nostrils.
But compared to human comprehension, this analysis takes place at an abstract level. Computer systems don’t understand faces in the way we do; they’re simply looking for patterns of pixels. If you know what patterns are being looked for, you can easily trick machine vision systems into seeing animals, people, and objects in what are just abstract patterns. This is exactly what the researchers from Carnegie Mellon did.
Pass yourself off as Milla Jovovich with 87.87 percent accuracy
First they worked out the patterns associated with specific faces, and then they printed them onto a set of wide-rimed glasses (so as to better occupy more of the frame; about 6.5 percent of the available pixels in the end). In 100 percent of their tests, the researchers were able to use the glasses to effectively blind facial recognition systems to their identities. Results were more mixed when it came to impersonating other individuals. A 41-year-old white male researcher was able to pass himself off as actress Milla Jovovich to facial recognition systems with 87.87 percent accuracy, but a 24-year-old South Asian female researcher was only able to convince a computer she was 79-year-old Colin Powell 16 percent of the time.
There are obvious limitations to this system. For a start, although the glasses are a more subtle disguise than wearing a mask or using, say, CV Dazzle (a form of bold makeup that also confuses facial recognition systems), they’re hardly inconspicuous. There’s also the problem of how and where the image is taken: the researchers didn’t test how well the glasses worked at a distance, for example, or in different lighting conditions. As ever, tests in a lab don't always equal workable results in the real world. And besides, imagine having to wear these glasses to every party you go to in future, just to avoid getting tagged by Facebook's algorithms.