Google just announced a face unlock feature for the upcoming Pixel 4, which claims to be just as accurate and fast as the iPhone’s Face ID. There have been hardware and code leaks that suggested this feature was coming to the Pixel 4, but there was also another big hint: Google has been publicly gathering data to improve it.
As first revealed by ZDNet and Android Police, Google employees have been roving the streets of American cities, offering $5 gift certificates in exchange for a facial scan. Reached by The Verge, Google confirmed that it has been conducting what it calls “field research” to collect face-scanning data in order to improve its algorithms and thereby improve the Pixel 4’s accuracy.
Google needs a wide array of face data
A company spokesperson confirmed that the purpose of the scans is to ensure that the Pixel 4 works with a diverse set of faces. Biometric features — including facial recognition — have a bad history of gender and racial bias. Amazon, in particular, has come under fire because of racial bias in its face-matching algorithms.
Google wants to avoid that with the Pixel 4. “Our goal is to build the feature with robust security and performance. We’re also building it with inclusiveness in mind, so as many people as possible can benefit,” the spokesperson said in an email.
If the Pixel 4 is going to avoid that bias, Google needs to train its new algorithm on a wide array of faces — and just using employees’ faces isn’t going to work. So Google’s fix is to approach people on the streets, pay them, and get affirmative consent for each scan.
Apple, for what it’s worth, achieved the same thing in a similar way; it just probably didn’t send teams of people out into the world to do it. Here’s how Cynthia Hogan, Apple’s VP for public policy and government affairs for the Americas, described Apple’s process in 2017:
The accessibility of the product to people of diverse races and ethnicities was very important to us. Face ID uses facial matching neural networks that we developed using over a billion images, including IR and depth images collected in studies conducted with the participants’ informed consent. We worked with participants from around the world to include a representative group of people accounting for gender, age, ethnicity, and other factors. We augmented the studies as needed to provide a high degree of accuracy for a diverse range of users. In addition, a neural network that is trained to spot and resist spoofing defends against attempts to unlock your phone with photos or masks.
Google is collecting infrared, color, and depth data from each face along with time, ambient light level, and some related “task” information like picking up the phone from the table. The company initially collected location information as well, but it tells me it doesn’t need that info, so it will cease collecting it and will delete it.
Those data types give us a sense of how Google’s face unlock will work — it will, indeed, create a full depth map of your face to ensure accuracy and security — and it should work in the dark, thanks to the infrared camera.
Google will keep the data for 18 months, but people can ask for it to be deleted sooner
Here’s how Google is handling the data it’s collecting, according to their spokesperson: “Although face samples inherently can’t be anonymous, each participant is assigned an abstract identity number. We separately keep each participant’s email address, in order to remove data upon request.” The last part is important: anybody who participated in Google’s field research can request to have their face data deleted.
The face data will be kept for 18 months (though, originally, the first consent forms for the field research said five years). Google says the faces are never associated with a Google ID and that the face samples are “encrypted and access restricted.”
To be clear, when the Pixel 4 ships, the data it uses to unlock will not be uploaded to Google’s servers. In the blog post announcing the feature, Google writes that:
Security and privacy are core principles for Pixel. Face unlock uses facial recognition technology that is processed on your device, so that image data never leaves your phone. The images used for face unlock are never saved or shared with other Google services. To protect your privacy and security, your face data is securely stored in Pixel’s Titan M security chip. Similarly, Soli sensor data is also processed on your phone, and it’s never saved or shared with other Google services.
I guess if you’re going to collect facial recognition data, there are creepier ways to go about it than walking up to somebody, giving them a consent form, and straight up saying you’re from Google and that the data will be used for a “future Google product.” If nothing else, Google’s efforts to gather face data at this scale confirm that it is serious about making a face unlock feature that is on par or better than what you can get on modern iPhones.