“Your phone’s front camera is always securely looking for your face, even if you don’t touch it or raise to wake it.” That’s how Qualcomm Technologies vice president of product management Judd Heape introduced the company’s new always-on camera capabilities in the Snapdragon 8 Gen 1 processor set to arrive in top-shelf Android phones early next year.
Depending on who you are, that statement can either be exciting or terrifying. For Qualcomm, it thinks this new feature will enable new use cases, like being able to wake and unlock your phone without having to pick it up or have it instantly lock when it no longer sees your face.
But for those of us with any sense of how modern technology is used to violate our privacy, a camera on our phone that’s always capturing images even when we’re not using it sounds like the stuff of nightmares and has a cost to our privacy that far outweighs any potential convenience benefits.
Qualcomm’s main pitch for this feature is for unlocking your phone any time you glance at it, even if it’s just sitting on a table or propped up on a stand. You don’t need to pick it up or tap the screen or say a voice command — it just unlocks when it sees your face. I can see this being useful if your hands are messy or otherwise occupied (in its presentation, Qualcomm used the example of using it while cooking a recipe to check the next steps). Maybe you’ve got your phone mounted in your car, and you can just glance over at it to see driving directions without having to take your hands off the steering wheel or leave the screen on the entire time.
The company is also spinning it as making your phone more secure by automatically locking the phone when it no longer sees your face or detects someone looking over your shoulder and snooping on your group chat. It can also suppress private information or notifications from popping up if you’re looking at the phone with someone else. Basically, if you’re not looking at it, your phone is locked; if it can see you, it will be unlocked. If it can see you and someone else, it can automatically lock the phone or hide private information or notifications from displaying on the screen.
But while those features may sound neat and perhaps even convenient, I’m not convinced that having an always-on camera is worth the tradeoff in privacy concerns.
The always-on camera features are discussed at hour three of Qualcomm’s four-hour presentation introducing the Snapdragon Gen 1 system-on-chip.
Qualcomm is framing the always-on camera as similar to the always-on microphones that have been in our phones for years. Those are used to listen for voice commands like “Hey Siri” or “Hey Google” (or lol, “Hi Bixby”) and then wake up the phone and provide a response, all without you having to touch or pick up the phone. But the difference is that they are listening for specific wake words and are often limited with what they can do until you do actually pick up your phone and unlock it.
It feels a bit different when it’s a camera that’s always scanning for a likeness.
It’s true that smart home products already have features like this. Google’s Nest Hub Max uses its camera to recognize your face when you walk up to it and greet you with personal information like your calendar. Home security cameras and video doorbells are constantly on, looking for activity or even specific faces. But those devices are in your home, not always carried with you everywhere you go, and generally don’t have your most private information stored on them, like your phone does. They also frequently have features like physical shutters to block the camera or intelligent modes to disable recording when you’re home and only resume it when you aren’t. It’s hard to imagine any phone manufacturer putting a physical shutter on the front of their slim and sleek flagship smartphone.
Lastly, there have been many reports of security breaches and social engineering hacks to enable smart home cameras when they aren’t supposed to be on and then send that feed to remote servers, all without the knowledge of the homeowner. Modern smartphone operating systems now do a good job of telling you when an app is accessing your camera or microphone while you’re using the device, but it’s not clear how they’d be able to inform you of a rogue app tapping into the always-on camera.
In his presentation, Heape said that “always-on camera data never leaves the secure sensing hub while it’s looking for faces,” implying that the data isn’t sent to the cloud and that apps on the phone won’t be able to access it.
In a follow up conversation after this article was originally published, Heape further clarified how the system works on a technical level. The always-on camera is part of a chain that includes a new low-power image signal processor (ISP) and a machine learning step. All three stages — the camera, the ISP, and the machine learning — are “hardened”, according to Heape, he says that it isn’t possible to inject code into the chain.
The always-on camera is limited to a VGA (640 x 480 pixel) resolution, even though it uses the phone’s existing front-facing camera, and the rest of the phone’s systems — the main processor, cellular radios, Wi-Fi, Bluetooth, etc — are turned off. Heape said this low-power ISP “cannot capture photos or video” and “no image or video is stored” in this process, it only scans to detect if there is a face or not. If the machine learning algorithm determines a face is detected, the rest of the phone’s systems are awakened to authenticate the user and unlock the phone or deny access to it. “It is impossible in the low power state to get an image out of it,” says Heape.
The system does not have a way to inform you that its always-on camera is running, like other facial recognition systems use. Heape said that an OEM could choose to add an indicator such as an LED light if they wanted.
Another Qualcomm Technologies vice president of product management Ziad Asghar told my colleague Chaim Gartenberg that users will also be able to disable the always-on camera feature or possibly even select which features they want to use and which they don’t. “The consumer has the choice to be able to pick and choose as to what is enabled and what’s not enabled,” he said.
It’s also possible that the smartphone manufacturers using the Snapdragon 8 Gen 1 won’t even enable this feature at a hardware level. Since Qualcomm doesn’t actually make the smartphones its chips go in (outside of one-off novelties that aren’t widely purchased), companies like Samsung, OnePlus, and Xiaomi can customize which features are enabled on their phones and which aren’t. Heape said that an OEM can request a Snapdragon chip with the feature disabled at the hardware level. Some of those companies already bypass Qualcomm’s image processing components in favor of their own solutions — it’s not hard to see them just skipping the criticisms of privacy concerns and forgoing this feature as well.
But even if it’s not found in every phone next year, the mere presence of the feature means that it will be used by someone at some point. It sets a precedent that is unsettling and uncomfortable; Qualcomm may be the first with this capability, but it won’t be long before other companies add it in the race to keep up.
Ultimately, it comes down to a level of trust — do you trust that Qualcomm has set up the system in a way that prevents the always-on camera from being used for other purposes than intended? Do you trust that the OEM using Qualcomm’s chips won’t do things to interfere with the system, either for their own profit or to satisfy the demands of a government entity?
Even if you do have that trust, there’s a certain level of comfort with an always-on camera on your most personal device that goes beyond where we are currently.
Maybe we’ll just start having to put tape on our smartphone cameras like we already do with laptop webcams.
Update, 5:15PM ET, December 2nd, 2021: After this article was published, Qualcomm vice president of product management Judd Heape gave further information about how the always-on camera system works on a technical level. The company also clarified that the system does not identify the faces it detects; that is handled by the phone’s existing facial identification authentication systems. The article has been updated with this new information and the headline has been changed.