Lighthouse is an Andy Rubin-backed smart security camera that identifies people and pets

The team at Lighthouse, a startup out of Android co-founder Andy Rubin’s Playground accelerator, doesn’t see its new hardware product as a home security camera. Instead, they see it as an “interactive assistant.” But Lighthouse, at least at first, will definitely be perceived as another new entrant in the smart camera market.

The device, unveiled for the first time today, sits in the home just like a Nest Cam to monitor what’s going on indoors. That’s where the overlap with Nest ends, however. Lighthouse incorporates deep learning and 3D-sensing technology to determine who is in the home, where they are inside, and if that’s a normal occurrence or not. The camera pairs with a companion iOS / Android app over Wi-Fi, so users can determine remotely whether an intruder is in their house. More innocuously, Lighthouse can also determine whether a dog’s been walked and send alerts when kids get home. Those pings of information are certainly the most novel thing about Lighthouse, but let’s start by discussing the hardware.

The actual Lighthouse rig includes a RGB camera, 3D sensor, speaker, microphone, and siren. The camera captures video in 1080p and has automatic night vision. These more intense hardware components help build a smarter security camera that could serve as a 24/7 assistant to the home, as Lighthouse puts it. While the camera is stationary, it learns who to expect in your home on a daily basis — including pets — and who is out of place.

Photo: Lighthouse

During a demo at The Verge’s office in New York, Lighthouse marketing chief Jessica Gilmartin showed me how you can pull up a voice recognition system in the company’s app that resembles Siri or Google Assistant to ask questions and issue commands. For instance, Gilmartin told the app to tell her when John (her husband) got home. The app would then send a push notification when it next saw her husband.

You can also ask questions like, “What did the kids do while I was out today?” The app will then subsequently pull up the last video footage it logged of the kids. For this demo, the company’s technology wasn’t relying on facial recognition, but rather using 3D sensing with an RGB-D camera, relying on deep learning techniques to improve pattern and object recognition over time. Lighthouse fed its system over a million training examples to get it to where it is now, although it does plan to incorporate facial recognition before launch.

When setting the camera up, users just have to identify the person the camera sees. The first time it sees your daughter, for example, you’ll get a notification saying it doesn’t recognize someone in the home. You can then assign a name for your daughter, thereby telling Lighthouse to stand down going forward. In contrast, Nest uses motion detection and computer vision algorithms to send pings while other companies like Netatmo rely on standard facial recognition. You can also talk to whoever is in the home over the built-in speaker. Audio goes both ways.

Photo: Lighthouse

Lighthouse is definitely intriguing, at least from the demo I saw. But it raises some privacy concerns. The company says it deploys bank-level encryption and strong authentication protocols to keep data secure. User information is only viewable by the users themselves, although if there was a legal law enforcement request, Lighthouse would be required to turn over whatever data they had. However, information is only kept for 30 days before it’s deleted permanently.

For now, I’m interested in trying Lighthouse. The team makes big promises about its assistant and 3D sensing, so we’ll have to test how well its person recognition works, how much of a delay there is between the camera and its app, and whether all this data makes us feel safer at home or like we have an all-knowing spy in the house.

Lighthouse and its “Intelligence AI service” are only available as bundles: $399 for a device with a year of the service, $499 for a device and three years of service, or $599 for a device and five years of service. Preorders start today with the device scheduled to start shipping in September.

Correction 5/11, 9:43 AM ET: Updated to reflect that Nest also uses computer vision algorithms for intruder detection.