Skip to main content

Niantic shows off a stunning AR demo that lets Pikachu hide behind real-world objects

Niantic shows off a stunning AR demo that lets Pikachu hide behind real-world objects


It’s a machine learning-assisted technique Niantic is calling occlusion

Share this story

Pokémon Go creator Niantic announced today that it’s planning on letting third-party developers use its augmented reality platform to develop apps. To showcase how that technology, which Niantic calls its Real World Platform, is improving all the time, the company made a demo video to showcase a technique it calls occlusion.

Basically, Niantic used machine learning techniques it picked up from an acquisition of London-based startup Matrix Mill to create a neural network that could, in real time, obscure virtual images behind real-world objects. This will let creatures like Pikachu hide behind flower pots and park benches, and it will even obscure the virtual pocket monster behind dynamic parts of the scene, like moving people.

Here’s what you might normally see through your smartphone camera’s viewfinder with a standard AR application today:

And here’s what Niantic can do with its new occlusion technique:

This is just an experimental proof-of-concept, Niantic CEO John Hanke told reporters at a meeting at its San Francisco headquarters yesterday. However, it’s a strong indication of the type of next-generation AR techniques the company is pioneering for games like Pokémon Go and the upcoming Harry Potter: Wizards Unite.

We don’t know when this kind of sophisticated AR will arrive in actual products, but it’s safe to say Niantic wants its Real World Platform to be as robust as possible, so developers decide to build apps on top of it. Hanke and other Niantic executives stressed that they see the platform much like Amazon does its AWS cloud computing platform, so we can expect it to be a big focus for the company’s most ambitious AR advancements going forward.