Skip to main content

The iPad-mounted Structure Sensor 3D scanner turns your world into a video game

The iPad-mounted Structure Sensor 3D scanner turns your world into a video game

Share this story

Gallery Photo:
Gallery Photo:

Augmented reality has never been nearly as cool as its name suggests. Even when it isn't gimmicky, it's virtually useless, in large part because your mobile camera's version of "reality" is short one dimension. Scanning and computer vision company Occipital, however, wants to add real depth to your tablet's vision with the Structure Sensor, a Kickstarter- backed product that shipped to backers late last year. The $349 Structure Sensor is a Kinect-like camera that fits on the back of an iPad or, with some hacking, any other device. With the bracket on, your tablet doesn't just see objects, it can figure out how far away they are, doing anything from turning an object into a 3D model to measuring the distance of a room.

Occipital's creators aren't in the business of making games or augmented reality apps. They're not really even in the business of camera hardware, though that's the product they're selling. Their hope is that the bundled Structure SDK will become an industry standard for camera makers and developers alike. Today, though, they showed us a few of the ideas they have in mind, including one of the simplest uses of the camera: 3D scanning.

The Structure Sensor can't pick up as many details as a stationary scanner, but it's a clear winner in speed

Stationary 3D scanners, most notably MakerBot's Digitizer, have started percolating through the 3D printer world — and if you're looking for fine detail, you should stick with them. The Structure Sensor doesn't deal well with objects smaller than the medium-sized stuffed animal sitting in the hospitality suite, and it overlays larger ones with a soft, relatively low-detail mesh. The upside, though, is that it's pretty cheap and extremely fast. Scanning a small object on the Digitizer takes about ten minutes; creating a model with the Structure Sensor takes as much time as you need to walk slowly around the object. The iPad camera can capture and overlay textures on the model, and the software lets you seal up gaps and export it for use as a virtual asset or a 3D-printable design. A large bust of Julius Caesar looked fantastic as a textured 3D design, a stuffed Winnie the Pooh looked like a well-rendered video game character from 1998, and my DSLR was a vaguely camera-shaped blob.

These 3D scanner capabilities caught the attention of printing company 3D Systems, which has struck a deal to offer self-branded versions of the Structure Sensor. But the augmented reality uses, rough as they are right now, are intriguing. One demo is a more sophisticated version of AR special effects apps, letting you scan a corner of a room and throw balls for a virtual pet — toss the ball at a chair and it will bounce off; throw it under the table and the cartoon cat will disappear underneath to chase it. Another captures the walls, floor, and furniture of a room, noting the real dimensions and turning it into a virtual map, albeit one that my clumsy scanning left full of holes. Ikea already lets you place virtual products in your house, but this could take the guesswork out of scale and, Occipital suggests, even let you move scanned models of your own furniture around the new additions.

The software platform still has some obvious glitches. The cat would slip through thin chair legs and pop through the table at the edge of the frame, and Occipital is still working on getting its tracking mechanism to work with cylindrical objects like the Coke can I completely failed to capture. It's also only as strong as the ecosystem that can be built upon it, which means Occipital is depending on early adopters to really take advantage of its work. The Structure Sensor's saving grace, though, is that it's one of the cleverest and most ambitious mobile augmented reality projects I've seen — even the failures are fun to watch.

Today’s Storystream

Feed refreshed Sep 25 Not just you

E
Twitter
Emma RothSep 25
Rihanna’s headlining the Super Bowl Halftime Show.

Apple Music’s set to sponsor the Halftime Show next February, and it’s starting out strong with a performance from Rihanna. I honestly can’t remember which company sponsored the Halftime Show before Pepsi, so it’ll be nice to see how Apple handles the show for Super Bowl LVII.


E
Twitter
Emma RothSep 25
Starlink is growing.

The Elon Musk-owned satellite internet service, which covers all seven continents including Antarctica, has now made over 1 million user terminals. Musk has big plans for the service, which he hopes to expand to cruise ships, planes, and even school buses.

Musk recently said he’ll sidestep sanctions to activate the service in Iran, where the government put restrictions on communications due to mass protests. He followed through on his promise to bring Starlink to Ukraine at the start of Russia’s invasion, so we’ll have to wait and see if he manages to bring the service to Iran as well.


E
External Link
Emma RothSep 25
We might not get another Apple event this year.

While Apple was initially expected to hold an event to launch its rumored M2-equipped Macs and iPads in October, Bloomberg’s Mark Gurman predicts Apple will announce its new devices in a series of press releases, website updates, and media briefings instead.

I know that it probably takes a lot of work to put these polished events together, but if Apple does pass on it this year, I will kind of miss vibing to the livestream’s music and seeing all the new products get presented.


E
External Link
Emma RothSep 24
California Governor Gavin Newsom vetoes the state’s “BitLicense” law.

The bill, called the Digital Financial Assets Law, would establish a regulatory framework for companies that transact with cryptocurrency in the state, similar to New York’s BitLicense system. In a statement, Newsom says it’s “premature to lock a licensing structure” and that implementing such a program is a “costly undertaking:”

A more flexible approach is needed to ensure regulatory oversight can keep up with rapidly evolving technology and use cases, and is tailored with the proper tools to address trends and mitigate consumer harm.


Welcome to the new Verge

Revolutionizing the media with blog posts

Nilay PatelSep 13
A
Youtube
Andrew WebsterSep 24
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.


A
The Verge
Andrew WebsterSep 24
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.


T
Twitter
Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.