Like Google Glass for cars: I tried Mini’s Augmented Vision concept

Qualcomm and Mini think the future of AR starts in your car


While Google Glass may have fallen flat, companies big and small are still chasing the dream of seamless augmented reality — information delivered to your field of vision without distraction (and without making you look ridiculous). What I didn't expect was for automaker Mini, best known for its adorable compact cars, to be in that mix.

Announced today at the Shanghai auto show, I got to try the Mini Augmented Vision system at the company’s lavishly appointed downtown San Francisco dealership last week and get a guided tour of how they might work in the real world. From a fashion perspective, the glasses aren’t exactly subtle — they closely resemble ski goggles, or old-timey aviation goggles, but with a camera mounted dead-center to let it see what you’re seeing.

The project began life a year and a half ago as a collaboration with Qualcomm; it’s just an experimental prototype for now, but the two companies believe that the automobile is a clear path toward finding a use for head-mounted displays. It’s also a path that hides most of the technology’s current shortcomings.

"We think the long-term vision for something on your head where it's a head-up display for your life is super compelling," says Qualcomm augmented reality VP Jay Wright. "The challenge is there's a lot of technology required to make it happen." As Qualcomm sees it, those challenges include privacy, fashion, interacting with a device that has few standard control schemes, and battery life to make it last all day — all problems that Google Glass was unable to really overcome during its "Explorer" phase.

Is a car the right place for this technology? Mini thinks so

Mini’s parent company BMW already offers heads-up displays in a variety of models — a rudimentary form of augmented reality. "It became pretty clear to [Qualcomm] that maybe the heads-up display for your life starts out as the heads-up display in your car," says Wright, which made the partnership a natural fit. Privacy and battery concerns seemed less pressing for a device mostly worn while driving, and the bevy of controls already on a steering wheel or a dashboard could provide a good way for controlling the device without taking your hands off the wheel.

BMW’s DesignworksUSA studio worked on Mini Augmented Vision’s physical design, with Qualcomm and San Francisco-based Osterhout Design Group building and manufacturing the technology inside of the glasses. The end result isn't something you’d want to wear all day, the goggles aren’t uncomfortable. They are bulky, though — a necessity to fit in the twin 720p displays (one for each eye) as well as the Snapdragon 805 processor and networking gear, all contained inside the headset.

With the goggles on, the displays are bright enough; the resolution isn’t great, but it’s probably sufficient for the use case. (You wouldn’t want to watch a movie or play games on it.) The graphics don’t fill your entire field of vision, but the augmented imagery becomes far less important toward the periphery anyway. It’s also pretty easy to look "through" the graphics on the display to the real world, though trying to talk to someone while the goggles were on was a pretty weird experience.

Once I had the goggles properly fitted on my head, I got to step through Mini’s carefully constructed demo showing off some of the headset’s key features. While the goggles are primarily designed as driving tools, Mini demonstrated them as something you can wear on all legs of a journey — getting to your car, actually driving to a destination, and then the "final leg" of walking to where you’re going.

With the glasses calibrated, I was directed to look at a few different posters up on the wall in the demo area, meant to represent ads for events you might see around your city. When I looked at one, more info popped up about it: exact location, ticket availability, and so on. Turns out it was sold out (bummer), but the next fictional outing I looked at was still available, and my goggles asked me if I wanted to navigate to it. Using a small touchpad / button on the top of the glasses, I could swipe back and forth between options and click down to confirm it as a destination. It works pretty well for minimal interaction and a quick confirmation of an action, but I could see it being a bit laborious if you were scrolling through longer lists — the touch area is quite small, only about as large as your fingertip. That’s the only interface element here — there’s no gesture recognition of any kind.

Once I confirmed that I did indeed want to navigate to this trendy event, my goggles kicked into the three-stage navigation mode. First, I got directions from my current location to wherever I parked my Mini. When I got to the car, my glasses prompted me to get inside, at which point the navigation features were handed off to the car itself (instead of from your smartphone, which you’d need to get the goggles connected when you’re away from Wi-Fi).

Inside the car, the demo kicked off in earnest — in lieu of getting to drive, the wall in front of me projected a driving simulation, and the goggles layered data on top of it. As I "drove" to my destination, I got to see what Mini and Qualcomm deemed as essential driving information. For starters, my speed and the current speed limit were pretty much always displayed at the bottom center of my vision — handy, but not much easier than the quick glance down at your speedometer. As we drove by certain buildings, I’d get little bits of pop-up info in the corners of the display for points of interest. Like many navigation systems, you can customize this to show you gas stations or any other particular thing you might be looking for, but the system is careful not to overwhelm you with data or obscure what you need to see to drive safely.

Navigate your way to trendy events without looking away from the road

Navigation made up the main part of the Augmented Vision experience — animated arrows would show you where to go as well as the usual voice commands for turning left or right. Again, it’s not wildly different than other navigation systems, but you truly don’t need to divert your gaze to get visual assistance on where you’re going. As someone with no sense of direction who usually just relies on voice commands from my GPS, being able to more safely see the visual aids sounds pretty good to me, though having any information overlaid in my field of view would probably make me nervous when driving in the real world.

The Augmented Vision goggles can also indicate when you’ve received a text message, at which point you can use your steering wheel controls to tell the system to dictate the message back to you. For safety’s sake, it won’t ever actually display the contents of the message on your goggles — it’ll only read it back over the car’s audio system.

Cameras all over the car mean you can look right through it

The last notable trick showed off in the demo was a sort of "x-ray vision" where you can "look through" your car and see what’s outside of it. For example, you can see your tire position when parking to make sure you don’t roll over the curb, or you can see a basketball rolling nearby. The system works thanks to a host of cameras installed on the exterior of a specially equipped Mini prototype — when the driver looks toward the door, the necessary cameras kick on and show you what’s on the other side. There’s a positional sensor on top of the goggles that tracks your head movement inside the car to make this all work. It could be great for tight parking and avoiding damage to your car or the world around you, but it’s less useful (and possibly even distracting or dangerous) at speed.

Finally, I arrived at my virtual destination, with the goggles pointing out an open parking space nearby and the x-ray vision helping me parallel park better than I ever could in real life — this was an application of the feature that actually felt useful. After exiting the car, it hands navigation back to the goggles and navigates you on the last leg of the trip. At that point, I guess I’d take off the headset and stow it… somewhere? I’d rather leave it in the car, personally, but then I wouldn’t be able to do this last-leg navigation or find my car on the return trip. An inconvenience, but I’m not exactly interested in wearing these augmented reality ski goggles out in public. (If a future version of the system was basically indistinguishable from a pair of sunglasses, it might be a different story.)

Ultimately, this isn’t a system I could see myself using in its current incarnation. There’s the social stigma left in the wake of Google Glass, and the value proposition simply isn’t that strong yet — not to mention the concerns I have about augmented reality graphics interfering with my driving focus. But Mini and Qualcomm were both quick to note that this is a prototype built for research and not something they’ll be shipping to consumers any time soon. Still, the data they gather from this research will certainly inform a future project, and Qualcomm’s Jay Wright did make it clear he thinks an updated version of these goggles could be available at some point in the future. Hopefully that research will help Mini and Qualcomm find the "killer app" for augmented reality goggles in the car; for now, I’m happy just to look at my speedometer the old-fashioned way.