Yesterday, Magic Leap CEO Rony Abovitz and NBA commissioner Adam Silver announced plans to eventually bring Turner-owned NBA content to Magic Leap’s elusive AR goggles, by casting virtual screens in front of a wearer’s face through the goggles. It wasn’t an earth-shattering announcement, but it was an indication that Magic Leap’s tech could be more than billion-dollar vaporware, and that the company is thinking about the kind of content needed to convince people to wear light-field face computers.
What’s more interesting is that Abovitz believes volumetric video — a way of capturing video as 3D objects and scenes — will be live-streamed to people’s faces within a relatively short period of time, and that the kind of cameras used to make this happen will eventually replace traditional broadcast cameras.
In an interview with The Verge, Abovitz said that within “two to five years,” it will be technically possible for people wearing Magic Leap goggles to watch an NBA game (or other media) live, but in a holographic, interactive form.
“You can stream over the top and to the screens, the virtual screens — you can do that now,” he said. “We’re looking at, how do you derive the information to move the volumetric stuff from that? And then, how do you do volumetric live-streaming as well ... if you time where processing power is going, particularly backends, you’re single-digit years away from that happening.”
Abovitz said that in order for it to work, it would require multiple fixed camera sensors to be placed around a venue. He also said that instead of using “30 big TV cameras shooting from different angles,” broadcasters would instead use a larger number of much smaller, super high-resolution cameras that would send all the data to a backend computing stack.
“[It will] move from standard 2D television broadcasting to full volumetric capture. That’s probably going to take a decade before you see it happening across multiple sports and news, but you’ll see early adopters,” he said.
Right now, making volumetric video is a production-intensive process. It often requires cylindrical green screen stages, like the kind Microsoft and 8i use; a multi-camera setup; and plenty of video processing and rendering time. The end result is video assets that you can walk up to, walk around, and even talk to, whether in a VR headset or through the lens of a smartphone. Many of the examples that exist to date have been avatars created for marketing or educational purposes, like this Buzz Aldrin hologram.
What Abovitz is proposing is not only something that requires more computational power, but also means outfitting everyday spaces or venues with these kinds of cameras, and on the consumer end, getting people to buy and wear the headset. Abovitz pointed out that one of the benefits of working with a sports league is that it’s a fixed physical space, which could make the production challenges easier to tackle.
Abovitz also shared more about how apps on Magic Leap’s goggles and its proprietary operating system will work, claiming that the number of possible interactions on the developer version of the goggles is “insanely large.” The system is capable of interpreting commands from someone’s head, eyes, voice, gestures, and a physical control, he said, which developers can currently customize. “Because it’s the creator edition [of the goggles], if you want to blink twice and use your voice, we’re not stopping you.”
App icons will also appear in volumetric format on the app launcher, and by just looking at the icon, a preview of the app will be revealed, whether it’s “an NBA experience, a Mario Kart, or whatever it is you want to do,” he said.
-Adi Robertson contributed to this report.
Comments
Alternate Title: Vaporware guy talks about more vaporware-y stuff.
By NAM37 on 02.14.18 9:55am
I find it odd that the number of comments I see that cry vaporware seems to increase as Magic Leap shows off more of their stuff. Seems counter intuitive. But I get it. People want to know what is going on and Magic Leap refuses to tell us. It is frustrating.
When they are ready, they will show you the full product. But they have something. It might turn out crap. It might be stupid. It might be bad.
But they have something. You can’t partner with the NBA if you don’t have something. You can’t partner with Sigur Ros if you don’t have something. You can’t partner with Star Wars and Disney if you don’t have something. You don’t hire 1200+ employees unless you have something. You don’t build a manufacturing arm unless you have something. You don’t continue to get investment from huge names over many, many years unless you have something. This company might fall on its face and consumers might hate what they have built. But to suggest they haven’t built anything makes you sound crazy. We even saw it on Shaq’s face.
By jonomacd on 02.14.18 12:55pm
Well said. Looking forward to it too. As well as options from other vendors, we’re just in the warm up area for AR tech.
By SasparillaFizz on 02.14.18 2:13pm
I guess you‘re on to something!
By Furtin on 02.14.18 2:28pm
What you have said makes perfect sense, but it is unsual to see a new tech company emerge and blow up like this, especially when apple/google/amazon/samsung/microsoft/facebook are basically buying up any promising company these days. It’s just too hard to compete with existing tech ecosystems and it’s one of the reasons why Viv had to sell themselves to samsung, as much as I didn’t want that to happen. Magic leap is doing something that isn’t mainstream yet and it will be interesting if Mixed reality takes off, and who will reign supreme in that market.
By Syntaxkita on 02.14.18 6:09pm
You’re confusing "vaporware" with "scam."
Vaporware is a product that doesn’t exist and can’t be purchased because it’s still being created. Over time, the term has kind of shifted to products that never get released or officially canceled, or to something that is constantly hyped but doesn’t appear to be moving towards a true release.
Three definitions. Magic Leap fits two of them. It’s something that doesn’t exist in purchasable form because it’s still under development, and it’s a constantly hyped product of which we’ve seen nothing really marketable nor a clear path to launch.
It’s not a scam. They’re clearly spending tons of money on actual things. Like product development and manufacturing facilities. Unless the CEO is giving himself a multi-million dollar salary (and he’d see his investors bail if he did), there’s no scam here. He believes. Others believe.
But people believed in Duke Nukem 3D from 3D Realms. People believed in Half Life 3. People believed in Infinium Labs. Glaze 3D. Until we see something tangible, that meets what they’re claiming to be able to do, this is vaporware. Vaporware has been released, and has met expectations. It’s rare, but it happens.
By MosquitoControl on 02.14.18 6:23pm
Yeah, I think this is a fair comment. The term vaporware has a few definitions. I guess the other point is that they do have hardware. Shaq was wearing it in that video. You can even see flashes of light from the lenses implying this isn’t a mock up but real, working hardware. Still, until it gets in more hands it’s fair to be skeptical of what the hardware can actually do.
I’m mostly responding to the huge cynicism that exists around magic leap. I wish we could just be more excited for them and if it turns out to be a crap, well, who cares? It isn’t my investment money!
By jonomacd on 02.15.18 11:12am
Hilarious how impatient everyone is with stuff like this
By fatty bunter on 02.15.18 7:11am
With every word this guy says it sounds more like vapourware.
For the sake of the argument: If, just if, he can get venues fitted with all the cameras, and whatever tech needed to mix it all together, and the broadband to transmit it – how many people are going to want to sit on their couch wearing stupid, cut-you-off-from-the-world goggles, and how many of those few are actually going to plonk down the cash?
I’m sorry, it’s not going to happen.
By Mha71 on 02.14.18 10:02am
There are legitimate concerns to be sure but I think you miss the point. These particular goggles are not of the "cut-you-off-from-the-world" variety.
By shabanga on 02.14.18 10:55am
You may be able to see the outside world, but the ugliness and stigma that comes with wearing those will make the rest of the world cut you out. Professional use might happen, but this will not become mainstrean.
By Rockl0bster on 02.14.18 12:11pm
Thanks for that pointless reply! The OP specifically mentioned "sit on their couch" so I wasn’t considering the ramifications of wearing this thing around out on the street.
By shabanga on 02.14.18 7:40pm
VR cuts you off from the world, not AR. These are AR.
By Stone Cold Dan Quinn on 02.14.18 10:57am
Because of their design they will cut off your peripheral vision. The Hololens way is the better way. At least it has the visual canvas size to improve the field of view with.
By Doge Man on 02.14.18 11:26am
Watch an NBA game on the TV or courtside? I’d at least give the dorky glasses a shot, cut-me-off-from-the-world-please. Problem with Magic Leap, I can’t tell if that’s even what they’re proposing. Doesn’t really sound like it.
By dansplosion on 02.14.18 11:25am
The problem with your argument is assuming that it’s just Magic Leap who are trying to get this off the ground. Intel just showed at CES that they are pushing this hard, they are the ones taking care of the venues and tools to shoot this stuff. The other issue is that Abovitz acknowledges this will be a niche market for "early adopters" for the first few years. As far as vaporware, they go on sale sometime around spring. I’d wait at least until that passes.
By Malkmus on 02.14.18 11:50am
This is AR so it doesn’t cut you off from the world. Also, the technology will advance to the point that these won’t be bulky goggles.
By Disdain on 02.14.18 12:43pm
I know right? Imagine, you have to wear these crazy things on your head. They have a huge wire down to your pocket and your senses are cut off from the world!
Why would anyone every buy headphones!!!???
By jonomacd on 02.14.18 12:57pm
But seriously, I think you are right about this iteration. It is just too bulky. The point is to look further than that. It’s like saying "The palm pilot is crap, smartphones can never take off."
I think, if they can keep shrinking the optics and electronics, there is a good chance this sort of technology is the future. And, well, if it is, someone has to build it. It doesn’t just appear. Magic Leap might be those someones. They may be building the equivalent of the palm pilot. Apple or Google or Microsoft might eventually take over the market. But for comparison, you can’t have smartphones without the palm pilot. Someone has to work on these things to get them built, train up people on how to make them and show others what is possible. They don’t happen in a vacuum.
So we can slag Magic Leap off all we want. But at least they have ambition and are trying to make something genuinely new and interesting. It might fail, but most new and interesting things do. Let’s celebrate those that try.
By jonomacd on 02.14.18 1:10pm
The problem is that the Palm Pilot actually worked, and worked really well for what it did, better than what was already available. These glasses have too many hurdles to work – and even though they are AR, they still cut you off from those next to you – you see stuff that they don’t. Unless you have a whole family or group of friends with the same glasses on. And even then you are taking part in something artificial. The social aspect is what will keep these from taking off.
AR or VR for playing games can work. For designing stuff, or other specific work-related stuff, it can work. But in place of a TV? Nope. Why do you thing 3D TV never became more than a buzz word…
By Mha71 on 02.14.18 2:22pm
The two hurdles you mentioned are not a big enough detriment though. That people can’t see what you see and that it won’t replace TV’s are outweighed by the enormous potential they offer for an innumerable amount of applications. BTW, I wouldn’t be surprised if there ends up being some sort of way to mirror what you see to others, considering the number of cameras on it.
By Malkmus on 02.14.18 6:10pm
Consumers don’t like wearing tech. How would this be different then the failed attempt to bringing 3D video into the home over, and over again over the years?
The naive assumption that they are doing something better or cooler then the glasses used to bring 3D movies into the home 10 years ago, or the recent attempt of bringing VR into the home which doesn’t seem to be taking off either, I just don’t see this company cracking the nut that consumers will suddenly opt to wear something on their face for an enhanced visual experience.
Wearables are a dead end if you can’t get consumers to want to actually wear them. About the only concept I have seen for AR that would work is in the show Altered Carbon having some kind of contact lens you can easily put in and take out, but if something has to sit on your nose, then they blew it.
By ChrisCW on 02.14.18 10:33am
I’m not sure the big issue is wearing it. In most cases you’ve mentioned the issue was cost, coupled with poor performance and lack of compelling content.
3DTV was doomed from the start. The tech was never particularly good, and nobody wanted to re-buy their TV just so they could watch a few movies that were just terrible conversions. Due to the fact nobody bought the TVs, none of the broadcasters truly committed to creating new and better content and so it was a self-fulfilling prophecy.
VR is still so early to consider a failure but again, I don’t think it’s because of the head gear. If anything, the physical problem with first gen VR is the ridiculous space requirement along with the mess of cables tethering you to a high end PC. Yes there is cell phone VR but there isn’t really enough power there yet to do anything truly interesting. Then back to cost, even now the cheapest entry to VR on a PC will set you back $800-1000 if you don’t already own a PC. And yet again we have the issue of content creation not really improving because of the slow adoption rate, but in this case we also have the fun of separate segregated ecosystems which are only now beginning to show signs of cross compatibility but it still isn’t even fully baked which means my $1000 rig might not be able to play that game because I have the wrong headset. No thanks.
Anyhow, that was really long but the point is that wearables aren’t dead because they’re wearables, they’re dead because none of the companies involved are committed enough to make their products anything beyond a novelty act.
By shabanga on 02.14.18 11:09am
"But if something has to sit on your nose, then they blew it."
Tell that to the majority of the population, who wears glasses.
By urname on 02.14.18 1:09pm
Apples and oranges.
You have three choices, wear glasses, contacts or go with poor vision. Completely different decision to make.
By Mergatroid Mania on 02.14.18 3:30pm