As you might have seen in our second episode of On The Verge, our own Joshua Topolsky recently toured Microsoft's Building 99, where the company conducts a variety of wild research. Today we're pleased to share a look at Kinect Fusion and Lightspace — two technologies that bridge the physical and virtual worlds with sensors and imaging.
Kinect Fusion is a system that uses the Kinect's sensors to create an interactive, real-time 3D model of the environment — the demonstration shows virtual balls bouncing around on the objects captured and rendered directly from the real world. (It's not the first time we've seen the Kinect used for 3D modeling, but it's nice to see an official effort.) Microsoft's Kevin Schofield is quick to point out that the $150 Kinect sensor can accomplish the same tasks as industrial versions of the technology that cost about $50,000.
Lightspace works in the opposite direction: with a combination of depth cameras and projectors, it can create linked interactive screens on different surfaces. In the video, principal researcher Andy Wilson demonstrates how objects projected on a table can be moved around, re-sized, and even carried to another display using his hands. It's something you have to see to really understand, so fire up that video.


There are 72 Comments. Add yours.
Loving all this behind the scenes coverage of Microsoft Josh. Keep it coming.
Posted on Dec 21, 2011 | 2:42 PM EST reply Recommend (35) Flag actions
Microsoft gets props in my book — they clearly have very awesome humans at work over there.
Posted on Dec 21, 2011 | 2:52 PM EST reply Recommend (30) Flag actions
They make the craziest and coolest stuff what you can imagine, but the management at MS is not capable of bringing the ideas from the lab to real world products, which is really sad. The world of tomorrow could be much closer.
Posted on Dec 21, 2011 | 3:48 PM EST reply Recommend (4) Flag actions
wrong. Its because Shareholders want a profitable buissness, not one that creates the future.
This is exactly what happened to IBM. And the reason they had to quit the PC buisness. They spent too much on R&D.
Posted on Dec 21, 2011 | 4:08 PM EST reply Recommend (7) Flag actions
Where do you think Kinect came from?
Posted on Dec 21, 2011 | 4:26 PM EST reply Recommend (11) Flag actions
PrimeSense (an Israeli company). www.primesense.com/
Posted on Dec 21, 2011 | 6:40 PM EST reply Recommend (2) Flag actions
There was a Wired article about Kinect that went into a reasonable amount of detail – http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer.
They did one part of it, but it isn’t like MS just bought the whole thing.
Posted on Dec 21, 2011 | 7:00 PM EST reply Recommend (3) Flag actions
well the kinect as it is now is quite different from what was licensed to them by rimesense.
Posted on Dec 22, 2011 | 8:40 AM EST reply Recommend (1) Flag actions
Kinnect is the proof you are wrong! ;)
Posted on Dec 21, 2011 | 6:57 PM EST reply Recommend Flag actions
No kinect proves that he’s right. Kinect is a really cool product that uses really cool technology but in the worst possible way. What uses kinect? A handful of shitty games that are really nothing more than demos, and a stupid dancing game. Certainly not much of a game changer.
Posted on Dec 22, 2011 | 12:09 AM EST reply Recommend (2) Flag actions
Fruit Ninja is awesome
Posted on Dec 22, 2011 | 12:15 PM EST reply Recommend (1) Flag actions
Funny, I thought I read somewhere there were doctors using it for some kind of work. People doing some rehabilitation and other companies using it for there work.
Posted on Dec 22, 2011 | 12:50 PM EST reply Recommend (2) Flag actions
http://www.youtube.com/watch?v=U67ESHV8f_4
Posted on Dec 23, 2011 | 5:50 PM EST reply Recommend Flag actions
It was a cool trip.
Posted on Dec 21, 2011 | 5:50 PM EST reply Recommend (1) Flag actions
Both this clip and all of the others in these series are almost as incredible as the technology they show off. Wow.
Posted on Dec 21, 2011 | 3:52 PM EST reply Recommend (2) Flag actions
Could we have a storystream of all the content from MS Research for the lazy among us?!
Posted on Dec 21, 2011 | 3:58 PM EST reply Recommend (5) Flag actions
Great idea! We’ll make it so.
Posted on Dec 21, 2011 | 5:45 PM EST reply Recommend (2) Flag actions
www.microsoft.com/next is one place to stop off for this kind of stuff Mark :)
Posted on Dec 28, 2011 | 8:18 PM EST reply Recommend Flag actions
I’ve wondered why that projected interactive display demo doesn’t include a phone. It seems obvious to tap a phone to an object to pick it up and then flick it to another surface… like a vessel.
Posted on Dec 21, 2011 | 2:50 PM EST reply Recommend (2) Flag actions
Guess they’re going for a simpler approach, maybe. That said, Lightspace is the closest thing I’ve seen to a real-world Minority Report computer in action. So awesome.
Posted on Dec 21, 2011 | 3:00 PM EST reply Recommend (2) Flag actions
I think they are shooting for a more natural interface with this. Involving a phone would just create an extra step. I would like the user to be able to just sweep the image off the table and have the cameras track the person instead of having the user “hold” it in his palm.
Posted on Dec 21, 2011 | 6:13 PM EST reply Recommend (2) Flag actions
Or more likely drag an item onto a phone and then just take it with you somewhere else (of course by the time we see this in real products I imagine everything would be on a server somewhere anyway so the idea of storing it on your phone would seem pretty odd).
Posted on Dec 21, 2011 | 6:52 PM EST reply Recommend (1) Flag actions
THAT, was AWESOME.
Posted on Dec 21, 2011 | 2:59 PM EST reply Recommend (1) Flag actions
Nice!
Posted on Dec 21, 2011 | 2:59 PM EST reply Recommend Flag actions
This is not an exclusive look.. It was shown long back by Microsoft.
Posted on Dec 21, 2011 | 3:02 PM EST reply Recommend (1) Flag actions
I would imagine it is an exclusive trip into the lab where they showed it
Posted on Dec 21, 2011 | 3:23 PM EST reply Recommend (2) Flag actions
It’s as exclusive as the articles on your ad filled blog : http://wmpoweruser.com/
Posted on Dec 21, 2011 | 3:26 PM EST reply Recommend (7) Flag actions
An ad-filled blog? More like ads with some badly written text around it. Used to frequent wmpoweruser a lot, and hoped they’d change their ways, but gave up after a month or so. Annoying as hell.
Posted on Dec 21, 2011 | 3:33 PM EST reply Recommend (3) Flag actions
wmpoweruser is barely usable as a site. Crashed my Chrome browser twice!
Posted on Dec 21, 2011 | 6:09 PM EST reply Recommend Flag actions
Just went on their site again. It looked clean at first, and then I noticed that it was loading for a minute. Every single domain it was querying was ad-related during that entire minute.
Posted on Dec 21, 2011 | 3:34 PM EST reply Recommend Flag actions
Why are you such a troll? It’s exclusive because Josh actually got to go see this first hand. How about you? You have an exclusive first look at how to plaster ads everywhere right?
Posted on Dec 21, 2011 | 3:59 PM EST reply Recommend (4) Flag actions
What’s more hilarious is how he’s calling out people here, but at his own blog regularly bans anyone who criticizes him.
Posted on Dec 22, 2011 | 12:15 AM EST reply Recommend (1) Flag actions
Microsoft demoed Kinect Fusion to a few hundred people at UIST in October. The NYTimes was there, although I don’t know if they covered it. Lightspace seems new to me, though, so it seems fair to call it an exclusive.
Posted on Dec 22, 2011 | 4:18 PM EST reply Recommend Flag actions
http://www.engadget.com/2011/08/09/microsofts-kinectfusion-research-project-offers-real-time-3d-re/
Posted on Dec 21, 2011 | 3:04 PM EST reply Recommend Flag actions
flagged this. Reason: engadget is evil
Posted on Dec 21, 2011 | 4:28 PM EST reply Recommend (7) Flag actions
Did Engadget actually visit the labs for this amazing new technology? I didn’t think so.
Posted on Dec 21, 2011 | 6:16 PM EST reply Recommend Flag actions
This is rather phenomenal.
Posted on Dec 21, 2011 | 3:07 PM EST reply Recommend Flag actions
… trying to get deep inside the causality of potential poison.
Posted on Dec 21, 2011 | 3:23 PM EST reply Recommend Flag actions
Totally awesome!
Posted on Dec 21, 2011 | 3:30 PM EST reply Recommend Flag actions
I love that you’re doing this. It’s a great idea. Don’t show everything at once, but gradually release clips.
Posted on Dec 21, 2011 | 3:35 PM EST reply Recommend (1) Flag actions
Very awesome! I’m loving all the multimedia on the Verge.
Posted on Dec 21, 2011 | 3:37 PM EST reply Recommend Flag actions
Iron Man OS here we come! ;)
Posted on Dec 21, 2011 | 3:38 PM EST reply Recommend Flag actions
I find MS to be very kool!
Posted on Dec 21, 2011 | 3:38 PM EST reply Recommend Flag actions
Nice work Josh!
Posted on Dec 21, 2011 | 3:40 PM EST reply Recommend Flag actions
The guys at MS research are becoming my heros… some of the things that they are developing are fan-fricking-tastic!!
Posted on Dec 21, 2011 | 3:46 PM EST reply Recommend (3) Flag actions
The lightspace would be great in a presentation room or for teachers.
Posted on Dec 21, 2011 | 4:01 PM EST reply Recommend (1) Flag actions
Would love for the future tech of tomorrow to hurry up and get here. Nice work Josh.
Posted on Dec 21, 2011 | 4:18 PM EST reply Recommend Flag actions
If you like this sort of stuff you may like channel9.msdn.com where they have a team within Microsoft constantly producing content like this, talks with dev teams and lots more. It’s been around for years and they show some really cool stuff.
Posted on Dec 21, 2011 | 4:22 PM EST reply Recommend (2) Flag actions
True, word, and church. My addiction to the Verge and Ping are getting in the way of my gaming habit. This might be good, might be bad, my brain cells haven’t got back to me yet.
The projected surface work MSR has done has a lot of possibilities for all kinds of practical applications.
Posted on Dec 21, 2011 | 9:33 PM EST via mobile reply Recommend Flag actions
Woah this is cool!
Posted on Dec 21, 2011 | 4:34 PM EST reply Recommend Flag actions
These kind of technology shouldn’t be taken for granted… it’s just incredible and so advanced…
This reminds me of why I love technology so much… It’s modern day magic!
If I were Josh, I would spend the whole day just transferring content from one surface to the other… just 4 fun!
Much Love!
Posted on Dec 21, 2011 | 4:48 PM EST reply Recommend Flag actions
This is awesome, I love all the technology that Microsoft develops. And things like this shows that the company innovates as many other companies do and they just don’t play catch up. I think this is cooler than the iPhone or iPad, said that, they are good devices as well.
Thanks,
PS: Great content that The Verge is producing keep it coming!
Posted on Dec 21, 2011 | 4:52 PM EST reply Recommend Flag actions
Wow. I just watched the 3D printer video and now this one.
Star Trek TNG is here.
How long before we get real Holodecks and Replicators?
Posted on Dec 21, 2011 | 4:58 PM EST reply Recommend Flag actions
Its amazing that Microsoft gave you such deep access to their new technologies and even more amazing that they allowed you to share it with us. Kudos to Microsoft and great job Josh. Thank you!
Posted on Dec 21, 2011 | 5:40 PM EST reply Recommend Flag actions
I was just talking about this kind of stuff with someone yesterday; Google has things like the self-driving car coming out of their labs but mostly seem pretty secretive (hard to hide a car I guess), Apple is notoriously secretive, and MS just seem happy to publish papers and pretty much just show stuff to people.
It makes me wonder if MS is happy to show us these kinds of things what on Earth are they working on that they don’t want to show us?
Posted on Dec 21, 2011 | 6:17 PM EST reply Recommend Flag actions
Microsoft research is so awesome. If MS ever pulls an OSX and ditches the stupid ass DOS for unix-based system, I may just switch.
Posted on Dec 21, 2011 | 6:37 PM EST reply Recommend Flag actions
Hmmm… what does it matter whether it’s running Unix or DOS at the core? Assuming you’re a regular user, all you care about is the user interface and the responsiveness. Windows 8 boots super-fast. Windows 7/8 are both good-looking and feature-rich.
Posted on Dec 21, 2011 | 6:39 PM EST reply Recommend (1) Flag actions
I’m not a typical user.
Posted on Dec 21, 2011 | 7:26 PM EST reply Recommend Flag actions
Why is the OS architecture of importance to you? Well, anyway MS better pull an OSX in order to be privileged to have you as an user of their OS.
Posted on Dec 21, 2011 | 7:04 PM EST reply Recommend Flag actions
Come on, cut the sarcasm.
Posted on Dec 21, 2011 | 7:27 PM EST reply Recommend Flag actions
3:05 mark, “That is… That is very freaky.”
lol.
Posted on Dec 21, 2011 | 7:51 PM EST reply Recommend Flag actions
I have been a huge Windows and OSX fan for a long time now and all i can say is that with each day Microsoft is pulling me away from Apple. They are so far ahead of anything Apple are doing it’s not even funny. I saw the last patent they submitted which was a pathetic attempt to try and get into the 3D action with a patent that is clearly not even close to what Kinect does today. The iPad2 is probably the last apple product i will buy and i will not even use it, it will only be for my daughter.
I don’t really know anything about Joshua but it clearly looks like he is so blown away it is hard to explain. Something is cool when it is a few month more advanced than others, things are freaky when they are a few years ahead of everyone else. That is Microsoft today they are so freaky their next 10 years will be incredible.
Posted on Dec 21, 2011 | 9:46 PM EST reply Recommend Flag actions
These extra videos are better than the first one
Posted on Dec 22, 2011 | 1:42 AM EST reply Recommend Flag actions
I really hope Apple “steals” the Kinect-technology for their new AppleTV.
Posted on Dec 22, 2011 | 4:50 AM EST reply Recommend Flag actions
Video doesn’t really go full screen in Safari (5.1.2 OS X 10.6.8) Youtube can do it, Vimeo can do it, Flash or HTML 5. Why can’t The Verge do it?
Posted on Dec 22, 2011 | 5:39 AM EST reply Recommend (1) Flag actions
We’re going to have a fix for that very, very soon.
Posted on Dec 22, 2011 | 10:06 AM EST reply Recommend Flag actions
In Firefox on Mac not only does it not go full-screen, but the rest of the page is overlayed on top of it! Good to hear there is a fix coming soon; I hope it is good for all browsers!
Posted on Dec 22, 2011 | 2:53 PM EST reply Recommend Flag actions
Any reason why MS isn’t stepping into the hardware game? With all these things that they are doing with imaging and touch it seems plausible that they will.. get rid of Ballmer !!
Posted on Dec 22, 2011 | 8:34 AM EST reply Recommend Flag actions
This is wild where technology is going. Thanks Josh for that video clip.
Posted on Dec 22, 2011 | 9:51 AM EST reply Recommend Flag actions
The only two real tech companies are Microsoft and Apple. The rest are just corporate suits.
Posted on Dec 22, 2011 | 11:41 AM EST reply Recommend (1) Flag actions
So Fucking Cool!
Posted on Dec 22, 2011 | 12:45 PM EST reply Recommend Flag actions
I believe we have seen the next kinect, or an early prototype: http://cl.ly/3b2b2k22090v103y3G0F
Posted on Dec 22, 2011 | 2:51 PM EST reply Recommend (1) Flag actions
Something to say? Choose one of these options to log in.