Every few years, Microsoft creates a video that looks at the future of productivity. This usually involves giant screens, lots of sensors, super thin tablets, and other software and hardware that will transform the way we use computers in the future. Instead of a future vision video for 2019, Microsoft opened the doors to its Envisioning Center this week, inviting a number of journalists to see Microsoft’s latest vision of the future.
Microsoft houses some of its prototype work inside a 7,000-square-foot lab at the company’s campus in Redmond, Washington. Inside, there are giant screens for collaboration, meeting rooms with devices that automatically recognize participants, and touch-powered desks that hint at how we might be working in the next decade or so. All of this hardware is driven by touch, voice, and even augmented reality, with software that reimagines how Windows and Office work today.
“The world that we work in is undergoing really dramatic changes at a really rapid pace,” says Anton Andrews, who runs the envisioning team at Microsoft.
The enormous amount of data now available to us can make it hard for everyone to keep up, and it requires companies like Microsoft to take a more fundamental approach to the future of productivity. Whereas in the past Microsoft may have attempted to modernize Windows, Office, and its other software and services, the company is increasingly looking to the web and open-source community to tackle these broader tech challenges.
Central to this vision is something Microsoft calls Fluid Framework. It’s a way of speeding up collaborative work on the web by breaking down document structures into modular components. Andrews described elements of this as being like Lego blocks, allowing Microsoft to split up data so you can easily move it from one experience to another. “For us it’s just content,” says Andrews. “The Fluid Framework just lets us play with all these bits.”
Andrews guided me through a number of demonstrations this week, all involving the idea of improving collaborative work on documents, meetings, and everything in between. One demo involved what Microsoft envisions as a “Surface Hub wall,” and it combines projection technology with 100-point Surface hub sensors to transform walls into smart surfaces that support touch, stylus, and even voice input. Much like the Surface Hub 2, the idea is that companies could transform entire walls into meeting spaces.
The demo broke apart how Windows and Office work today. It didn’t use a traditional keyboard or mouse. In fact, none of the demos I saw did. This Surface Hub wall included a prototype interface for how Windows could become a portal for launching new collaborative ways of working. Imagine a home screen with all of your information, recent documents, and data in a news feed-style way. Microsoft is doing similar things for the Surface Hub 2 software that will appear next year, allowing Windows to be more lightweight and to rely on the web more and more.
I saw these ideas appear in every prototype scenario inside Microsoft’s Envisioning Center. Andrews walked me through another demo involving a multiperson meeting, a common experience in any office. I work remotely at The Verge, so I experience the headaches of meeting technology first-hand. Whether it’s connectivity issues, weird microphone problems, or just the basics of not seeing everyone on a video call, meetings can be frustrating and leave remote employees feeling left out of the conversation.
Microsoft hopes to solve this with a combination of sensors and cameras that can better track what’s going on in a video call. At the center of this future meeting was prototype hardware that could recognize up to nine meeting participants and use AI to pick up subtle body language that a remote participant might miss. There’s also a live transcription service, Cortana integration to help run the meeting, and augmented reality to label every participant so you never forget someone’s name.
Part of the demo involves a meeting participant referencing a document and having that automatically flow into the transcription feed of the meeting. Microsoft’s Graph platform, which connects multiple services and devices, knows what that document is and is able to feed it to an AI like Cortana. That’s increasingly where Cortana is heading now: to be a digital assistant that helps office workers organize their day with conversational interactions.
Microsoft revealed earlier this week that it will allow developers to access developer kits of hardware that will enable some of this future meeting work to come to life. It seems likely that we’ll eventually see hardware from Microsoft that aims to replace the Polycom phones you typically find in business meeting rooms with devices that are packed full of sensors to enable far better meetings in the future.
The final demo that hinted at Microsoft’s immediate future was one involving desks that doubled as giant touchscreens. We’ve seen Microsoft already start to push people in this direction with devices like the Surface Studio, but the company’s future vision work involves entire desks and working spaces that are dominated by huge touchscreen computers. While the other concepts largely focused on meeting scenarios, this prototype is where workers might be collaborating across multiple documents and combining data in Microsoft’s new Fluid Framework.
By breaking documents into modular components, Microsoft envisions allowing almost anything being combined together by drag and drop. Multiple people’s work could be linked into a single document, and then Microsoft imagines using AI to understand the images and text and combine it all into a single style. This vision is clearly a ways off, but it gives some hints about where the company is thinking of taking its Fluid Framework.
“The underlying architecture can whole handle any kind of data,” explains Mike Morton, a program manager for Fluid Framework at Microsoft. “We haven’t shown audio or video… but I’ll say there’s absolutely the underlying technology to sort of support that.” That could mean a scenario where you drop a live-streamed video link into a document, and it’s automatically transcribed.
Microsoft is also planning to make sure its Fluid Framework is open, so the community will be able to contribute to it and help the company develop it over time. It’s still early, but Microsoft has been working on this since 2016 in prototype form, and it moved it into a bigger engineering effort around 12 months ago. Microsoft has been pivoting a lot of its efforts to focus on the web, and CEO Satya Nadella has been pushing the company to go faster on the web and even adopt Chromium for its Edge browser.
“Satya has been an advocate for our investments on the web since he’s been CEO,” says Morton. “Satya has been so phenomenally forward-looking, and naturally people can always be resistant to change but Satya is very good at encouraging that [change] for direction.”
Microsoft’s challenge now is to bring this future to life before its competitors in a way that’s not too much of a change for people who are used to Office and Windows. We’ve seen the company push too fast with Windows 8 in the past, and Microsoft is now looking at adapting Windows to different modes for different hardware. That involves a lighter version of Windows, known as Windows Lite, that strips back a lot of the complexity of Windows in favor of a more basic user interface and perhaps even a bigger focus on the web.
Microsoft didn’t talk about Windows Lite this week at Build, leaving the bigger Windows announcements for more developer-focused features like Windows Terminal or the Linux kernel coming to Windows 10. We’ve already seen Windows get more basic on devices like the HoloLens 2 or Surface Hub 2, and if Microsoft wants to deliver on the vision it’s outlining in these demos, then the notion of Windows will be changing and adapting a lot over the coming decade.
It’s starting to look like Microsoft will increasingly open-source parts of Windows and allow the community to improve it. We’ve seen that with the Windows calculator and PowerToys recently, and it seems to be an approach that will continue in the foreseeable future. Microsoft is redesigning its future to be far more open internally, and with technologies like Fluid Framework, it looks set to deliver on the future vision it has been dreaming up for years.