Facebook F8 conference 2017: the latest news and biggest changes coming to the platform

Gary Shteyngart's Super Sad True Love Story feels more prescient every year. The 2010 dystopian fiction novel imagined a world where holographic, smartphone-like devices called äppäräts project everyone’s personal information all of the time, while a mammoth world-spanning social network called GlobalTeens stratifies society by their looks and net worth.

In Shteyngart's not-too-distant future, everyone is ranked with ludicrous metrics like Hotness and Fuckability, on credit score-esque scales out of 800. Society is forever on the brink of economic collapse, and yet the tech-obsessed populace worries only about its corporate status and the availability of life extension and cosmetic surgery. The novel, which ended up inadvertently predicting Google Glass and even Occupy Wall Street to some extent, now feels like it keeps telling dark truths about the time we live in, seven years after it was published.

The rumblings started months ago. Through a series of peculiar job listings and key hires, it became clear Facebook was up to something unlike anything it had ever pursued. Building 8, as the company would name it, was to be a new division under famed technologist Regina Dugan, former director of the government's Defense Advanced Research Projects Agency (DARPA). Dugan had transitioned to the tech industry in 2012, serving as the head of Google’s experimental ATAP group. Among other things, it was responsible for the promising but now defunct Ara modular smartphone project.

On Wednesday, Facebook took the wraps off Building 8 and had Dugan tell the world what exactly her fast-growing team has been working on. At the day-2 keynote at the company’s F8 developer conference in San Jose, Dugan announced Facebook’s plans for two ambitious projects: one to develop a system for letting you type with just your thoughts, and another to let you “hear” using vibrations on your skin. This would be done through brain-computer interfaces — devices that can read neural activity and translate it into digital signals, and vice versa.

Facebook is banking on the smartphone camera being the most influential and ubiquitous augmented reality platform in the future, and it’s spent a healthy chunk of its F8 developer conference showing the world what this will look like. While virtual game boards and underwater effects are neat, the company is also developing AR features that are more subtle and way more useful.

One involves using a mix of the same type of computer vision and artificial intelligence-powered algorithms that place virtual images in a scene to do standard photo effects. So instead of placing a giant whale next to your cereal bowl, the camera emulates a low-aperture photo with a blurred-out background. Onstage at F8, Facebook’s Joaquin Candela, the head of its applied machine learning team, showed how its software is capable of “dynamically estimating what is foreground and what is background.” Once it differentiates between the two, it can blur on out to produce the desirable bokeh effect you get with professional DSLR camera with a low-aperture lens.

Facebook’s advanced hardware group is working on technology to let you “hear with your skin.” The technology could be used to help deaf people communicate, but Facebook also envisions it as a way to advance communications for people who can already hear, allowing for such things as a conversation to be automatically translated into another language.

The technology is being developed by Facebook’s Building 8 research group, led by ex-DARPA director and former head of Google’s experimental research group Regina Dugan.

Facebook today unveiled a project from its secretive Building 8 research group that’s working to create a brain-computer interface that lets you type with your thoughts. Regina Dugan, a former director of DARPA and the ex-head of Google’s experimental ATAP research group, announced the news today onstage at Facebook’s F8 developer conference. Dugan, who now heads up Building 8, says the goal is “something as simple as a yes-no brain click” that could fundamentally change how we interact with and use technology. While it does not exist today outside of very specific medical research trials, Dugan says her team is actively working to make it a reality.

Dugan refers to the technology as a “brain mouse for AR,” meaning it could be an ideal way to receive direct input from neural activity that would remove the need for augmented reality devices to track hand motions or other body movements. For instance, the Microsoft HoloLens uses hand tracking to let you tap your finger in front of you as if you were clicking a mouse. Facebook’s theoretical device could also be used for patients with severe paralysis, acting as a “speech prosthetic” Dugan says.

Facebook today is launching a new developer toolkit that will make it easier for users to capture and share their personal experiences in VR, be it a moment in an Oculus Rift game or their personal point of view in a 360-degree live-action video. As it stands today, the best way to do this is to capture every frame of a scene, stitch it together into one image, and then encode the whole batch of images as a new video. The other method has been to simply stream whatever is happening onscreen through Twitch or YouTube, or to record it with third-party game-capture software.

The stitching method is resource and time intensive, while the streaming method leaves you with a low-quality video you can only rewatch in 2D. So Facebook engineers say they came up with a new process, called cube mapping, that’s more efficient and retains the same high image quality as the original 4K stream. The company is now giving the tools away as part of its 360 Capture SDK, announced this morning at the F8 developer conference, so creators of VR apps and films can make use of it. “We realized that the standard method is not really feasible for real-time capture of 4K 360-degree video,” says Chetan Gupta, a product manager for Facebook’s 360 Media division. “We instead came up with a new method, which is where we capture the final output of that 360 video right from the game engine.”

Facebook announced at its F8 developer conference that the Android version of Instagram is getting offline functionality. In fact, the features are already rolling out in certain parts of the world. (I was able to get some of it to work in New York on a phone running Android Nougat, for example.) Offline mode could eventually make it to iOS as well, according to TechCrunch.

The offline mode features go beyond just saving a draft or queueing up a photo at the top of the feed, which the app already let users do when they tried to post with poor service. You can now like or comment on other users’ photos, or even follow and unfollow accounts, without any data connection. The next time your phone accesses the internet, Instagram will go back through this history and complete each of those actions.

Facebook is developing a small helicopter that can be deployed in emergencies to deliver internet access, the company said. Speaking at the F8 conference in San Jose today, the company said that the helicopter, dubbed “Tether-tenna,” would provide “instant infrastructure” during times of crisis. “When completed, this technology will be able to be deployed immediately and operate for months at a time to bring back connectivity in case of an emergency,” said Yael Maguire, who runs Facebook’s Connectivity Lab, in a blog post.

Tether-tenna could be deployed in cases when cellular infrastructure was damaged but fiber lines in the area still work, Maguire said. In those instances, the helicopter will tether to a fiber line and electricity, and then rise hundreds of feet in the air to broadcast a signal.

Facebook today announced the second generation of its Surround 360 video camera design, and this time the company is serious about helping potential customers purchase it as an actual product. The Surround 360, which Facebook unveiled last year as an open-source spec guide for others to build off of, has been upgraded as both a larger, more capable unit and a smaller, more portable version.

Facebook is calling the big model the x24, because it now has a 24-camera array arranged in an orb instead of the 17 cameras the original flying saucer-shaped Surround 360 called for. The small model is the x6, with just six cameras but in a far more manageable package. Instead of just releasing the design schematics for these cameras online, as it did last year, Facebook is now teaming up with a select group of hardware partners to manufacture and sell finished products later this year. It’s unclear if these products will be Facebook-branded in any way, but the company is still stressing that it has no plans to sell the cameras directly.

Facebook just announced a bunch of changes to Messenger, with the highlight being new ways for apps to integrate with it. And while it’s not available yet, one of the standout partners announced was Apple Music.

Near the end of his presentation at Facebook’s F8 conference this afternoon, David Marcus, head of Messenger, said, “I’m really excited to share with you that Apple Music will soon be on the platform as well.”

Facebook Messenger is adding third-party extensions and a new discovery tab in a new effort to make it users’ primary messaging app. A year after the rocky rollout of bots, Messenger is taking another swing with changes focused on making them easier to find and use. The changes, which were announced at Facebook’s F8 developer conference today in San Jose, could give users another reason to look at bots after their underwhelming debut in 2016.

Changes rolling out on Messenger today include:

Facebook’s team collaboration software, Workplace, is getting a free version anyone can sign up for. The company is calling the tier its standard version, and it’s currently testing it with a select group of users with a plan to roll it out later this year for any group that’s eager to sign up. The freemium move emulates popular chat app Slack’s multi-tier model. Facebook’s aim is to get as many users trying its own workplace communication app as possible in the hopes it spreads to larger organizations and becomes a viable competitor to both Slack and Microsoft’s new Teams software, which it gives away for free to users of Office 365.

Workplace has two big advantages here. The first is obvious: nearly 2 billion people are on Facebook, giving it an existing and accessible group of users to target. The second is tactical. Facebook is currently giving away Workplace’s more costly, enterprise version for free until September 30th. When it does start charging, it will do so at rates far below Slack’s. The plan is to charge organizations only $3 a user for the first 1,000 members, and then $2 a user for the next 9,000 members added, and then $1 a user from then on out. Slack, on the other hand, charges between $6.50 to $12.50 a month for things like added storage and search.

Facebook’s most fascinating virtual reality experiment, a VR hangout session where you can interact with friends as if you were sitting next to one another, is now ready for the public. The company is calling the product Facebook Spaces, and it’s being released today in beta form for the Oculus Rift. The news, announced this morning at Facebook’s F8 developer conference in San Jose, means anyone with a Rift and Touch controllers can join up to three other people in a virtual playground. There you can watch videos, take photos, and engage in a number of different VR activities together.

Spaces was first shown off at the Oculus Connect conference in October, when Mark Zuckerberg donned a Rift onstage and joined other Facebook employees in an early version of the product. We saw the Facebook exec play a game of chess, teleport to different locations, and even take a mixed-reality selfie with his wife Priscilla Chan, who dialed into the VR room using Facebook Messenger. While it built off similar experiences, like the existing Oculus Rooms feature for Gear VR and Oculus’ Toybox demo from two years ago, Spaces was bizarre and powerful enough to get everybody talking about what the future of VR technology could enable.

Facebook co-founder and chief executive Mark Zuckerberg, speaking today at the company’s annual developers conference, expressed condolences for the victim of a recent murder in Cleveland, Ohio, that was shared in a video on the social networking site.

Zuckerberg mentioned Robert Godwin Sr., the 74-year-old victim, by name, expressed condolences to his family and friends, and said, “We have a lot of work ... we will keep doing all we can to prevent tragedies like this from happening.” The brief remarks came on the heels of an official statement from Facebook, in which the company said it would be reviewing its “reporting flows” in order to allow people better report content that violates Facebook policies.

Facebook today announced a platform for developers to build new experiences into its in-app cameras, saying it would bring augmented reality into the mainstream and position Facebook to reap the majority of the benefits. Speaking onstage at Facebook’s F8 developer conference, CEO Mark Zuckerberg said that AR would be the next major platform for computing. A closed beta that opens today will let developers begin experimenting with photo and video filters, games, art projects, and more.

In a demonstration, Zuckerberg showed a variety of dazzling camera effects. Swiping to the stories camera that Facebook introduced last month, users will soon find thousands of augmented reality effects, he said. These go beyond the art frames and face filters of today to include three-dimensional text and images. In one demo, giant puffy words reading “It’s feeding time” rose out of a breakfast table, where a series of sharks swam around a cereal bowl.