Qualcomm officially took the wraps off the Snapdragon 855, its next generation processor, yesterday at the Snapdragon Tech Summit. At day two of the conference, Qualcomm is providing all the details on what the new processor will offer for the next wave of flagship phones.

Qualcomm is focusing on several key areas for improvement with the Snapdragon 855, including connectivity, the camera, artificial intelligence, entertainment, and more.

Connectivity and performance

On the connectivity front, the 855 chipset itself will feature Qualcomm’s new X24 modem — notably, not the X50 modem that will enable 5G functionality, which will be a separate add for companies that want to offer it. That’s important: Snapdragon 855 alone won’t give you full 5G. But the X24 is still a big jump over the existing X20 modem on the Snapdragon 845: the new chip supports up to 2Gbps speeds on LTE (Cat. 20), making it the first chip in the world to support that spec.

And the 855 will support both fast mmWave and longer-range sub-6GHz 5G networks when paired with an X50 modem to give devices the full range of 5G support, something we’re already seeing with the upcoming 5G Moto Mod attachment, which has a full Snapdragon 855 processor inside.

“When it’s millimeter wave you’ll get really fast speeds, but if you roam out of that area we want to make sure users with that 5G mod will still get the fastest speeds,” Motorola’s Doug Michau told The Verge.

Plus, the 855 will add faster Wi-Fi, too — Qualcomm claims it’s the first mobile chip with faster Wi-Fi 6 (formerly 802.11ax) and 802.11ay millimeter wave Wi-Fi. Both are theoretically capable of up to 10Gbps speeds, though with millimeter wave in particular we’re talking about fairly short-range data transfers.

When it comes to performance, Qualcomm says the 855, which is the chipmaker’s first to use the 7 nanometer process (similar to Apple’s A12 Bionic), will offer 20 percent more graphics performance thanks to its Adreno 640 GPU. The Kryo 485 CPU will also bring a 45 percent CPU performance increase thanks to its eight cores; the CPU will have one prime core, three performance cores, and four efficiency cores, and Qualcomm says it’s the biggest CPU leap it’s ever produced. The 855 also has a Hexagon processor specifically dedicated to accelerated AI-related tasks.

Artificial Intelligence

As for as those AI performance gains go, Qualcomm is saying the Snapdragon 855 will big sizable improvements to tasks that rely on neural network-assisted software. The Adreno, Kyro, and Hexagon processors are all capable of working together to power AI tasks, and they all form what Qualcomm is calling its fourth generation AI engine. What all of that actual amounts to is a three-fold performance gain for AI-related tasks, and double the performance of competing mobile chips with regards to AI tasks.

Qualcomm says it’s been working with Google to advance the search giant’s open-source machine learning TensorFlow library specifically for on-device mobile performance. In terms of real-world products that partnership will impact, Qualcomm says the object and photo-recognition and AR platform Google Lens will see some big gains from the 855. Google says it’s moved some key Lens processing to the 855 directly, so phones using the chip won’t see latency or any other detriments caused by having to rely on cloud communication.

One example is text recognition using Lens, so you’ll be able to now point an 855-equipped phone and more efficiently identify and even translate text in the real world. In addition to text recognition, on-device TensorFlow improvements will let the 855 perform all sorts of vision-related tasks faster than before, including AR tricks like placing virtual furniture in a room to experiment with different home decors.

Another unique AI tasks that doesn’t involve vision at all is the ability to isolate someone’s voice on a phone call, to let you better hear that person if they’re in a noisy environment. Qualcomm also demoed onstage the ability to add a background blur effect to any photo, including those you download off the internet, using so-called real-time segmentation algorithms, which is a step above the portrait mode effects you get from iPhone and Pixel devices that require you use a special camera mode.

This is thanks to a partnership with an AI company called Nalbi, which is working with Qualcomm to add other impressive new computer vision features like changing your hair color in real time, in the event you want to see a new style before deciding to actually go through with it.

The camera

The camera is also getting some exciting new changes with the 855, with an updated Spectra 380 ISP (image signal processor). According to Qualcomm, with the 855, computational processing for things like computer vision-powered portrait modes, AR/VR functions, and more are integrated directly into the ISP, offering a significant speed boost and power reduction for computationally heavy photography tasks.

That translates to some big changes on a practical level: 855 can do depth sensing at 60 fps, enabling things like portrait mode previews in real time (instead of requiring processing after the fact), and even portrait mode videos with real-time HDR, at 4K resolution, that can cut out and blur a background behind a person in real time. Qualcomm also showed off an ad-hoc green-screen style effect, that used similar tech to completely replace the background behind a person in live footage. 

Some of those camera improvements are now possible because, Qualcomm is also shifting from JPEG to HEIF images for pictures. The reason for the shift is simple: JPEG is not designed to capture things like HDR, computer vision, video-integrated files (like Apple Live photos), and everything else that gets packed into a smartphone image today. 

Apple already did this change two years ago, but Qualcomm’s support will push this harder for Android. (Android 9 Pie added system level support for HEIF too, which is useful). Devices that have multiple lenses like LG V40 can benefit from this — HEIF will allow phones like that to shoot all three images and store all three as a single photo file.

And it could mean better battery life when using your camera: Qualcomm’s promising a 2-4x battery life improvement from its new image signal processor, because it doesn’t have to fire up the CPU and GPU as often.

Gaming and entertainment

It’s a given that when CPU and GPU performance improve, so will gaming performance — but Qualcomm says the Snapdragon 855 will go further that that. Today, the company’s announcing its new Snapdragon Elite Gaming Platform as a marketing banner that encompasses a host of new features.

There’s support for Vulkan 1.1, the low-level graphics API that let game developers get more performance from a given phone than they would otherwise. Qualcomm says it’s a 20 percent performance improvement over the competing OpenGL ES. With the bumps in performance, Qualcomm says your next phone will now support 10-bit HDR gaming for the first time. There’s also physically based rendering techniques, where materials (like a suit of armor) are rendered based on the virtual “materials” they’re made of. That can also provide up a 20 percent performance improvement too, according to Qualcomm, assuming developers take advantage.

The bright Snapdragon 855 will make their way into the video realm too, where Qualcomm says it’ll now support the HDR10+ and Dolby Vision standards for brighter, more vivid displays, and Qualcomm says the chip has enough performance for HDR video at 120 frames per second, or even 360-degree panoramic video at 8K resolution for volumetric VR.

And with hardware-accelerated H.265 and VP9 video support, Qualcomm says you’ll see a 7x power savings compared to playing back video without it. That’s potentially more battery life, too.

It’s never certain how many of a given smartphone chip’s features will make their way into the real world. Qualcomm’s job is to provide the capabilities and then let smartphone manufacturers, carriers, and OS providers run with them. But there certainly seem to be a lot of new feature in the Snapdragon 855, should those partners be willing.