Skip to main content

Adobe’s prototype AI tools let you instantly edit photos and videos

Adobe’s prototype AI tools let you instantly edit photos and videos

/

They’re just prototypes, but they show the power of AI to manipulate the visual world

Share this story

Advances in artificial intelligence are making it easier than ever to seamlessly edit and manipulate photos and images. At The Verge, we’ve already talked about this quite a bit, but Adobe keeps on coming up with new, ever more compelling demonstrations of this tech. As spotted by FastCo.Design, the company recently showed off some prototype AI tools at its annual MAX conference. They’re all being developed by the company’s Sensei unit — a machine learning division that’s previously showed off a voice-activated Photoshop assistant and an automatic selfie tweaker.

These new tools are a little more substantial though. One, called Scene Stitch, is basically an advanced version of Content Aware Fill, Adobe’s go-to method for removing unwanted items from images. But while Content Aware Fill looks within the picture being edited for plausible filler (copy and pasting in that patch of grass, or this rock) Scene Stitch uses Adobe’s library of stock photos as a paint palette instead. It analyzes the image the user is trying to edit, and swaps in similar scenes and perspectives. You can see it in action in the video below:

Adobe says it’s also working on making this tool semantically aware, so it not only recognizes the composition of the image, but also the content. (E.g. it would suggest swapping out people with more people; trees with different trees.) From the perspective of someone trying to spot an edited image, this makes it much harder, as you can’t just look for repeated patches of pixels.

Another feature, demoed under the codename Project Cloak, is essentially Content Aware Fill but for video. It allows users to isolate a specific part of a clip (say, a strap on a backpack, or a streetlight that’s in the way of a nice view) and remove it with the click of a button. Making this sort of edit is already possible, but it usually requires going through a video frame-by-frame. It’s hard work and takes skilled labor. In the video below, you can see Cloak managing the same task in just a few seconds:

Contemporary AI is proving to be a remarkably able tool for this sort of task. And although image and video editing isn’t anything new, the possibility of giving easy-to-use tools like this to anyone who wants them may be a cause for concern. (See also: using AI to fake someone’s voice.) Fake news, for example, could be turbo-charged by these sorts of capabilities, as could information warfare from nation states. In the future, “seeing is believing” might not be the axiomatic truth it is today.