The “zoom and enhance” trope is a TV cliché, but advances in AI are slowly making it a reality. Researchers have shown that machine learning can enlarge low-resolution images, restoring sharpness that wasn’t there before. Now, this technology is making its way to consumers, with image editor Pixelmator among the first to offer such a feature.
The Photoshop competitor today announced what it calls “ML Super Resolution” for the $60 Pro version of its software: a function that the company says can scale an image up to three times its original resolution without image defects like pixelation or blurriness.
AI super resolution adds lost sharpness to blurry images
After our tests, we would say this claim needs a few caveats. But overall, the performance of Pixelmator’s super resolution feature is impressive.
Pixelation is smoothed away in a range of images, from illustration to photography to text. The results are better than those delivered by traditional upscaling algorithms, and although the process is not instantaneous (it took around eight seconds per image on our 2017 MacBook Pro), it’s fast enough to be a boon to designers and image editors of all stripes. There are some examples below from Pixelmator, with a zoomed-in low resolution image on the left, and the processed ML Super Resolution image on the right:
You can see more images over on Pixelmator’s blog, including comparisons with traditional upscaling techniques like the Bilinear, Lanczos, and Nearest Neighbor algorithms. While ML Super Resolution isn’t a magic wand, it does deliver consistently impressive results.
The algorithm learns to predict new details pixel by pixel
Research into super resolution has been ongoing for some time now, with tech companies like Google and Nvidia creating their own algorithms in the past few years. In each case, the software is trained on a dataset containing pairs of low-resolution and high-resolution images. The algorithm compares this data and creates rules for how the pixels change from image to image. Then when it’s shown a low-resolution picture it’s never seen before, it predicts what extra pixels are needed and inserts them.
Pixelmator’s creators told The Verge that their algorithm was made from scratch in order to be lightweight enough to run on users’ devices. It’s just 5MB in size, compared to research algorithms that are often 50 times larger. It’s trained on a range of images in order to anticipate users’ different needs, but the training dataset is surprisingly small — just 15,000 samples were needed to create Pixelmator’s ML Super Resolution tool.
The company isn’t the first to offer this technology commercially. There are a number of single-use super resolution tools online, including BigJPG.com and LetsEnhance.io. In our tests, the output from these sites was of a more mixed quality than Pixelmator’s (though it was generally good), and free users can only process a small number of images. Adobe has also released a super resolution feature, but the results are, again, less dramatic.
Overall, Pixelmator seems to be offering the best commercial super resolution tool we’ve seen (let us know in the comments if you know of a better one), and every day, “zoom and enhance” becomes less of a joke.
Correction: An earlier version of this story included comparisons between images that had been non-destructively downsized then upscaled using Pixelmator’s ML Super Resolution, resulting in unrealistically improved results. These have been removed. We regret the error.