A network of fictional journalists, analysts, and political consultants has been used to place opinion pieces favorable to certain Gulf states in a range of media outlets, an investigation from The Daily Beast has revealed. At least 19 fake personas were used to author op-eds published in dozens of mainly conservative publications, with AI-generated headshots of would-be authors used to trick targets into believing the writers were real people.
It’s not the first time AI has been used in this way, though it’s unusual to see machine learning tech deployed for online misinformation in the wild. Last year, a report from The Associated Press found a fake profile on LinkedIn, part of a network of likely spies trying to make connections with professional targets, that also used an AI-generated headshot.
AI-generated headshots are increasingly realistic
AI-generated profile pictures created by sites like ThisPersonDoesNotExist.com have some unique advantages when it comes to building fake online personas. The most important characteristic is that each image is uniquely generated, meaning they can’t be traced back to a source picture (and thus quickly proved to be a fake) using a reverse image search.
However, the current generation of AI headshots isn’t flawless. They share a number of common tells, including odd-looking teeth, asymmetrical features, hair that blurs into nothing, earlobes that are strangely melted, and indistinct background imagery.
Some of these features can be seen in a number of headshots used by fake writers uncovered by The Daily Beast’s investigation. Others, though, just use stolen avatars. The personas share a number of attributes, which suggest they’re part of a single, coordinated campaign:
The personas identified by The Daily Beast were generally contributors to two linked sites, The Arab Eye and Persia Now; had Twitter accounts created in March or April 2020; presented themselves as political consultants and freelance journalists mostly based in European capitals; lied about their academic or professional credentials in phony LinkedIn accounts; used fake or stolen avatars manipulated to defeat reverse image searches; and linked to or amplified each others’ work.
Although it’s not clear who created the network, op-eds published by the fake writers do share certain editorial values. They argue for more sanctions against Iran, praise certain Gulf states like the United Arab Emirates, and criticize Qatar (currently the subject of a diplomatic and economic embargo from the UAE and other states in the Middle East because of the country’s alleged support for terrorism).
The network was used to create op-eds published in US outlets like the Washington Examiner and the American Thinker, as well as Middle Eastern papers like The Jerusalem Post and Al Arabiya, and even in the English-language Hong Kong-based publication the South China Morning Post. As a result of The Daily Beast’s investigation, Twitter has suspended 15 accounts belonging to the fake writers.