Skip to main content

Twitch says deepfake porn is now grounds for instaban — here’s why

Twitch says deepfake porn is now grounds for instaban — here’s why

/

The livestreaming service says it was already technically banned under other policies, but Twitch is making things explicit after a January incident.

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

a mosaic representing porn
A blurred mosaic representing porn.

It’s been five full years since most online platforms made it crystal clear that face-swapped porn is not okay — but not Twitch, apparently. Today, in a blog post titled “Addressing Explicit Deepfake Content,” the livestreaming service now says that synthetic non-consensual exploitative images (NCEI) will not be tolerated.

Even a brief unintentional glimpse at those sorts of images “will be removed and will result in an enforcement,” the company writes. And if you intentionally promote, create, or share deepfake porn, that’s grounds for an instaban: doing that “can result in an indefinite suspension on the first offense.”

The company isn’t doing this on a whim — as BuzzFeed News and NBC News reported last month, Twitch recently had its own deepfake scandal. On January 30th, Twitch streamer Brandon “Atrioc” Ewing left a browser window open on stream that reportedly showed the faces of popular female Twitch streamers, including Pokimane, QTCinderella, and Maya Higa, “grafted onto the bodies of naked women,” as BuzzFeed tells it. In a tearful apology stream, Atrioc admitted he visited a deepfake site out of “morbid curiosity” about the images. “I just clicked a fucking link at 2AM, and the morals didn’t catch up to me,” he said while promising never to do anything like that again.

Needless to say, those female streamers weren’t happy.

It’s not clear if Twitch took any enforcement action against Atrioc at the time — the company didn’t immediately respond to a fact-check request — but the new policy makes it clear that at least some action would be taken.

Twitch does tend to clamp down on accounts sharing sexual images, even when they accidentally make their way into a livestream. Atrioc himself was previously banned for showing a flaccid penis on screen, according to streaming news site Win.gg, and Pokimane famously got a warning (not a ban) after accidentally opening PornHub in a browser tab. But Twitch’s previous stance on deepfakes was extremely limited: it only mentioned them in the context of “sharing negative doctored or artistic content to abuse or degrade another person.”

Twitch did previously prohibit “broadcasting or uploading content that contains depictions of real nudity” and threatened instabans for “sexual violence and exploitation,” however.

Originally, QTCinderella vowed to sue the deepfake porn site that Atrioc brought to the world’s attention, but she’s since told NBC News that she’s given up: “Every single lawyer I’ve talked to essentially have come to the conclusion that we don’t have a case; there’s no way to sue the guy.”

We reported on the legality of deepfake porn back in 2018, and... things haven’t necessarily improved much since. According to the Cyber Civil Rights Initiative, only four US states have deepfake laws that aren’t specific to political elections. The UK, EU, and China are looking into crackdowns, too.