Skip to main content

OpenAI has published the text-generating AI it said was too dangerous to share

OpenAI has published the text-generating AI it said was too dangerous to share

/

The lab says it’s seen ‘no strong evidence of misuse so far’

Share this story

Illustration by Alex Castro / The Verge

The research lab OpenAI has released the full version of a text-generating AI system that experts warned could be used for malicious purposes.

The institute originally announced the system, GPT-2, in February this year, but withheld the full version of the program out of fear it would be used to spread fake news, spam, and disinformation. Since then it’s released smaller, less complex versions of GPT-2 and studied their reception. Others also replicated the work. In a blog post this week, OpenAI now says it’s seen “no strong evidence of misuse” and has released the model in full.

GPT-2 can write fake news articles, stories, poems, and code

GPT-2 is part of a new breed of text-generation systems that have impressed experts with their ability to generate coherent text from minimal prompts. The system was trained on eight million text documents scraped from the web and responds to text snippets supplied by users. Feed it a fake headline, for example, and it will write a news story; give it the first line of a poem and it’ll supply a whole verse.

It’s tricky to convey exactly how good GPT-2’s output is, but the model frequently produces eerily cogent writing that can often give the appearance of intelligence (though that’s not to say what GPT-2 is doing involves anything we’d recognize as cognition). Play around with the system long enough, though, and its limitations become clear. It particularly suffers with the challenge of long-term coherence; for example, using the names and attributes of characters consistently in a story, or sticking to a single subject in a news article.

The best way to get a feel for GPT-2’s abilities is to try it out yourself. You can access a web version at TalkToTransformer.com and enter your own prompts. (A “transformer” is a component of machine learning architecture used to create GPT-2 and its fellows.)

1/6

Apart from the raw capabilities of GPT-2, the model’s release is notable as part of an ongoing debate about the responsibility of AI researchers to mitigate harm caused by their work. Experts have pointed out that easy access to cutting-edge AI tools can enable malicious actors; a dynamic we’ve seen with the use of deepfakes to generate revenge porn, for example. OpenAI limited the release of its model because of this concern.

However, not everyone applauded the lab’s approach. Many experts criticized the decision, saying it limited the amount of research others could do to mitigate the model’s harms, and that it created unnecessary hype about the dangers of artificial intelligence.

“The words ‘too dangerous’ were casually thrown out here without a lot of thought or experimentation,” researcher Delip Rao told The Verge back in February. “I don’t think [OpenAI] spent enough time proving it was actually dangerous.”

In its announcement of the full model this week, OpenAI noted that GPT-2 could be misused, citing third-party research stating the system could help generate “synthetic propaganda” for extreme ideological positions. But it also admitted that its fears that the system would be used to pump out a high-volume of coherent spam, overwhelming online information systems like social media, have not yet come to pass.

The lab also noted that its own researchers had created automatic systems that could spot GPT-2’s output with ~95% accuracy, but that this figure was not high enough “for standalone detection” and means any system used to automatically spot fake text would need to be paired with human judges. This, though, is not particularly unusual for such moderation tasks, which often rely on humans in the loop to spot fake images and videos.

OpenAI says it will continue to watch how GPT-2 is used by the community and public, and will further develop its policies on the responsible publication of AI research.

Today’s Storystream

Feed refreshed An hour ago The tablet didn’t call that play by itself

E
Twitter
Emma RothAn hour ago
Missing classic Mario?

One fan, who goes by the name Metroid Mike 64 on Twitter, just built a full-on 2D Mario game inside Super Mario Maker 2 complete with 40 levels and eight worlds.

Looking at the gameplay shared on Twitter is enough to make me want to break out my SNES, or at least buy Super Mario Maker 2 so I can play this epic retro revamp.


R
External Link
Russell BrandomTwo hours ago
The US might still force TikTok into a data security deal with Oracle.

The New York Times says the White House is still working on TikTok’s Trump-era data security deal, which has been in a weird limbo for nearly two years now. The terms are basically the same: Oracle plays babysitter but the app doesn’t get banned. Maybe it will happen now, though?


Asian America learns how to hit back

The desperate, confused, righteous campaign to stop Asian hate

Esther Wang12:00 PM UTC
R
Youtube
Richard LawlerTwo hours ago
Don’t miss this dive into Guillermo del Toro’s stop-motion Pinocchio flick.

Andrew Webster and Charles Pulliam-Moore covered Netflix’s Tudum reveals (yes, it’s going to keep using that brand name) over the weekend as the streamer showed off things that haven’t been canceled yet.

Beyond The Way of the Househusband season two news and timing information about two The Witcher projects, you should make time for this incredible behind-the-scenes video showing the process of making Pinocchio.


E
External Link
Emma Roth4:13 PM UTC
Netflix’s gaming bet gets even bigger.

Even though fewer than one percent of Netflix subscribers have tried its mobile games, Netflix just opened up another studio in Finland after acquiring the Helsinki-based Next Games earlier this year.

The former vice president of Zynga Games, Marko Lastikka, will serve as the studio director. His track record includes working on SimCity BuildIt for EA and FarmVille 3.


A
External Link
Andrew J. Hawkins3:37 PM UTC
Vietnam’s EV aspirant is giving big Potemkin village vibes

Idle equipment, absent workers, deserted villages, an empty swimming pool. VinFast is Vietnam’s answer to Tesla, with the goal of making 1 million EVs in the next 5-6 years to sell to customers US, Canada and Europe. With these lofty goals, the company invited a bunch of social media influencers, as well as some auto journalists, on a “a four-day, multicity extravaganza” that seemed more weird than convincing, according to Bloomberg.


J
James Vincent3:17 PM UTC
Today, 39 years ago, the world didn’t end.

And it’s thanks to one man: Stanislav Petrov, a USSR military officer who, on September 26th, 1983, took the decision not to launch a retaliatory nuclear attack against the US. Petrov correctly guessed that satellite readings showing inbound nukes were faulty, and so likely saved the world from nuclear war. As journalist Tom Chivers put it on Twitter, “Happy Stanislav Petrov Day to those who celebrate!” Read more about Petrov’s life here.


Soviet Colonel who prevented 1983 nuclear response
Photo by Scott Peterson/Getty Images
J
The Verge
James Vincent3:03 PM UTC
Deepfakes were made for Disney.

You might have seen the news this weekend that the voice of James Earl Jones is being cloned using AI so his performance as Darth Vader in Star Wars can live on forever.

Reading the story, it struck me how perfect deepfakes are for Disney — a company that profits from original characters, fans' nostalgia, and an uncanny ability to twist copyright law to its liking. And now, with deepfakes, Disney’s most iconic performances will live on forever, ensuring the magic never dies.


E
External Link
Elizabeth Lopatto2:41 PM UTC
Hurricane Fiona ratcheted up tensions about crypto bros in Puerto Rico.

“An official emergency has been declared, which means in the tax program, your physical presence time is suspended,” a crypto investor posted on TikTok. “So I am headed out of the island.” Perhaps predictably, locals are furious.


R
The Verge
Richard Lawler2:09 PM UTC
Teen hacking suspect linked to GTA 6 leak and Uber security breach charged in London.

City of London police tweeted Saturday that the teenager arrested on suspicion of hacking has been charged with “two counts of breach of bail conditions and two counts of computer misuse.”

They haven’t confirmed any connection with the GTA 6 leak or Uber hack, but the details line up with those incidents, as well as a suspect arrested this spring for the Lapsus$ breaches.


R
The Verge
Richard Lawler1:00 PM UTC
Green light.

Good morning to everyone, except for the intern or whoever prevented us from seeing how Microsoft’s Surface held up to yet another violent NFL incident.

Today’s big event is the crash of a NASA spaceship this evening — on purpose. Mary Beth Griggs can explain.


D
David Pierce12:54 PM UTC
Thousands and thousands of reasons people love Android.

“Android fans, what are the primary reasons why you will never ever switch to an iPhone?” That question led to almost 30,000 comments so far, and was for a while the most popular thing on Reddit. It’s a totally fascinating peek into the platform wars, and I’ve spent way too much time reading through it. I also laughed hard at “I can turn my text bubbles to any color I like.”