OpenAI has published the text-generating AI it said was too dangerous to share

Illustration by Alex Castro / The Verge

The research lab OpenAI has released the full version of a text-generating AI system that experts warned could be used for malicious purposes.

The institute originally announced the system, GPT-2, in February this year, but withheld the full version of the program out of fear it would be used to spread fake news, spam, and disinformation. Since then it’s released smaller, less complex versions of GPT-2 and studied their reception. Others also replicated the work. In a blog post this week, OpenAI now says it’s seen “no strong evidence of misuse” and has released the model in full.

GPT-2 is part of a new breed of text-generation systems that have impressed experts with their ability to generate coherent text from minimal prompts. The system was trained on eight million text documents scraped from the web and responds to text snippets supplied by users. Feed it a fake headline, for example, and it will write a news story; give it the first line of a poem and it’ll supply a whole verse.

It’s tricky to convey exactly how good GPT-2’s output is, but the model frequently produces eerily cogent writing that can often give the appearance of intelligence (though that’s not to say what GPT-2 is doing involves anything we’d recognize as cognition). Play around with the system long enough, though, and its limitations become clear. It particularly suffers with the challenge of long-term coherence; for example, using the names and attributes of characters consistently in a story, or sticking to a single subject in a news article.

The best way to get a feel for GPT-2’s abilities is to try it out yourself. You can access a web version at TalkToTransformer.com and enter your own prompts. (A “transformer” is a component of machine learning architecture used to create GPT-2 and its fellows.)

Samples of GPT-2 in action. The underlined text is generated by the AI.

Apart from the raw capabilities of GPT-2, the model’s release is notable as part of an ongoing debate about the responsibility of AI researchers to mitigate harm caused by their work. Experts have pointed out that easy access to cutting-edge AI tools can enable malicious actors; a dynamic we’ve seen with the use of deepfakes to generate revenge porn, for example. OpenAI limited the release of its model because of this concern.

However, not everyone applauded the lab’s approach. Many experts criticized the decision, saying it limited the amount of research others could do to mitigate the model’s harms, and that it created unnecessary hype about the dangers of artificial intelligence.

“The words ‘too dangerous’ were casually thrown out here without a lot of thought or experimentation,” researcher Delip Rao told The Verge back in February. “I don’t think [OpenAI] spent enough time proving it was actually dangerous.”

In its announcement of the full model this week, OpenAI noted that GPT-2 could be misused, citing third-party research stating the system could help generate “synthetic propaganda” for extreme ideological positions. But it also admitted that its fears that the system would be used to pump out a high-volume of coherent spam, overwhelming online information systems like social media, have not yet come to pass.

The lab also noted that its own researchers had created automatic systems that could spot GPT-2’s output with ~95% accuracy, but that this figure was not high enough “for standalone detection” and means any system used to automatically spot fake text would need to be paired with human judges. This, though, is not particularly unusual for such moderation tasks, which often rely on humans in the loop to spot fake images and videos.

OpenAI says it will continue to watch how GPT-2 is used by the community and public, and will further develop its policies on the responsible publication of AI research.

Comments

It’s not working for me, sounds really cool.

I think there is good research going on at OpenAI. But when they started out they made it their thing to be the non-profit moral authority in AI. And when that did not work out financially and they needed to make money without destroying their image, they developed this aggressive moral-panic-style of hype around their "product" – and that is bound to cause disappointment.

What GPT-2 could do was impressive given the general state of the art, but not anywhere as good or "dangerous" as the hype made it out to be. AI research has suffered several disappointment-"winters" resulting from hype cycles, and I think OpenAI does a disservice to the field with their behaviour. Fortunately more and more projects are catching up or even ahead of them, so I would hope the field is sufficiently robust this time.

I am not going to let this thing write my tweets (and I do not tweet myself, so no tweets from me).

I just tried a number of things and every time it is the same, it appears to just look the text in whatever collection of texts it has in its database, and then starts quoting the next few paragraphs from that text.

For the input "I just tried a number of things and every time it is the same, it appears to just look the text in whatever collection of texts it has in its database, and then starts quoting the next few paragraphs from that text. "
it continues with:

"There was no text in the file that was not there, and I just couldn’t get a decent answer out of the program.
A: Ok, I’m going to tell you what I did.
Q: Ok, I just opened a file in a text editor and opened that text file in that text editor. And that text file contained the text, and I copied it.
A: Right, right. And I can tell you that it’s not the file I’ve been working on for the last three years and that the text is in that text file and I just used the copy command to copy the text from the file I’ve been working on into your file. So, I copied from the text file in that file to your text file.
Q: And I copied what?"

So it can’t even help me make better comments…

So it can’t even help me make better comments.

It’s like when you’re in a room with a couple, and you’ve done the dance moves.

I don’t know how to do that, so I don’t try’that’s my fault. I should have looked for a better dance move.

Then you look around and you’re like: Oh! I forgot to do that one.

Or you’re like: Well, you could have done this better, but you’re not gonna do that one anyway.

It’s just, like, you have to do them.

It’s a very different kind of experience.

Wow. That was a trip.

It’s a very different kind of experience. A lot of these guys have never been in a room that big and have never been this close to winning an NBA title."

The Lakers also will play four of their first five games at Staples Center – and it will be the second time they’ve had a home opener there.

When it is over, the players and coaches said they will be eager to enjoy the night with their families, even if it was just for a few hours.

"It’s a great opportunity for a lot of us," said Kobe Bryant, who was among the last to leave the court after the game. "It’s a lot of fun to be back home."

Bryant was on the court for a couple of minutes on his own with teammates, and he looked as though he was in high spirits after a night where he had 17 points and eight rebounds in 26 minutes.

"I was happy I could get a little bit of an

Legolas and Gimli advanced on the orcs, raising their weapons with a harrowing war cry.

They were met by orcs in armor, their weapons drawn and pointing in Gimli’s direction. "I’m not going to kill you," the orc commander warned, "I’m going to kill you."

"It’s no use! This is a battle of wits," Gimli replied.

"I’m going to give you a choice: surrender, or die," the orc leader explained, "and you can’t win."

"You want me to choose?" Gimli challenged.

"I’m going to kill you," the orc commander assured Gimli.

Gimli’s reply was instantaneous: "I’m not going to kill you!"

"But you’ll die for it," the orc commander reminded him.

Gimli drew his sword and leapt towards the orc commander, intent on cutting him down before he could make good his threat.

Suddenly, the orc commander’s

I put in a sentence about cats and the rest of the ‘generated’ text had one sentence about how cats are well behaved domesticated animals, but the rest of it was about how gentlemen prefer ladies with large bosoms. Dangerous this ain’t, more like a dumb novelty.

I was bored so I typed in the first part of a Lennon song. This is supposed to complete poems, right? Output is gibberish that has no concept of the meaning of the words heaven/hell. Just randomly putting together nouns and verbs.

Input:
Imagine there’s no heaven
It’s easy if you try
No hell below us
Above us only sky

Output:
It’s easy to say that we want
No heaven below us
It’s easy to say that we want
No heaven above us
The way to heaven
(If it’s you, my love)
No heaven below us
(It’s easy to say that we want)
No hell above us
(If it’s you, my love)
The way to heaven
It’s easy if you try
There is no heaven below us
It’s easy to say that we want
There is no heaven above us
No hell below us
It’s easy to say that we want
There is no heaven below us
It’s easy to say that we want
There is no heaven above us

Tricia punched Tony in the crotch. She hit him again. Again. He let out a loud fart. She hit him again. Tony got up and began to walk away but her fist hit him again. Tony tried to walk away but she grabbed his arm and made him keep walking. He said to her, "It’s my property, I can do what I want. If you want to have sex with me then that’s fine with me, if not then that’s fine with you, but I will not go away or stop being my man. If you want to have sex with me, do whatever you want. But I am not going to stop being a man, so if you want to have sex with me then you can have sex with me. If you don’t want to have sex with me, then don’t."

Then, they got into a fight on the floor, and he said, "No, no, no. I want you to take the key,
-—————
Pulitzer, here I come.

More like Nobel Prize for Literature.

Donald Trump has announced plans to nuke Australia "off the face of the planet" to protect the Great Barrier Reef, which he says he has seen in his own life as a businessman.

Mr Trump, who was speaking at a rally in North Carolina, said he would "get out and get it over with" if the US was unable to protect the reef.

The president-elect said he would use a "major, major nuclear strike" on the Australian territory if it continued to block his plan to allow American coal ships to come into its port.

"Believe me. There is a problem with them. You see what’s happening," he said at a rally in Wilmington, North Carolina.

"Believe me. We’re going to get it done the first day. We are going to get it done the first day. I’ve spoken to the president of China, who I have a great relationship with. We’re going to work it out.

"Believe me. They understand it

View All Comments
Back to top ↑