You might not have noticed, but over the weekend a little coup took place. On Friday night, in front of a crowd of thousands, an AI bot beat a professional human player at Dota 2 — one of the world’s most popular video games. The human champ, the affable Danil "Dendi" Ishutin, threw in the towel after being killed three times, saying he couldn’t beat the unstoppable bot. “It feels a little bit like human,” said Dendi. “But at the same time, it’s something else.”
The bot’s patron was none other than tech billionaire Elon Musk, who helped found and fund the institution that designed it, OpenAI. Musk wasn’t present, but made his feelings known on Twitter, saying: “OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go.” Even more exciting, said OpenAI, was that the AI had taught itself everything it knew. It learned purely by playing successive versions of itself, amassing “lifetimes” of in-game experience over the course of just two weeks.
But how big a deal is all this? Was Friday night’s showdown really more impressive than Google’s AI victories at the board game Go? The short answer is probably not, but it still represents a significant step forward — both for the world of e-sports and the world of artificial intelligence.
Yes, video games are more complex than chess
First, we need to look at Musk’s claim that Dota is “vastly more complex than traditional board games like chess & Go.” This is completely true. Real-time battle and strategy games like Dota and Starcraft II pose major challenges that computers just can’t handle yet. Not only do these games demand long-term strategic thinking, but — unlike board games — they keep vital information hidden from players. You can see everything that’s happening on a chess board, but you can’t in a video game. This means you have to predict and preempt what your opponent will do. It takes imagination and intuition.
In Dota, this complexity is increased as human players are asked to work together in teams of five, coordinating strategies that will change on the fly based on which characters players choose. To make things even more complex, there are more than 100 different characters in-game, each with their own unique skill set; and characters can be equipped with a number of unique items, each of which can be game-winning if deployed at the right moment. All this means it’s basically impossible to comprehensively program winning strategies into a Dota bot.
But, the game that OpenAI’s bot played was nowhere near as complex as all this. Instead of 5v5, it took on humans at 1v1; and instead of choosing a character, both human and computer were limited to the same “hero” — a fellow named the Shadow Fiend, who has a pretty straightforward set of attacks. My colleague Vlad Savov, a confirmed Dota addict who also wrote up his thoughts on Friday’s match, said the 1v1 match represents “only a fraction of the complexity of the full team contest.” So: probably not as complex as Go.
You can’t beat a calculator at calculation
The second major caveat is knowing what advantages OpenAI’s agent had over its human opponents. One of the major points of discussion in the AI community was whether or not the bot had access to Dota’s bot API — which would let it tap directly into streams of information from the game, like the distances between players. OpenAI’s Greg Brockman confirmed to The Verge that the AI did indeed use the API, and that certain techniques were hardcoded in the agent, including the items it should use in the game. It was also taught certain strategies (like one called “creep block”) using a trial-and-error technique known as reinforcement learning. Basically, it did get a little coaching.
Andreas Theodorou, a games AI researcher at the University of Bath and an experienced Dota player, explains why this makes a difference. “One of the main things in Dota is that you need to calculate distances to know how far some [attacks] travel,” he says. “The API allows bots to have specific indications of range. So you can say, ‘If someone is in 500 meters range, do that,’ but the human player has to calculate it themselves, learning through trial and error. It really gives them an advantage if they have access to information that a human player does not.” This is particularly true in a 1v1 setting with a hero like Shadow Fiend; where players have to focus on timing their attacks correctly, rather than overall strategy.
Brockman’s response is that this sort of skill is trivial for an AI to learn, and was never the focus of OpenAI’s research. He says the institute’s bot could have done without information from the API, but “you’d just be spending a lot more of your time learning to do vision, which we already know works, so what’s the benefit?”
Some skills aren’t taught — they’re learned
So, knowing all this, should we dismiss OpenAI’s victory? Not at all, says Brockman. He points out that, perhaps more important than the bot’s victory, was how it taught itself in the first place. While previous AI champions like AlphaGo have learned how to play games by soaking up past matches by human champions, OpenAI’s bot taught itself (nearly) everything it knows.
“You have this system that has just played against itself, and it has learned robust enough strategies to beat the top pros. That’s not something you should take for granted,” says Brockman. “And it’s a big question for any machine learning system: how does complexity get into the model? Where does it come from?”
As OpenAI’s Dota bot shows, he says, we don’t have to teach computers complexity: they can learn it themselves. And although some of the bot’s behavior was preprogrammed, it did develop some strategies by itself. For example, it learned how to fake out its opponents by pretending to trigger an attack, only to cancel at the last second, leaving the human player to dodge an attack that never comes — exactly like a feint in boxing.
Others, though, are still a little skeptical. AI researcher Denny Britz, who wrote a popular blog post that put the victory in context, tells The Verge that it’s difficult to judge the scale of this achievement without knowing more technical details. (Brockman says these are forthcoming, but couldn’t give an exact time frame.) “It’s not clear what the technical contribution is at this point before the paper comes out,” says Britz.
Theodorou points out that although OpenAI’s bot beat Dendi onstage, once players got a good look at its tactics, they were able to outwit it. “If you look at the strategies they used, they played outside the box a bit and they won,” he says. The players used offbeat strategies — the sort that wouldn’t faze a human opponent, but which the AI had never seen before. “It didn’t look like the bot was flexible enough,” says Theodorou. (Brockman counters that once the bot learned these strategies, it wouldn’t fall for them twice.)
All the experts agree that this was a major achievement, but that the real challenge is yet to come. That will be a 5v5 match, where OpenAI’s agents have to manage not just a duel in the middle of the map, but a sprawling, chaotic battlefield, with multiple heroes, dozens of support units, and unexpected twists. Brockman says that OpenAI is currently targeting next year’s grand Dota tournament in 12 months’ time to pull this off. Between now and then, there’s much more training to be done.