Google's DeepMind AI can beat 49 Atari video games. That may not sound like much, but it taught itself to play — a significant step for machine learning.
Google acquired DeepMind Technologies last year, beating out Facebook on the bid. At the time, there was speculation about whether the acquisition would matter for search or robotics. The answer may be both: the machine learning techniques that led to today's study — published in the journal Nature — let computers discover patterns in data. And while the methods used to help the computers learn to play video games have been around for "several decades," according to an accompanying editorial, they hadn't been combined in such a useful way before.
"The approach displays impressive adaptability.""The approach displays impressive adaptability," writes Bernhard Schölkopf, the director of the Max Planck Institute for Intelligent Systems. Although each system only got trained on one game, each one learned the other 48 readily, he wrote. The games the computers were learning are a better model of real-world chaos than previous games computers have mastered, like chess, Schölkopf says in his editorial. That's good news for self-driving cars and other kinds of machines that will have to interact with the world — it brings Google a step closer to being able to handle noise and complexity.
The learning relied on an old technique: positive reinforcement. Whenever a computer beat a high score or moved on to a new level, it was rewarded. The AI performed better than previous methods in 43 games, and it did better than humans in 29 of them. Here's a video of how the AI learned to beat Breakout — notice how its strategy evolves as it plays more:
The positive reinforcement technique is noteworthy in that it's a form of human and animal learning, not one that's typically used for computers, write the study authors, led by Demis Hassabis, a co-founder of DeepMind and vice president of engineering at Google. Using biologically inspired techniques may be a new route for those hoping to create artificial intelligence, the study says. "Taken together, our work illustrates the power of harnessing state-of-the-art machine learning techniques with biologically inspired mechanisms to create agents that are capable of learning to master a diverse array of challenging tasks," the authors write. One day, that may mean something more complex than an Atari game.