Nvidia is best known for its graphics cards, but the company conducts some serious research into artificial intelligence, too. For its latest project, Nvidia researchers taught an AI system to recreate the game of Pac-Man simply by watching it being played.
There’s no coding involved, no pre-rendered images for the software to draw on. The AI model is simply fed visual data of the game in action along with the accompanying controller inputs and then recreates it frame by frame from this information. The resulting game is playable by humans, and Nvidia says it will be releasing it online in the near future.
“It learns all of these things just by watching”
The AI version is by no means a perfect facsimile, though. The imagery is blurry and it doesn’t seem like the AI managed to capture the exact behavior of the game’s ghosts, each of which is programmed with a specific personality that dictates its movement. But the basic dynamics of Pac-Man are all there: eat pellets, avoid ghosts, and try not to die.
“It learns all of these things just by watching,” Nvidia’s Rev Lebaredian, vice president of simulation technology, told journalists in a briefing. “[It’s] similar to how a human programmer can watch many episodes of Pac-Man on YouTube and infer what the rules of the games are and reconstruct them.”
Lebaredian said the work had been done in collaboration with Pac-Man’s creator, Bandai Namco, which is celebrating the 40th anniversary of the arcade classic today.
Nvidia says work like this shows how artificial intelligence will be used for game design in the future. Developers can input their work into the AI and use it to create variations or maybe design new levels. “You could use this to mash different games together,” Sanja Fidler, director of Nvidia’s Toronto research lab, told journalists, “giving additional power to games developers by [letting them] blend together different games.”
Creating AI that can learn the rules of a virtual world just by watching it in action also has implications for tasks like programming robots. “Eventually we’d like it to learn the rules of the real world,” says Lebaredian. The AI might watch videos of robotics trolleys navigating a warehouse, for example, and use that information to design navigation software of its own.
The program that recreated Pac-Man is called GameGAN. GAN stands for generative adversarial network and is a common architecture used in machine learning. The basic principle of a GAN is that it works in two halves. The first half of the GAN tries to replicate the input data, while the second half compares this to the original source. If they don’t match, the generated data is rejected and the generator tweaks its work and resubmits it.
Using AI to generate virtual worlds like video games has been done before. But Nvidia’s researchers introduced several new aspects, including a “memory module” that allowed the system to store an internal map of the game world. This leads to greater consistency in the game world, a key characteristic when recreating the mazes of Pac-Man. They also allow for the static elements of the game world (like the maze) to be separated from the dynamic ones (like the ghosts), which suits the company’s goal of using AI to generate new levels.
David Ha, an AI researcher at Google who’s worked on similar tasks, told The Verge that the research was “very interesting.” Earlier teams have tried to recreate game worlds using GANs, said Ha, “but from what I know, [this] is the first to demonstrate good results.”
“All in all, a very exciting paper, and I look forward to see more developments using this approach,” said Ha.
Some elements of the process definitely need tweaking, though, and demonstrate the particular fragility of artificial intelligence when learning new tasks. Fidler told journalists that to recreate Pac-Man, GameGAN had to be trained on some 50,000 episodes. Getting that gameplay data from humans wasn’t feasible, so the team used an AI agent to generate the data. Unfortunately, the AI agent was so good at the game that it hardly ever died.
“That made it hard for the AI trying to recreate the game to learn the concept of dying,” says Fidler. Instead, in early versions of the AI-generated Pac-Man, GameGAN tweaked the game so that ghosts never actually reached the title character but trail directly behind it like baby ducks following a parent. “It’s a funny effect of the way we trained it,” says Fidler.