Later this week a $200,000 tournament will pit an AI system built by Carnegie Mellon University against four of the world’s top pros in the game of No-Limit Texas Hold’em poker. The team from CMU was hoping to be the first to lay claim to an AI that could defeat the best humans in the world at the game. But a rival academic outfit claims to have just beaten them to the punch.
A group from the University of Alberta, in partnership with two Czech universities, has made public a paper on DeepStack, an AI system that it believes is the first to consistently beat professional players at this extremely popular and challenging game. It’s important to note that the paper has not yet been peer-reviewed, so it needs to be taken with a grain of salt. But the team behind it has a history of accomplishments in the space that lend credence to its claims.
In 2008 a group with many of the same researchers devised the first system that could beat top-level humans at Limit Texas Hold’em, a version of the game that has a far more constrained variety of possible bets. In 2015 that team laid claim to a system that played a near perfect version of Limit Hold’em.
In heads-up play, DeepStack was matched against 33 professional players from the International Federation of Poker. Over the course of 44,852 hands, the paper claims the program bested its human opponents by a wide margin. “Over all games played, DeepStack won 492 mbb/g. This is over 4 standard deviations away from zero, and so highly significant. Note that professional poker players consider 50 mbb/g a sizable margin,” the DeepStack team wrote.
But Professor Tuomas Sandholm, who helped design the system from CMU, argued that there is a still a major milestone at stake. “They were not playing against top pros, so one can’t say they bested mankind,” he wrote in an email to The Verge. Poker forums seem to agree.
“We could have done that two years ago. (Already then we had a bot that played the BEST humans to a statistical draw.) But that was not our goal,” Sandholm wrote. “Our goal has never been expert-level AI, but rather AI that is super-human level like Deep Blue and Watson.”
So far, in both Limit and No-Limit, these AI systems only attempt to solve for a “heads-up” game, a 1–1 match between two players. While many poker matches involve five or more players at once, that level of complexity is still far out of reach for AI. Still, the results make clear that at least two separate AI systems are closing in on a new milestone in gaming.
“DeepStack is a paradigmatic shift in approximating solutions to large, sequential imperfect information games,” the system’s creators wrote. Games like poker, where you don’t know what cards your opponent is holding, or what will be drawn next from the deck, are considered more challenging for AI than games like chess, where both players have access to a complete picture of the game. DeepStack’s achievement, the team writes, means that “for the first time the gap between the largest perfect and imperfect information games to have been mastered is mostly closed.”
Like AlphaGo before it, this system doesn’t try to solve the entire game, because there are simply too many possibilities to consider. Instead, both systems use deep learning to hone what they describe as an intuitive approach. According to its creators “[DeepStack] does not compute and store a complete strategy prior to play....It avoids reasoning about the entire remainder of the game by substituting the computation beyond a certain depth with a fast approximate estimate. This estimate can be thought of as DeepStack’s intuition: a gut feeling of the value of holding any possible private cards in any possible poker situation.”