The film world is full of intriguing what-ifs. Will Smith famously turned down the role of Neo in The Matrix. Nicolas Cage was cast as the lead in Tim Burton’s Superman Lives, but he only had time to try on the costume before the film was canned. Actors and directors are forever glancing off projects that never get made or that get made by someone else, and fans are left wondering what might have been.
For the people who make money from movies, that isn’t good enough.
If casting Alicia Vikander instead of Gal Gadot is the difference between a flop and smash hit, they want to know. If a movie that bombs in the US would have set box office records across Europe, they want to know. And now, artificial intelligence can tell them.
artificial intelligence turns filmmaking into fantasy football
Los Angeles-based startup Cinelytic is one of the many companies promising that AI will be a wise producer. It licenses historical data about movie performances over the years, then cross-references it with information about films’ themes and key talent, using machine learning to tease out hidden patterns in the data. Its software lets customers play fantasy football with their movie, inputting a cast, then swapping one actor for another to see how this affects a film’s projected box office.
Say you have a summer blockbuster in the works with Emma Watson in the lead role, says Cinelytic co-founder and CEO Tobias Queisser. You could use Cinelytic’s software to see how changing her for Jennifer Lawrence might change the film’s box office performance.
“You can compare them separately, compare them in the package. Model out both scenarios with Emma Watson and Jennifer Lawrence, and see, for this particular film … which has better implications for different territories,” Queisser tells The Verge.
Cinelytic isn’t the only company hoping to apply AI to the business of film. In recent years, a bevy of firms has sprung up promising similar insights. Belgium’s ScriptBook, founded in 2015, says its algorithms can predict a movie’s success just by analyzing its script. Israeli startup Vault, founded the same year, promises clients that it can predict which demographics will watch their films by tracking (among other things) how its trailers are received online. Another company called Pilot offers similar analyses, promising it can forecast box office revenues up to 18 months before a film’s launch with “unrivaled accuracy.”
The water is so warm, even established companies are jumping in. Last November, 20th Century Fox explained how it used AI to detect objects and scenes within a trailer and then predict which “micro-segment” of an audience would find the film most appealing.
Looking at the research, 20th Century Fox’s methods seem a little hit or miss. (Analyzing the trailer for 2017’s Logan, the company’s AI software came up with the following, unhelpful tags: “facial_hair,” “car,” “beard,” and — the most popular category of all — “tree.”) But Queisser says the introduction of this technology is overdue.
“On a film set now, it’s robots, it’s drones, it’s super high-tech, but the business side hasn’t evolved in 20 years.”
“On a film set now, it’s robots, it’s drones, it’s super high-tech, but the business side hasn’t evolved in 20 years,” he says. “People use Excel and Word, fairly simplistic business methods. The data is very siloed, and there’s hardly any analytics.”
That’s why Cinelytic’s key talent comes from outside Hollywood. Queisser used to be in finance, an industry that’s embraced machine learning for everything from high-speed trading to calculating credit risk. His co-founder and CTO, Dev Sen, comes from a similarly tech-heavy background: he used to build risk assessment models for NASA.
“Hundreds of billions of dollars of decisions were based on [Sen’s work],” says Queisser. The implication: surely the film industry can trust him as well.
But are they right to? That’s a harder question to answer. Cinelytic and other companies The Verge spoke to declined to make any predictions about the success of upcoming movies, and academic research on this topic is slim. But ScriptBook did share forecasts it made for movies released in 2017 and 2018, which suggest the company’s algorithms are doing a pretty good job. In a sample of 50 films, including Hereditary, Ready Player One, and A Quiet Place, just under half made a profit, giving the industry a 44 percent accuracy rate. ScriptBook’s algorithms, by comparison, correctly guessed whether a film would make money 86 percent of the time. “So that’s twice the accuracy rate of what the industry achieved,” ScriptBook data scientist Michiel Ruelens tells The Verge.
An academic paper published on this topic in 2016 similarly claimed that reliable predictions about a movie’s profitability can be made using basic information like a film’s themes and stars. But Kang Zhao, who co-authored the paper along with his colleague Michael Lash, cautions that these sorts of statistical approaches have their flaws.
One is that the predictions made by machines are frequently just blindingly obvious. You don’t need a sophisticated and expensive AI software to tell you that a star like Leonardo DiCaprio or Tom Cruise will improve the chances of your film being a hit, for example.
Algorithms are also fundamentally conservative. Because they learn by analyzing what’s worked in the past, they’re unable to account for cultural shifts or changes in taste that will happen in the future. This is a challenge throughout the AI industry, and it can contribute to problems like AI bias. (See, for example, Amazon’s scrapped AI recruiting tool that penalized female candidates because it learned to associate engineering prowess with the job’s current male-dominated intake.)
Because AI learns from past data, it can’t predict future cultural shifts
Zhao offers a more benign example of algorithmic shortsightedness: the 2016 action fantasy film Warcraft, which was based on the MMORPG World of Warcraft. Because such game-to-movie adaptations are rare, he says, it’s difficult to predict how such a film would perform. The film did badly in the US, taking in only $24 million in its opening weekend. But it was a huge hit in China, becoming the highest grossing foreign language film in the country’s history.
Who saw that coming? Not the algorithms.
There are similar stories in ScriptBook’s predictions for 2017 / 2018 movies. The company’s software correctly greenlit Jordan Peele’s horror hit Get Out, but it underestimated how popular it would be at the box office, predicting $56 million in revenue instead of the actual $176 million it made. The algorithms also rejected The Disaster Artist, the tragicomic story of Tommy Wiseau’s cult classic The Room, starring James Franco. ScriptBook said the film would make just $10 million, but it instead took in $21 million — a modest profit on a $10 million film.
As Zhao puts it: “We are capturing only what can be captured by data.” To account for other nuances (like the way The Disaster Artist traded on the memeiness of The Room), you have to have humans in the loop.
Andrea Scarso, a director at the UK-based Ingenious Group, agrees. His company uses Cinelytic’s software to guide investments it makes in films, and Scarso says the software works best as a supplementary tool.
“Sometimes it validates our thinking, and sometimes it does the opposite.”
“Sometimes it validates our thinking, and sometimes it does the opposite: suggesting something we didn’t consider for a certain type of project,” he tells The Verge. Scarso says that using AI to play around with a film’s blueprint — swapping out actors, upping the budget, and seeing how that affects a film’s performance — “opens up a conversation about different approaches,” but it’s never the final arbiter.
“I don’t think it’s ever changed our mind,” he says of the software. But it has plenty of uses all the same. “You can see how, sometimes, just one or two different elements around the same project could have a massive impact on the commercial performance. Having something like Cinelytic, together with our own analytics, proves that [suggestions] we’re making aren’t just our own mad ideas.”
But if these tools are so useful, why aren’t they more widely used? ScriptBook’s Ruelens suggests one un-Hollywood characteristic might be to blame: bashfulness. People are embarrassed. In an industry where personal charisma, aesthetic taste, and gut instinct count for so much, turning to the cold-blooded calculation of a machine looks like a cry for help or an admission that you lack creativity and don’t care about a project’s artistic value.
Ruelens says ScriptBook’s customers include some of the “biggest Hollywood studios,” but nondisclosure agreements (NDAs) prevent him from naming any. “People don’t want to be associated with these AIs yet because the general consensus is that AI is bad,” says Ruelens. “Everyone wants to use it. They just don’t want us to say that they’re using it.” Queisser says similar agreements stop him from discussing clients, but that current customers include “large indie companies.”
Hollywood is unlikely to accept AI having the final say anytime soon
Some in the business push back against the claim that Hollywood is embracing AI to vet potential films, at least when it comes to actually approving or rejecting a pitch. Alan Xie, CEO of Pilot Movies, a company that offers machine learning analytics to the film industry, tells The Verge that he’s “never spoken to an American studio executive who believes in [AI] script analysis, let alone [has] integrated it into their decision-making process.”
Xie says it’s possible studios simply don’t want to talk about using such software, but he says script analysis, specifically, is an imprecise tool. The amount of marketing spend and social media buzz, he says, are a much more reliable predictor of box office success. “Internally at Pilot, we’ve developed box office forecast models that rely on script features, and they’ve performed substantially worse than models that rely on real-time social media data,” he says.
Despite skepticism about specific applications, the tide might be turning. Ruelens and investment director Scarso say a single factor has convinced Hollywood to stop dismissing big data: Netflix.
The streaming behemoth has always bragged about its data-driven approach to programming. It surveils the actions of millions of subscribers in great detail and knows a surprising amount about them — from which thumbnail will best convince someone to click on a movie to the choices they make in Choose Your Own Adventure-style tales like Black Mirror: Bandersnatch. “We have one big global algorithm, which is super-helpful because it leverages all the tastes of all consumers around the world,” said Netflix’s head of product innovation, Todd Yellin, in 2016.
It’s impossible to say whether Netflix’s boasts are justified, but the company claims its recommendation algorithm alone is worth $1 billion a year. (It surely doesn’t hurt that such talk puts fear into the competition.) Combined with its huge investments into original content, it’s enough to make even the most die-hard Hollywood producer reach for a fortifying algorithm.
Ruelens says the transformation has been noticeable. “When we started out four years ago, we had meetings with big companies in Hollywood. They were all super skeptical. They said ‘We have [decades] of expertise in the industry. How can this machine tell us what to do?’” Now, things have changed, he says. The companies did their own validation studies, they waited to see which predictions the software got right, and, slowly, they learned to trust the algorithms.
“They’re starting to accept our technology,” says Ruelens. “It just took time for them to see.”
Correction Wednesday May 29th, 04:00AM ET: An earlier version of this piece suggested that Cinelytic’s software analyzes scripts. That is incorrect; it only uses film summaries to organize data. We regret the error.