20th Century Fox is using AI to analyze movie trailers and find out what films audiences will like

2017’s dark superhero flick Logan was one of the case studies.
Ben Rothstein, 20th Century Fox

Machine learning is, at heart, the art of finding patterns in data. That’s why businesses love it. Patterns help predict the future, and predicting the future is a great way to make money. It’s sometimes unclear how these things fit together, but here’s a perfect example from film studio 20th Century Fox, which is using AI to predict what films people will want to see.

Researchers from the company published a paper last month explaining how they’re analyzing the content of movie trailers using machine learning. Machine vision systems examine trailer footage frame by frame, labeling objects and events, and then compare this to data generated for other trailers. The idea is that movies with similar sets of labels will attract similar sets of people.

As the researchers explain in the paper, this is exactly the sort of data movie studios love. (They already produce lots of similar data using traditional methods like interviews and questionnaires.) “Understanding detailed audience composition is important for movie studios that invest in stories of uncertain commercial,” they write. In other words, if they know who watches what, they will know what movies to make.

It’s even better if this audience composition can be broken down into smaller and more accurate “micro segments.” A good example of this is 2017’s Logan. It’s a superhero movie, yes, but it has darker themes and a plot that attracts a slightly different audience. So can AI be used to capture those differences? The answer is: sort of.

To create their “experimental movie attendance prediction and recommendation system” (named Merlin), 20th Century Fox partnered with Google to use the company’s servers and open-source AI framework TensorFlow. In an accompanying blog post, the search giant explains Merlin’s analysis of Logan.

First, Merlin scans the trailer, labeling objects like “facial hair,” “car,” and “forest”:

While this graph only records the frequency of these labels, the actual data generated is more complex, taking into account how long these objects appear on-screen and when they show up the trailer.

As 20th Century Fox’s engineers explain, this temporal information is particularly rich because it correlates with a film’s genre. “For example,” they write, “a trailer with a long close-up shot of a character is more likely for a drama movie than for an action movie, whereas a trailer with quick but frequent shots is more likely for an action movie.” This definitely holds true for Logan, with its trailer featuring lots of slow shots of Hugh Jackman looking bloody and beaten.

By comparing this information with analyses of other trailers, Merlin can try to predict what films might interest the people who saw Logan. But here’s where things get a little dicey.

The graph below shows the top 20 films that people who went to see Logan also watched. The column on the right shows Merlin’s predictions, and the column on the left shows the actual data (collected, one assumes, by using that pre-AI method of “asking people”).

Merlin gets quite a few of the films correct, including other superhero movies like X Men: Apocalypse, Doctor Strange, and Batman v Superman: Dawn of Justice. It even correctly identifies John Wick: Chapter 2 as a bedfellow of Logan. That’s an impressive intuition since John Wick is certainly not a superhero movie. However, it does feature a similarly weary and jaded protagonist with a comparably rugged look. All in all, Merlin identifies all of the top five picks, even if it does fail to put them in the same order of importance.

What’s more revealing, though, are the mismatches. Merlin predicts that The Legend of Tarzan will be a big hit with Logan fans for example. Neither Google nor 20th Century Fox offers an explanation for this, but it could it have something to do with the “forest,” “tree,” and “light” found in Logan — elements which also feature heavily in the Tarzan trailer.

Similarly, The Revenant has plenty of flora and facial hair, but it was drama-heavy Oscar bait rather than a smart superhero movie. Merlin also misses Ant-Man and Deadpool 2 as lure’s for the same audience. These were superhero films with quick-cut trailers, but they also took offbeat approaches to their protagonists, similar to Wolverine’s treatment in Logan.

20th Century Fox presumably knew all of this already, but it’s interesting to see what artificial intelligence could and couldn’t spot. The movie industry has been keen to adopt AI for these sorts of analytics, and several companies already claim that they can predict the success of a movie just by using machine learning to dig through its script. Analyses like these show, though, that computers are not yet movie buffs. They need to spend longer at the cinema before they can truly predict our taste.

Comments

"But will it get them off their tractors?"

So movie making will not be an art in a few years….just check a box here and there and boom, script.

Movie making has not been an "art" since the blockbuster took over in the late 1970’s. It’s been a slow death for art in Hollywood movies since then.

Just look at the top grossing movies in 1976: https://www.the-numbers.com/market/1976/top-grossing-movies

vs. the top grossing movies last year: https://www.the-numbers.com/market/2017/top-grossing-movies

The top grossers used to be a mix of drama, comedy, thrillers, and yeah, a few popcorn flicks. But now it’s just 100% "event" films that are more amusement park ride than film. (In fact, many modern top grossing films become actual amusement park rides!)

So this AI thing is just a continuation of the process. The goal is just to make the most money possible, not to make art. That’s a lost ideal.

Why does the AI keep spitting out the number "8008S" for most common associated and highly liked label?

The obvious question here is why are they using trailers to compare to the actual films people went to see? Maybe there’s a reason but it doesn’t seem to be explained. It seems like if you wanted to predict what types of other films people who saw Logan liked, you’d compare the films themselves, not the trailer for Logan vs. other trailers.

Comparing only the trailers makes it seem like they think that trailers are some kind of silver bullet for marketing – everyone who sees a film does so because of the trailer, and everyone who sees a trailer they like automatically goes to see that film. But reality doesn’t work that way, and if they’re using this to try to make more money on the movies, I can’t see how it’s going to work. We all know that trailers may or may not show stuff that’s even in the movie.

and Logan even had a history with the shit and okay movies, promising the character justice finally. And it had the good hype, not just because the trailers but critic reaction and WOM after the premier.

View All Comments
Back to top ↑