Skip to main content

The Last Jedi Russian troll study shows we still don’t know how to interpret speech online

The intersection of politics, pop culture, and social media is a dangerous place

Share this story

Earlier this week, a researcher at the USC Annenberg School for Communication and Journalism released a paper about the online backlash against Star Wars: The Last Jedi. Author Morten Bay had analyzed tweets sent to director Rian Johnson after The Last Jedi’s release, and he’d found something alarming: more than half the people sending angry tweets to director Rian Johnson were bots, trolls, or activists using The Last Jedi to “propagate political messages supporting extreme right-wing causes.” Even worse, “a number of these users appear to be Russian trolls.”

That last detail caught the attention of lots of reporters, including ones at The Hollywood Reporter, ABC News, and The Verge. Russian “troll farms” were caught sowing discord around the 2016 presidential election, and The Last Jedi was a divisive film, so it wasn’t surprising that they might have weaponized that division. But there wasn’t actually much evidence for this theory — and it wasn’t the paper’s main focus, either.

The paper only claims to have found 16 Russia-linked accounts, and these were identified using circumstantial evidence like posting patterns, not direct ties to known trolls. The paper was mainly designed to gauge how much Last Jedi backlash was fueled by partisan politics in general, and Bay says he was bemused by the resulting coverage. “I knew that people were going to jump onto it. But I’m honestly a little surprised that people have been this uncritical,” he tells The Verge.

Not every outlet got the detail about Russian trolls wrong. CNET specifically rebutted the misleading reports, and Polygon focused on the general issue of politicization.

But there’s a deeper problem with saying political operatives weaponized The Last Jedi — a framing that Bay’s paper and most reports on it have accepted. At a time when pop culture is incredibly intertwined with politics, it suggests that talking about politics in pop culture is artificial and manipulative. And the paper’s most interesting implication may not be that there was a strategic attempt to politicize the Last Jedi debate, but that the internet makes it incredibly hard to gauge how deep political divisions run — because anybody on Twitter can sound important, even when they’re not.

Here’s how Bay’s methodology worked: he collected tweets from 967 accounts and manually sorted them into positive, negative, and neutral categories. Based on the content of the tweets and the larger context of users’ Twitter timelines, he concluded that 101 of the negative comments were “purely motivated by a negative stance towards the film.”

The remaining 105 accounts were divided into three categories: 11 automated bots, 33 seemingly inauthentic “sock puppets” or trolls, and 61 humans who had “a political agenda.” Of those bots, sock puppets, and trolls, 16 had a combination of traits that mirror known Russia-linked accounts.

Bay admits this isn’t an exhaustive analysis — “I’m a lowly researcher, I’m not the CIA,” he says. But Navied Shoushtarian, co-founder and CTO of social media analytics company Dovetale, says it’s similar to the way his company classifies bots, although he and Dovetale co-founder Mike Schmidt think Bay was “potentially grasping too much” at the clues.

Schmidt also says that this wouldn’t prove that trolls specifically targeted The Last Jedi, since their goal is to “stir the pot” by covering a whole spread of controversial topics. “One minute you’re talking about the Venezuelan elections and the second minute you’re talking about cryptocurrencies,” he says. “Maybe it’s Star Wars, maybe it’s Deadpool.”

A lot of real, individual humans decided to express genuinely held political views about a movie that they didn’t care much about

Bay agrees that Star Wars might have just been one item on a list of topics, and in the phone interview, he calls the 16 allegedly Russian accounts “a small little curiosity” in the larger scope of his results. There’s no hard evidence that they meaningfully amplified the backlash against Johnson, either. “It’s not so much their influence that made this blow up — it was the general troll and political activist activity,” he says.

“Political activist” covers more than half of Bay’s manipulative accounts. Bay defines it as an account that tweets overwhelmingly about partisan political topics, particularly the threat of feminist or pro-LGBT “social justice warriors,” and then “all of a sudden jumps on board the Star Wars fandom and inserts that political opinion into it,” firing off some anti-Last Jedi tweets but otherwise showing no interest in the series. “If somebody is a fan and clearly tweets a lot about Star Wars and did so before The Last Jedi came out, and also expresses political opinions, I don’t put them in the political activist category.”

The paper doesn’t indicate an explicitly coordinated campaign to create an artificial political dispute. It suggests that a lot of real, individual humans decided to express genuinely held political views about a movie that they didn’t care much about. But the structure of the study also establishes these two categories as part of a single phenomenon of “deliberate, organized political influence measures,” or as the paper title puts it, “social media manipulation.” That’s a much more sweeping suggestion.

The paper portrays “activism” as a strategic attempt to make the world of fandom seem more contentious than it really is by slipping in political tweets “disguised as fan arguments.” But Star Wars is one of the biggest media franchises ever created, and the study doesn’t allege that users were presenting themselves as dedicated fans, rather than ordinary moviegoers with a strong opinion on a popular film. Even the term “fan” — especially the line between “fan” and “consumer” — can be ambiguous, although Bay is using it to broadly describe people with any sustained interest in Star Wars.

As it stands, Bay’s “political activist” category sounds like it would also cover a Twitter user who talks mostly about feminist politics, but takes time to call out sexism in a movie series she doesn’t normally follow (or tweet about). That’s uncomfortably close to some arguments frequently leveled at so-called “social justice warriors,” who get accused of injecting politics into fun games or movies, then dismissed as not being “real” fans with sufficient knowledge of these franchises.

But to be fair, in the larger context of Gamergate and the online culture wars that have followed, comparing good-faith political critique to the wave of tweets Bay is analyzing looks naive. We’ve seen reactionary political figures claim to be spokespeople for groups they’ve previously disparaged, and angry online communities coordinate deceptive review-bombings of games and movies. And Bay’s larger argument makes sense even if — or perhaps especially if — The Last Jedi backlash was fueled by unorganized political junkies instead of calculating foreign operatives.

You don’t need a sophisticated bot network on a platform that’s so easy to game

“One of the things that also came out of the paper was to say, these are just very loud people, and they’re just trying to muddy the waters here. We shouldn’t pay attention to them,” says Bay. The Last Jedi backlash got widespread press coverage simply for a few bizarre (and, admittedly, very funny) stunts like a crowdfunded “Remake The Last Jedi” campaign. Twitter specifically has produced a form of journalism that treats a small cluster of online missives as a meaningful social trend. Its quote-tweet function is like a disease vector for bad takes, as generally non-toxic users amplify somebody else’s awful tweet to mock or rebut it. You don’t need a sophisticated bot network on a platform that’s so easy to game.

These angry users can make other people’s lives miserable. As Bay notes, The Last Jedi star Kelly Marie Tran suffered persistent, vicious abuse from people who hated the film, which was enough to drive her off of social media entirely. That just doesn’t make their opinions automatically valuable or important — whether they’re a real person or a Russian bot. “It’s easier to hide behind an avatar and say the most outrageous things,” says Bay. “I think we need to be able to say, ‘Well, okay, you said this. Who are you?’”

And people are already starting to realize that a sincere person isn’t behind every account. “This is most certainly a cultural shift, given how much activity there is,” says Dovetale’s Schmidt. “[P]eople are now engaged with politics more than they ever have been,” adds Shoushtarian. “I would say that it’s definitely a shift in that culture, too, where people are now, more or less being part of the discord of conversation of the society you’re part of.”

Update 3:10PM ET: Clarified the description of trolls, bots, and political activists.