Skip to main content

WSJ’s deep dive into eating disorder rabbit holes on TikTok explains a sudden policy change

WSJ’s deep dive into eating disorder rabbit holes on TikTok explains a sudden policy change

/

TikTok’s changes come just days after the WSJ says it asked for a statement about its upcoming story

Share this story

The TikTok logo repeated across a red background
Illustration by Alex Castro / The Verge

A troubling report from the Wall Street Journal digs into the personal experiences of young girls who were sent down rabbit holes of extreme weight loss challenges, purging techniques, and deadly diets through TikTok, contributing to the development of eating disorders, or making existing ones worse. The WSJ did its own experiment to see how TikTok’s algorithm can potentially promote this kind of harmful content — its findings may explain TikTok’s sudden decision to alter the way its video recommendation system operates.

As detailed in the report, the WSJ created over 100 accounts “that browsed the app with little human intervention,” 12 of which were bots registered to 13-year-olds that spent time on videos surrounding weight loss, alcohol, and gambling. A chart included in the report shows that as soon as one of the bots abruptly stopped watching gambling-related videos and began spending time on videos about weight loss instead, TikTok’s algorithm adjusted itself accordingly; it quickly increased the number of weight loss videos that the bot saw to account for this shift in behavior.

“While this experiment does not reflect the experience most people have on TikTok, even one person having that experience is one too many”

By the end of its experiment, the WSJ found that of the 255,000 videos that the bots watched in total, 32,700 of those contained a description or metadata that matched a list of hundreds of keywords pertaining to weight loss. 11,615 videos had text descriptions that matched keywords relevant to eating disorders, while 4,402 had a combination of keywords that indicated the normalization of eating disorders.

A number of these videos were reportedly using different spellings for eating disorder-related keywords to avoid getting flagged by TikTok. After the WSJ alerted the platform to a sample of 2,960 eating disorder-related videos, 1,778 were removed — the WSJ says it’s unclear whether it was taken down by TikTok or the creators themselves.

Just one day before the WSJ’s report dropped, TikTok announced that it’s working on new ways to stop these dangerous rabbit holes from forming. This change also occurred just days after the WSJ says it contacted TikTok for a statement about its upcoming story, so it’s possible that TikTok preemptively rolled out the update before the report was published.

In its post, TikTok acknowledges that it isn’t always healthy to view certain kinds of content over and over again, including videos related to extreme dieting and fitness. It’s now working on a way to recognize whether its recommendation system is unintentionally serving up videos that may not break TikTok’s policies, but could be harmful if consumed excessively. The platform also says it’s testing a tool that will let users stop videos containing certain words or hashtags from showing up on their For You page.

“While this experiment does not reflect the experience most people have on TikTok, even one person having that experience is one too many,” said TikTok spokesperson, Jamie Favazza, in a statement to The Verge. “We allow educational or recovery-oriented content because we understand it can help people see there’s hope, but content that promotes, normalizes, or glorifies disordered eating, is prohibited.” The spokesperson also pointed out that TikTok provides access to the National Eating Disorder Association Hotline within the app.

If this situation seems familiar to you, it’s because Instagram has already been through it (and is still dealing with it). After whistleblower Francis Haugen leaked the Facebook Papers, a collection of revealing internal Facebook documents, Instagram worked quickly to patch the holes in its sinking ship.

The papers show that Facebook did its own research on Instagram’s impact on teens, and found that the app could wreak havoc on teens’ mental health, and also made young girls’ body image worse. About a month later, Instagram announced its plans to introduce a feature that will “nudge” teens away from viewing potentially harmful content. It also rolled out a “take a break” feature that prompts users to close the app if they’ve spent a set amount of time on the platform, ranging from 10, 20, or 30 minutes.

Today’s Storystream

Feed refreshed Two hours ago The tablet didn’t call that play by itself

T
Thomas RickerTwo hours ago
The Simpsons pays tribute to Chrome’s dino game.

Season 34 of The Simpsons kicked off on Sunday night with an opening credits “couch gag” based on the offline dino game from Google’s Chrome browser. Cactus, cactus, couch, d’oh! Perfect.


T
Youtube
Thomas Ricker7:29 AM UTC
Table breaks before Apple Watch Ultra’s sapphire glass.

”It’s the most rugged and capable Apple Watch yet,” said Apple at the launch of the Apple Watch Ultra (read The Verge review here). YouTuber TechRax put that claim to the test with a series of drop, scratch, and hammer tests. Takeaways: the titanium case will scratch with enough abuse, and that flat sapphire front crystal is tough — tougher than the table which cracks before the Ultra fails — but not indestructible.


E
Twitter
Emma RothSep 25
Rihanna’s headlining the Super Bowl Halftime Show.

Apple Music’s set to sponsor the Halftime Show next February, and it’s starting out strong with a performance from Rihanna. I honestly can’t remember which company sponsored the Halftime Show before Pepsi, so it’ll be nice to see how Apple handles the show for Super Bowl LVII.


Welcome to the new Verge

Revolutionizing the media with blog posts

Nilay PatelSep 13
E
Twitter
Emma RothSep 25
Starlink is growing.

The Elon Musk-owned satellite internet service, which covers all seven continents including Antarctica, has now made over 1 million user terminals. Musk has big plans for the service, which he hopes to expand to cruise ships, planes, and even school buses.

Musk recently said he’ll sidestep sanctions to activate the service in Iran, where the government put restrictions on communications due to mass protests. He followed through on his promise to bring Starlink to Ukraine at the start of Russia’s invasion, so we’ll have to wait and see if he manages to bring the service to Iran as well.


E
External Link
Emma RothSep 25
We might not get another Apple event this year.

While Apple was initially expected to hold an event to launch its rumored M2-equipped Macs and iPads in October, Bloomberg’s Mark Gurman predicts Apple will announce its new devices in a series of press releases, website updates, and media briefings instead.

I know that it probably takes a lot of work to put these polished events together, but if Apple does pass on it this year, I will kind of miss vibing to the livestream’s music and seeing all the new products get presented.


E
External Link
Emma RothSep 24
California Governor Gavin Newsom vetoes the state’s “BitLicense” law.

The bill, called the Digital Financial Assets Law, would establish a regulatory framework for companies that transact with cryptocurrency in the state, similar to New York’s BitLicense system. In a statement, Newsom says it’s “premature to lock a licensing structure” and that implementing such a program is a “costly undertaking:”

A more flexible approach is needed to ensure regulatory oversight can keep up with rapidly evolving technology and use cases, and is tailored with the proper tools to address trends and mitigate consumer harm.


A
Youtube
Andrew WebsterSep 24
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.


A
The Verge
Andrew WebsterSep 24
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.


A
Andrew WebsterSep 24
Looking for something to do this weekend?

Why not hang out on the couch playing video games and watching TV. It’s a good time for it, with intriguing recent releases like Return to Monkey Island, Session: Skate Sim, and the Star Wars spinoff Andor. Or you could check out some of the new anime on Netflix, including Thermae Romae Novae (pictured below), which is my personal favorite time-traveling story about bathing.


A screenshot from the Netflix anime Thermae Romae Novae.
Thermae Romae Novae.
Image: Netflix