Skip to main content

Facebook is building a hidden, bot-only platform to learn about trolls and scammers

Facebook is building a hidden, bot-only platform to learn about trolls and scammers


No ‘unexpected interactions’ between bots and humans

Share this story

Illustration by William Joel / The Verge

Facebook wants to stop people from abusing its system, so it’s making a world of bots that can imitate them. Company researchers have released a paper on a “Web Enabled Simulation” (WES) for testing the platform — basically a shadow Facebook where nonexistent users can like, share, and friend (or harass, abuse, and scam) away from human eyes.

Facebook describes building a scaled-down, walled-off simulation of its platform populated by fake users modeling different kinds of real behavior. For example, a “scammer” bot might be trained to connect with “target” bots that exhibit behaviors similar to real-life Facebook scam victims. Other bots might be trained to invade fake users’ privacy or seek out “bad” content that breaks Facebook’s rules.

Software simulations are obviously common, and Facebook is expanding on an earlier automated testing tool called Sapienz. But it calls WES systems distinct because they turn lots of bots loose on something very close to an actual social media platform, not a mockup mimicking its functions. While bots aren’t clicking around a literal app or webpage, they send actions like friend requests through Facebook code, triggering the same kinds of processes a real user would.

That could help Facebook detect bugs. Researchers can build WES users whose sole goal is stealing information from other bots, for example, and set them loose on the system. If they suddenly find ways to access more data after an update, that could indicate a vulnerability for human scammers to exploit, and no real users would have been affected.

Scammer bot, meet target bot

Some bots could get read-only access to the “real” Facebook, as long as they weren’t accessing data that violated privacy rules. Then they could react to that data in a purely read-only capacity. In other cases, however, Facebook wants to build up an entire parallel social graph. Within that large-scale fake network, they can deploy “fully isolated bots that can exhibit arbitrary actions and observations,” and they can model how users might respond to changes in the platform — something Facebook often does by invisibly rolling out tests to small numbers of real people.

Researchers do, however, caution that “bots must be suitably isolated from real users to ensure that the simulation, although executed on real platform code, does not lead to unexpected interactions between bots and real users.”

Facebook calls its system WW, which Protocol plausibly pegs as an abbreviation for “WES World.” But as that sentence makes clear, Facebook isn’t building Westworld here at all. It’s making a simulacron: a world of artificial personality units designed to teach us more about ourselves. While researchers are presumably limiting these interactions for the sake of real users, they’re also helpfully preventing any catastrophic existential crises among bots. Which is only polite, because if you’re building a fake universe full of tiny beings who don’t know their true nature, you’ve basically guaranteed that you’re starring in a remake of World on a Wire and living in a simulation yourself.