Skip to main content

Chaos theory: what happens when Amazon buys you random stuff?

Chaos theory: what happens when Amazon buys you random stuff?


'Random Shopper' tries to outrun the recommendation engines

Share this story

amazon box stock 1020
amazon box stock 1020

Darius Kazemi is buying random things — or, more accurately, his computer is. He's been feeding his Amazon bot $50 each month, letting it buy whatever it wants, and documenting the haul at The first shipment contained a CD of avant-garde Hungarian music by Ákos Rózmann and Cartesian Linguistics by Noam Chomsky, one of the central theorists of machine cognition. When a book about sentient machines gets ordered by a bot, it’s hard not to think something is up. But Kazemi insists it's totally random. "It's like having a Martian as your personal shopper."

Genuinely random recommendations are remarkably hard to find on the web right now. Any given Amazon search is filtered by your history on the site, your purchases, the purchases of demographically similar users, a general popularity index, and countless other data sources. Recommendation engines like this are working their magic on most of the web's big databases, from Google and Amazon to Pandora, Netflix, YouTube, eBay, and so on.

These programs just weren't built for surprises

This system is great at sending junk to its rightful place at bottom of your search results, but the top of the list can feel airless and boring. If, based on your first six months of purchasing, Amazon's algorithm decides you aren't the kind of person who likes free jazz, you might never know what you're missing. Apply the same rules over dozens of sites, and it's easy to see how the experience could get claustrophobic. The buzzword for this is "filter bubble," what's left when everything unexpected has been whittled away by algorithms. To put it more simply, these programs just weren't built for surprises.

Random Shopper is an attempt to puncture the mathematical pronouncements of the personalized web. On the surface it doesn't bear much resemblance to the vast churning matrixes that power Google and Amazon, but it's doing roughly the same work. You give it a price range, and it tells you what to buy. The results are, for most purposes, kind of terrible. But they come with an element of surprise that data-driven algorithms have mostly given up on.

Random isn’t easy. Wriggling out of Amazon’s personalized algorithms meant searching random keywords from a brand new account and scrambling the results out of order to befuddle Amazon's indexing algorithms. Kazemi isn't the first to try this, either. As best we can tell, that goes to a coder named Paul Hunkin, who was playing off an XKCD comic. His project scraped TradeMe — the New Zealand equivalent of eBay. Totally random, it turns out, is also not very fun. Hunkin’s bot suggested what he diplomatically called "a lot of crap." A typical purchase was "watch batteries for a watch I don't own."

Rozmann2The avant garde CD randomly selected for Darius Kazemi.

"In shuffle, we are shuffling things we already know."

It's bad batteries like these that make most engineers wary of turning things over to chaos, even as they search for a way to add zest to machine-learned algorithms. But at least one theorist has ideas on how to combine the two. Tuck Wah Leong is an interaction specialist at the University of Technology in Sydney who's built a reputation as one of the biggest advocates for unpredictability in software. Leong wants us to think of randomness as a design resource, to be deployed whenever users need to feel refreshed or surprised. His prime example is the shuffle button on music players, particularly iTunes and the iPod shuffle — but he's careful to distinguish it from pure chaos. "In shuffle, we are shuffling things we already know," he told The Verge. "We have past experiences with those songs. Although it's random, there's a likelihood that it's more meaningful to us." The lesson, for Leong, is that randomness works best in combination with a strong filter.

It's something industry insiders are increasingly aware of. When we spoke to Thomas Pinckney, who works on consumer data analysis for eBay, he put it this way: "One of the things we spend a lot of time thinking about is, how do you do the contextual stuff well, but also really inspire people with things that they didn't even know they would like?" That means taking a step away from the algorithms that have ruled the space for the last ten years, and into something new, whether it's community management, human editors, or plain old experimentation. "In our experience, there's no one way that works everywhere," Pinckney told us. In other words, it’s all on the table.

Is surprise worth $50 a month?

In the case of Random Shopper, there are now a number of filters in place to ensure purchases aren't completely out of the blue. Kazemi says upfront that he limited the searches to CDs, DVDs, and paperback books and lets users set the price range. He runs the bot from its own account, so there's no shopping history to draw from, and randomizes each list, supposedly undoing all that data-driven indexing, but it's very possible that a little bit of Amazon's preference for popular goods still slipped through. As Pinckney put it, "it's hard to go off the rails entirely." And Random Shopper might be even better if it only crawled halfway out of the bubble of popular recommendations — not all the way into the world of spammy e-books, but not in the bubble of bestsellers and iPhone accessories either.

Whether that sounds like a good idea depends on what you want from online shopping. Do you want surprising but useful recommendations, or do you want to get a look at the wooly bazaar of uncurated commerce? Do you want good things or interesting things? Is surprise worth $50 a month?

In Kazemi's case, the answer is clear enough. "I like the idea of being forced to consume things outside of what these large corporations like Amazon think I want to consume," Kazemi told us. "Amazon is basically a bot that buys you things anyway."