clock menu more-arrow no yes mobile

Filed under:

The shady data-gathering tactics used by Cambridge Analytica were an open secret to online marketers. I know, because I was one

New, 36 comments

Market researchers have used these tricks for years

Illustration by William Joel / The Verge

The recently revealed Facebook data “breach” that allowed Cambridge Analytica to get access to millions of users’ worth of Facebook data has been greeted as a shocking scandal. Reporters and readers have been surprised to learn about the ability to gather personal data on the friends of people who install a Facebook app, the conversion of a personality quiz into a source of political data, the idea that you can target marketing messages based on individual psychographic profiles, and the surreptitious collection of data under the guise of academic research, later used for political purposes. But there is one group of people who are mostly unsurprised by these revelations: the market researchers and digital marketers who have known about (and in many cases, used) these tactics for years. I’m one of them.

Back when the Cambridge Analytica data was getting collected by an enterprising academic, I was the vice president of social media for Vision Critical, a customer intelligence software company that powers customer feedback for more than a third of the Fortune 100 companies. Our enterprise clients wanted to know how social media data could complement the insights they were getting from their customer surveys, and it was my job to come up with a way of integrating social media data with survey data.

Vision Critical’s blue chip customer roster made us an appealing target for the many social media vendors hawking data-gathering solutions. In 2012, the data analytics firm Microstrategy pitched me on their “Wisdom” tool for Facebook, which the company had touted as a data source based on ”12 million anonymous, opted-in Facebook users.” But when I spoke with an analyst at Microstrategy that December, he told me that the company’s data set — by then, nearly 17.5 million strong— was based on just 52,600 actual installs, each of which provided access to an average of 332 friends.

Nor was Microstrategy doing something unusual. The tactic of collecting friend data, which has been featured prominently in the Cambridge Analytica coverage, was a well-known way of turning a handful of app users into a goldmine.

“We were all conscious that friend data was accessible,” says Sam Weston, a communications consultant who has been working in digital marketing and market research for nearly two decades. “I don’t think that anybody had perspective on the potential consequences until it was slotted into this news story, where the consequence may have been the election of Donald Trump.”

Mary Hodder, a longtime privacy consultant who is now the product developer for the Identity Ecosystem Steering Group, was equally unsurprised. “I knew 10 years ago that Facebook’s API allowed an entity to gather friend data,” Hodder told me. “But I wasn’t surprised that the 95 percent of the population that didn’t understand this were shocked. They thought if Facebook was going to sell you out, it would just be you. They didn’t know you would take all your friends with you.”

If Facebook’s generous access to friend data was known to many marketers and software developers, so was the tactic of disguising data grabs as fun apps, pages, or quizzes. Another company I spoke with back in 2012 was LoudDoor, a Facebook advertising company that offered enhanced ad targeting based on the data they were gathering from millions of Facebook users. The company ran a network of Facebook pages that were essentially content farms, like the Diving Wrecks and Reefs page that consisted of pretty underwater photos. Interspersed with all the photos were occasional come-ons for fan surveys that would enter you in a contest; taking the survey meant installing a “fan satisfaction” app that gave LoudDoor access to all your data. Pages and apps like this might have seemed innocent to the average Facebook user, but people in the marketing community were hardly deceived.

“It was pretty common knowledge among people who understood the internet that if you were taking a quiz to find out what kind of cheese you are, somebody on the other end is very interested in getting that data,” says Susanne Yada, a Facebook ad strategist. “I wish I could say I was more surprised and more alarmed. I just assumed that if you take a quiz, someone would know who you are because you are signed into Facebook.”

Among the ubiquitous data-gathering apps of that period, the personality quiz that Cambridge Analytica created was nothing special. “It is actually stunning to think, with the clarity that perspective brings, that you could stand up the kind of ridiculous quiz or survey that they did and then walk away with psychographic profiles on 50 million Americans,” Weston muses now. “Even for someone who worked in the field, [the Cambridge Analytica story] was a moment that gave you real pause to reflect on the business that we walked away from, but that was a massive part of the industry for a long time.”

And yes, these “fun” apps gathered your friends’ data too: the LoudDoor salesperson I spoke with at the time told me that they had close to 12 million users, which gave them access to data on 85 million Americans. But that friend data grab was far from clear to fans of the page, even if the company’s disclosure notice explained that the purpose of its app was to ensure that “brands can make better decisions about which content they should promote to you.”

As for the idea that the purpose of gathering data is to target ads — well, Cambridge Analytica is scarcely an outlier there, either. Mary Hodder recalls working with a company called Apisphere that used location data for ad targeting back in 2008 and 2009.

“We did this project for the Hard Rock Cafe Casino in Las Vegas,” Hodder told me. “They wanted to put wands in the ceiling to collect the IMEI [identification] numbers of every phone that went by, map everywhere they went in the casino or on the property, and map them in the hallways up to their rooms. And then they could do a reverse lookup on IMEI numbers because there are companies that aggregate IMEI numbers, and as soon as they figured out who the person was, they could send them offers, text them offers, and the people had not opted in. So they were basically just intercepting your phone, and figuring out how to send messages to you in one form or another.”

Hodder remembers objecting to this as at a meeting where the rest of her colleagues saw nothing amiss with the practice: that’s how normal it was to harvest data and use it to target individual ads, long before Cambridge Analytica got in on the action. For those of us who were witness to the “look what we can do!” explosion of data-driven marketing tactics, it takes some reflection to understand why the practices of Cambridge Analytica have surprised so many people.

“When you say ‘We’re creating psychological profiles to sway people,’ marketers have been doing that since marketing existed,” Yada observes. “But I think there’s a difference between actually representing what your services are and how they can help people, versus being really clandestine and trying to sway people with fake news.”

“The fundamental problem is the gap in understanding about what Facebook’s business model actually is,” Weston says. ”You know how Target’s business model works, or how Apple’s business model works, but nobody understands how these folks [digital marketers] actually make money. That’s not just true for Facebook but for every ad-supported business and every data-supported business, which is just about every tech company… [Facebook] did a good job talking to Wall Street about how their business works, but at no point did they actually talk to their users.“

Given the widespread normalization of deceptive data gathering and marketing tactics, I count myself lucky that the company I worked with didn’t buy into the frenzy of the social media data gold rush. Because Vision Critical had its roots in the market research industry, where there are norms and codes of practice around how you handle respondent data, the idea of grabbing up friend data was utterly anathema: the company’s founder dismissed it as a non-starter the very first time it came up, and at every stage in developing our own Facebook app, we disclosed that we were using it to gather data.

But the whole time, it felt like we were swimming against the tide by following old-school standards for transparency and accountability in how we handled data. I hate to admit how many times I pitched my colleagues on some clever way of incentivizing people to connect to Facebook, based on some scheme or app I’d just stumbled across, only to be reminded that it would violate our data or privacy policies. If I’d been working in a digital marketing agency where gamifying data requests was the norm, I can easily see how I might have yielded to the temptation of disguising a data grab with a recreational app, or scooping up friend data just because it was there.

That experience points to how difficult it will be to reform not just Facebook, but the larger industry of data collectors and marketing shops that have evolved to maximize the amount of data collected and the precision of ad targeting. Social networks and other advertising platforms may set up various processes that notionally screen out data aggregators or manipulative advertisers, but as long as these companies run on advertising revenue, they have little incentive to promote transparency among data brokers and advertisers. And those industries, in turn, have little motivation to place ethics ahead of profit.

The outrage now directed at Cambridge Analytica and Facebook suggests there might be an appetite for an online ecosystem based on a different compact between consumers, platforms and advertisers. But we won’t build that ecosystem by pretending that this is a matter of a few bad actors. It’s time for us to face up to what online marketers and researchers have known for more than a decade: the contemporary Internet runs on the exploitation of user data, and that fact won’t change until consumers, regulators and businesses commit to a radically different model.

Alexandra Samuel is the former VP of Social Media for Vision Critical. She is now an independent technology writer and a regular contributor to The Wall Street Journal, JSTOR Daily, and The Harvard Business Review.