Facebook is rebuilding its ads to know a lot less about you

Illustration by Alex Castro

Facebook’s massive business was built on its ability to track users across the internet. But now, thanks to looming regulation and other moves to restrict such data collection, that is changing. Hundreds of engineers are rebuilding how Facebook’s ads work to place more value on user privacy, according to Graham Mudd, a top ad executive at the company.

The moves by Facebook, which are still in their infancy, illustrate how the ad-supported internet economy is in the process of being fundamentally reshaped. Along with Google, Facebook is examining several privacy-enhancing techniques to deliver personalized ads without knowing anything about the specific individuals who view them. That’s an about-face from how ad targeting has worked online to date.

“We definitely see that [ads] personalization will evolve very meaningfully over the course of the next five years,” said Mudd, Facebook’s VP of product marketing for ads, in an exclusive interview with The Verge. “And that investing well ahead of that will benefit all of our customers and enable us to help shape that future state of the ads ecosystem.”

The stakes couldn’t be higher for Facebook to get this right. Apple recently introduced a prompt to iPhones that makes developers ask for permission to track users across other apps for targeting ads. Facebook has said the prompt will likely hurt its revenue growth. Google is planning something similar for Android phones. The European Union is considering a ban on microtargeted ads as part of a sweeping legislative proposal called the Digital Services Act, and the Biden administration recently signaled interest in policing the “surveillance of users” by “dominant Internet platforms.”

Facebook’s new rhetoric about making advertising more privacy-conscious is also, in a sense, admitting defeat. Last year, it mounted a loud PR campaign in objection to Apple’s ad tracking prompt, arguing that Apple was acting anti-competitively and harming small businesses that relied on ads to reach customers. But the campaign ultimately fell flat, and now Facebook is working on some of the same privacy-conscious approaches to data collection that Apple uses. One example is “differential privacy,” a technique that intentionally messes with datasets to obfuscate individual identities.

A spokesperson for Facebook disagreed with the assertion that this privacy shift for ads marks a defeat in its fight with Apple. “We are advocating for a different and better approach to advancing privacy in advertising,” the spokesperson said. “One that is based on industry collaboration and a focus on supporting small businesses and an open internet economy. Apple’s approach is exactly the opposite: exerting its control over the App Store to benefit its own bottom line.”

Facebook’s problems with Apple, which is currently building up its own ads business, are far from over. Given Apple’s tight control of the iPhone, the two will likely butt heads on an area of technology Facebook is exploring called “on-device learning.” Instead of sending data about users to the cloud, an algorithm runs locally on a phone to determine the kinds of ads someone would find compelling and then show them said ads. The results are later sent back to the cloud in an anonymized and aggregate format for advertisers to review.

“I think one of the challenges with on-device learning is that the compute resources required to do it are obviously under the control of the operating systems themselves,” Mudd told The Verge.

Mudd, a nine-year Facebook veteran and key leader internally on this shift in philosophy about how ads work, spoke to me in conjunction with a corporate blog post published Wednesday that outlines the company’s approach to building privacy-minded ads. We covered a lot of ground, but fair warning that parts of the conversation get somewhat technical.

The following interview has been lightly edited for length and clarity.

Alex Heath, senior reporter, The Verge: What is driving all these changes that you all are working on as it relates to ads and privacy? Is it a fear of regulation that’s coming? You all raised a big stink about Apple’s ad tracking prompt for iPhone apps last year, saying it was anti-competitive and going to hurt small businesses. But this seems to me like capitulating to that philosophy to a degree.

Graham Mudd, VP of product marketing, Facebook: It’s a very reasonable question. I would say that in general we both understand and frankly believe in the long-term direction of travel that the industry is headed in, which is towards both more transparency [and] more control over a certain type of data. And that type of data is widely referred to as third-party data. But it’s basically the data that businesses share primarily, in the case of advertising, with ad platforms. It’s used for measurement, for ad delivery, and optimization, and targeting and so forth.

And I think what we see in terms of the trends and frankly our own work, both from a regulatory perspective and the platforms — Google and Apple — is that access to that kind of data will become more limited over the course of the next couple of years. That’s just a reflection, I think, of peoples’ changing expectations around privacy. And I think we’re embracing and trying to build for that future.

A big part of building for that future is developing new privacy-enhancing technologies that allow us to do the kinds of things that we have done in the past, like measurement and ad optimization, but in a way that is far more privacy conscious, that doesn’t allow either party, the advertiser or the ad platform — in this case Facebook — to learn new information about individual users.

Secure multi-party computation (MPC). It’s a mouthful. It’s this technology you all talk about in the new blog post. It essentially encrypts data about users that’s shared between companies for ad targeting. I don’t see how that will be easy to do under Apple’s new ad tracking prompt in iOS, which explicitly asks users if they want to block an app from sharing data with other companies. Google’s Android is planning something similar. Do you think that technique works under the platform rules that we’re already seeing?

I don’t think we’ve had any clear guidance from Apple or others about specific technology and whether that’s acceptable given their policy. I think what is clear is that if you read into the approach Apple is taking, when consent isn’t there, that is when a user hasn’t opted in, they absolutely have developed APIs — as have we — to move data back and forth between companies but in a more anonymous way, primarily using aggregation.

This [MPC] is just another approach to anonymous data sharing. Instead of it necessarily being aggregate, you’re using encryption and cryptography technology to ensure that the same end is met, which is that you don’t learn anything about an individual. So philosophically, I think it’s very well aligned. And in fact, I think a lot of experts would say that it is more privacy safe or conscious than just aggregation. Because with aggregation alone, unless there are other protections like differential privacy in place, you can still back into learning about an individual.

I think the most interesting technology you mention in the blog post is on-device learning, which essentially moves how ads are targeted from the cloud to an algorithm that runs locally on someone’s phone. Apple is already doing this to a degree. I’m curious if that’s a technique you think third-party apps like Facebook can do well without owning the mobile phone system like Apple and Google? Because I could see Apple opting to give itself preferential treatment for how this kind of data is locally treated and the ad creative it can process and all that.

Yes, I think one of the challenges with on-device learning is that the compute resources required to do it are obviously under the control of the operating systems themselves.

And so if standards are built around how those resources are accessed and used in ways that support competition, then that makes a lot of sense. One of the benefits of multi-party computation is that it doesn’t necessarily require this central operation through an operating system or a device manufacturer. That is one of the reasons that we’re very interested in MPC relative to federated learning or on-device. But we’re very much supportive of the development there as well.

I thought it was interesting that, based on your last earnings call, higher prices for Facebook ads are what will drive revenue growth for the next couple of quarters. I think most people probably don’t understand this, but Facebook and Google operate auctions for serving ads, meaning that prices are in a sense set by demand. So I’m curious if higher prices are a function of advertisers seeing Facebook as a place they have to spend, even if your ads are becoming less effective due to the Apple prompt or regulation? Or is it just because the ads are becoming less effective and thereby more expensive?

Good question. I think that it’s definitely more a function of the relative value of the outcomes that our ad platforms drive compared to the outcomes that [advertisers] can drive elsewhere. Like you said, it’s an auction. And so advertisers’ willingness to pay is dictated by basically two things: What is the performance of the platform? And what is that performance relative to what they can get elsewhere? And so those are the core drivers of it.

If our ads become significantly less performant over time, then you can expect that will drive down prices and not drive up prices, at least on a CPM basis, right? Because willingness to pay is dictated by the performance that can be derived through the platform.

So Facebook and Google are still the best places to advertise even if the data isn’t as good anymore.

Yeah, I mean obviously it’s hard for us to comment on what performance looks like on other platforms. One of the things that we are really concerned about is ensuring that the advertising ecosystem does remain competitive. And I would say that some of the changes that have been introduced make it particularly harder for smaller publishers or ad networks to operate competitively. So I think all the more reason, frankly, for these types of privacy enhancing technologies to consider not just what works for the big guys, but also smaller publishers that have much less opportunity to build out first-party datasets of the kind that larger platforms oftentimes have.

We can be honest that small businesses succeeding with advertising is how Facebook also succeeds with advertising. That’s your main cohort. I’ve heard from marketers who buy ads on Facebook. In the last few months, they’ve had some small business clients who probably won’t make it because their ads are less effective and therefore more expensive. They can’t spend to reach new customers. I don’t think most people understand how thin these margins can be for small businesses and how they’ve relied on cheap, effective ads to date.

Couldn’t agree more. In general, smaller businesses are the most vulnerable to these types of changes. And so not only is it important for us to build technologies that allow us to maintain the relevance and performance of ads while making meaningful progress on the privacy front; we also have to do it in a way that they can actually implement from a technical complexity perspective without the kinds of expertise that larger advertisers oftentimes have at their disposal. So yeah, it’s a combination of the technology itself and hopefully over time the ease with which it can be implemented.

Online commerce was a thing well before the pandemic, when you all really started adding shopping features in earnest. I’m curious why shopping wasn’t this much of a focus internally before? Does it have to do with Apple’s changes and other regulations making it harder to track users? If you can’t see when someone makes a purchase outside of Facebook, but you bring that purchasing behavior inside Facebook, you still get to track it and prove that the ad worked, right?

I was involved in some of the work on commerce during the early days of the pandemic, and what I can tell you is that the primary impetus for the acceleration of our work in that space was the recognition that smaller businesses were really disproportionately harmed, and they were having a really difficult time setting up an online presence and a storefront. And so it was born primarily out of a desire to help to the degree that we could.

I think part of what is so challenging about setting up an e-commerce presence if you are a smaller business these days is the way that data flows in our industry is becoming more and more complex and more and more constrained. And so the point you’re making is a very reasonable one. When you set up a storefront on Facebook, it will be easier for us over time to support that storefront with personalized advertising, given that we don’t need to rely on it being shared through an app over a platform that constraints it. It can be equally, if not more, privacy safe in the sense that it never needs to leave our walls. We’ve reduced the amount of data sharing that’s required, which is, at the end of the day, what I think a lot of these changes are meant to do.

Do you think Apple in particular will eventually make it harder for you to collect data about your own users in Facebook’s apps? I know this is a concern that some of your colleagues have. Because right now the focus is on limiting how data is shared between companies. But what if Apple comes for how apps collect data about their own users? I don’t think it’s beyond the realm of possibility.

We would be concerned about that for sure. That said, what’s important to us is that, at every step of the process, our users are fully aware of how data is being collected and used and that it’s intuitive to the degree that we can make it so. I think there’s a lot we as an industry can do to make sure that’s the case. And that limits the potential for those types of actions. In many respects, Apple has defined privacy in such a way that as long as data isn’t shared it can be used to support many different products, including advertising. I think many of their products reflect exactly that philosophy. So it might be a bit challenging to constrain it unless they did so for everyone, including themselves.

I think I know the answer to this one, but speaking of Apple’s prompt, can you give me a sense of how many Facebook and Instagram users have opted into the prompt since you all rolled it out?

Yeah, I think your intuition is correct. Unfortunately, we’re not sharing that.

Could we ever see Facebook show ads that aren’t personalized at all? Or introduce a toggle to let users decide if they want to see personalized ads or not? Is that something you all have thought about or something we could see?

Well, we already have controls that I think are pretty well related to that. We have what’s called an online behavioral advertising control. So you can turn off the use of third-party data for advertising already. You can also — through another control called Off-Facebook Activity— at the advertiser level, decide whether you want data associated with your account and used in advertising.

Now there are, of course, degrees of personalization. And so is there a way, if you’re a user, to reduce all personalization so the ads you see are entirely random? No, that’s not a control we have nor one I can imagine us building, because I think we believe pretty strongly that personalization in marketing is the best possible experience for people and for businesses. But the way that personalization occurs — does it use third-party data versus first-party data, or do I have control over what types of ads are useful and interesting to me — that’s the kind of thing that we are very much invested in and will continue to be.

Can you give me a sense of what this effort is like inside Facebook? Is there a dedicated team that has been set up to work on these new ad products? This reminds me of some big pivots Facebook has done over the years. I imagine that this is a new way of thinking about the business and the technology for you all.

I think you’re absolutely right that this is a very meaningful pivot. We definitely see that personalization will evolve very meaningfully over the course of the next five years, and that investing well ahead of that will benefit all of our customers and enable us to help shape that future state of the ads ecosystem.

So yeah, there are definitely dedicated teams and hundreds of engineers who are working on these types of technologies. Because data and personalization is at the heart of almost every one of our systems, from targeting to ad optimization to measurement, almost all our systems will be rebuilt over the next couple of years. And that’s already very much in progress, to reflect this change in how data is used in support of advertising.

I feel like there’s a disconnect between the media world and the world of regulators about data and advertising and what matters versus what normal people care about. I’m not sure normal people even know the difference between first-party or third-party data, much less think that sharing third-party data is somehow evil or anti-privacy. Obviously, we’ve seen examples of how that has been abused by data brokers, but those are edge cases it seems. I’m curious if you think this is actually a trend that will resonate with normal people who are Facebook users?

We certainly do a lot of research into peoples’ sentiments and beliefs around privacy, and those help to shape the investments that we’ve made. I would say that in most of these cases, regulators, policymakers, and the media tend to be both ahead of and helping to shape consumer opinion here, as is their role. What we’re trying to build for is that longer-term future state. And if you just look at the trends in both regulation, platform changes, and consumer sentiment, this approach to us makes a lot of sense.

So yeah, I think if your read is that the policy elite are ahead of the general population in terms of their understanding on this? Yes, I think that is accurate. But we are seeing consumer sentiments shift as well.

Do you agree that prioritizing first-party data and limiting third-party sharing creates further entrenched, large companies? I think it does. I think it makes you all more powerful. I think it makes Google more powerful. The apps with scale will adapt and get bigger. That could be an unintended consequence of Apple and all this regulation around data sharing.

I think there have been many folks who have pointed out the paradox between privacy and competition. I think that’s what you’re driving at, and I agree that relationship exists and it’s something that we need to be really conscious of. I think there definitely are ways to improve privacy that end up harming competition meaningfully. But I think there are also approaches that don’t have that effect, or at least have a much less meaningful impact on competition.

To be honest, I think one of the most important ways to do that is through industry collaboration. And in some cases, we’ve seen that work really well. And in other cases, when that’s not there, what we end up seeing is that the either intended or unintended consequence becomes much more real. Put simply, I would say collaboration is the best antidote to the potential competitive harms of privacy changes.

Are you all fairly aligned with Google on this? I know Google is more in the federated learning camp. It’s doing FLoC, which is getting a lot of pushback. Is this march toward more privacy-conscious ads something that you and Google will work together on?

Yeah, these are the types of technologies that we are very much seeking feedback on and talking about in forums like the W3C, where Google is a very meaningful participant. We absolutely hope that they will continue to engage on this and provide feedback. And the same is true of our other partners in the industry. But I just want to be super clear: I think our view is that the path forward here is likely an ensemble of privacy enhancing technologies.

It’s not a situation of it’s FLoC or MPC. FLoC addresses a specific use case — behavioral targeting — without revealing anything about a given individual. And the beta that we have running right now is really focused on measurement. In some cases federated learning is going to be the right underlying technology to implement to support a given use case. And in other cases it might be multi-party computation. I don’t think there’s necessarily a silver bullet here. And these technologies don’t necessarily compete with each other because they’re addressing, in some cases at least, different use cases.

Facebook seems to be setting the stage for big changes. You said you are in the process of rebuilding a lot of the core technology. I’m thinking about two things right now: You have custom audiences and you have lookalikes, which are the underpinnings of how you all have grown into the ad behemoth you are. They let advertisers easily use data from other places to match people on your apps and reach those people. Are custom audiences and lookalikes going away, or at least going in a more anonymous direction?

I think you can assume that over time they do become more anonymous, or at least more privacy conscious for sure. I’ll give you an example on lookalikes: Right now lookalikes often use a seed audience — a group of people that an advertiser knows or has a relationship with. If an advertiser and Facebook have the consent of that user to share that user-level information, then our expectation is that will continue and should continue. Consent will play a really important role in products like that.

But in cases where we don’t have consent, that’s where privacy enhancing technologies like multi-party computation can actually play a pretty meaningful role in trying to understand which types of people would find an ad relevant without ever learning about individual people. The whole point of a lookalike audience is actually not to target necessarily distinct individuals that have been identified by one of our partners; it’s to find new customers. And MPC allows us to identify those users without learning about the individuals that might have been the audience in the past.

Comments

(x) Doubt

Why would you doubt a company is trying to do more with less data as their access to data gets restricted?

Because…. its a company?

Companies don’t have to stop doing anything unless they are legally being told to. If facebook really wanted to get good favor they would stop being so intrusive, let the instagram platform remain detatched from facebook and stop using their facebook share buttons as trackers.

Maybe the issue is you see this as a favor to users. It’s a change to stay relevant/leading and deal with coming platform changes, regulation changes, and user preferences.

Everyone wants privacy but when there’s crime involving a relative, everyone wants to break that line of privacy and to know who did it… the paradox of privacy. The idea that you are safer on the web is also what what makes it less safe when it also protects who should not.

Due process and specific consent.

Uh huh, sure.

This could be good. I def will readt he whole article, but yeah I hope it works out.

This was a pretty thorough and well-researched interview – nice

Instead of sending data about users to the cloud, an algorithm runs locally on a phone to determine the kinds of ads someone would find compelling and then show them said ads.

Here we go. I hate the future.

Back to only using Instagram on the browser…

Apple got a lot flak for scanning on the device. This is way worse but everyone’s whisper quiet. They released this news now, for this very reason. No doubt.

100% – that was one of my immediate thoughts as well.

I mean, good on them for being honest about it for once I guess?

There’s also the reality that everyone knows Facebook sucks, so why wouldn’t they just lean into it?

Yea… I have zero trust in FB to do any good inroads towards privacy or towards users for that matter.

It’s not like they have shitloads of info on a lot of people already lol

That was a good interview, I thought it covered all sides and didn’t pull punches (that opening question, wow). Kudos.

…"Facebook is rebuilding its ads to know a lot less about you"…

.
.
.
Soooo…….. Instead of Facebook advertising in my face, say – - a giant Dildo for my Girlfriend that Facebook knows how long I’ve been with,

It’ll just instead show a bunch of Advertisements of all sorts of Sᴇx ᴛᴏʏs ᴀɴᴅ Oᴛʜᴇʀ Pᴀʀᴀᴘʜᴇʀɴᴀʟɪᴀ ??
.
.
Sounds good to me

View All Comments
Back to top ↑