Skip to main content

    Google personalizes search results even when you’re logged out, new study claims

    Google personalizes search results even when you’re logged out, new study claims


    A study, albeit from competitor DuckDuckGo, finds that Google search results can vary significantly

    Share this story

    Illustration: Alex Castro / The Verge

    The amount of personalization inherent in any one of Google’s many massive software products runs deep, based on everything from your search history to your location to every single search link you might have clicked. And avoiding that personalization seems to have become more difficult over the years. According to a new study conducted by Google competitor DuckDuckGo, it does not seem possible to avoid personalization when using Google search, even by logging out of your Google account and using the private browsing “incognito” mode.

    DuckDuckGo conducted the study in June of this year, at the height of the US midterm election season. It did so with the ostensible goal of confirming whether Google’s search results exacerbate ideological bubbles by feeding you only information you’ve signaled you want to consume via past behavior and the data collected about you.

    The study found that Google search results varied significantly, even in private mode

    It’s not clear whether that question can be reliably answered with these findings, and it’s also obvious DuckDuckGo is a biased source with something to gain by pointing out how flawed Google’s approach may be. But the study’s findings are nonetheless interesting because they highlight just how much variance there are in Google search results, even when controlling for factors like location.

    DuckDuckGo found that a majority of participants in its study saw different results when searching three divisive terms: “gun control,” “immigration,” and “vaccinations.” According to the company, “these discrepancies could not be explained by changes in location, time, by being logged in to Google, or by Google testing algorithm changes to a small subset of users.” DuckDuckGo says it controlled for location by treating local results that varied across regions as though they were identical. “Interestingly, this adjustment didn’t affect overall variation significantly,” the study reads.

    Image: DuckDuckGo

    For the study, DuckDuckGo compiled 87 result sets (76 on desktop and 11 on mobile), and it conducted the searches consecutively and simultaneously starting at 9PM ET on June 24th, 2018. It did one private, logged-out test and then a logged-in test immediately after, so as not to influence the private test with prior results. What DuckDuckGo found was that using private browsing and logging out of Google had almost no effect on the variation in search results: users saw a roughly equitable amount of variation across all three searches and when searching privately and while logged in.

    Some key elements to the variation included changes in news sources and the placement of sometimes identical links in different positions, which has a drastic impact on the likelihood that they get clicked. The study also found variations in how news articles and videos were laid out among standard text links, and as many as 22 different domains showing up in the first page of results for “vaccinations,” with a standard search result page typically containing 10 organic links.

    Image: DuckDuckGo

    Perhaps most importantly, there doesn’t seem to be any way to get a single, objective search result from Google that can be easily replicable across users or locations. To further drive home the point that private and logged-out searches are still equally variable, DuckDuckGo calculated the difference between text results for private, logged-out users and compared them to both the results of other anonymous users and the standard, logged-in search results for those initial users.

    “We saw that when randomly comparing people’s private modes to each other, there was more than double the variation than when comparing someone’s private mode to their normal mode,” the study reads. The data indicates that between standard, logged-in search results and private, logged-out ones, there was typically a difference of only one or two domains, while the variation between anonymous users was from three (for gun control) to five (for vaccinations) domain changes. That indicates that there is some heavy personalization going on for people using private browsing when logged out, and it’s not clear why that is or what effect it’s having on user behavior.

    Image: DuckDuckGo

    Ultimately, the study seems to prove that there’s no easy way to use Google search without the software seemingly trying to determine who you are and whether it can better serve your needs. That’s not necessarily a bad thing; technology companies all over the world built businesses around being smarter, faster, better automated, and more personal. (And by giving away really useful, ad-supported products for free.)

    A majority of internet users wouldn’t use Google search if it wasn’t constantly a best-in-class search engine, with a number of tangential benefits, like the Chrome browser tie-ins, that make it a lot easier and more convenient than, say, DuckDuckGo. But it’s still unsettling to know that there’s so much unseen, algorithmic tinkering going on when you perform a basic action, like searching one or two-word terms on Google. And that there’s really nothing we can do to get a neutral result.

    Google says incognito mode doesn’t base results on signed-in search histories

    Google could not comment on a study it did not have access to, but the company says that search results can change by the minute and sometimes even by the second, especially for news topics. Google also says that personalization is done on only a small fraction of the total number of queries entered into search, and relying on recent queries is often to determine the context for a search, like when a word may apply to a sports team and a city simultaneously. The company did confirm that it does not personalize results for incognito searches using signed-in search history, and it also confirmed that it does not personalize results for the Top Stories row or the News tab in search.

    Following the study’s publication this morning, Google told The Verge in a statement that it found the methodology flawed and the findings misleading. “This study’s methodology and conclusions are flawed since they are based on the assumption that any difference in search results are based on personalization. That is simply not true,” a Google spokesperson said. “In fact, there are a number of factors that can lead to slight differences, including time and location, which this study doesn’t appear to have controlled for effectively.”

    In a separate series of statements, Danny Sullivan, who ran the influential Search Engine Land blog and now works as Google’s public liaison for search, echoed similar sentiments on Twitter about the nature of personalization in Google Search:

    It’s important to note that any kind of personalization, even based on just location, isn’t a new feature or something Google has been doing behind the scenes without anyone’s knowledge. The company began personalizing search results for every user, even those without a Google account, way back in 2009 using an anonymous cookie that would take into account information like your location, language, search history, and other factors.

    More recently, the company says it’s moved away from personalizing search results, even as it increasingly uses personalization for products like Google Assistant and Gmail because it didn’t seem to really improve the search experience. “A query a user comes with usually has so much context that the opportunity for personalization is just very limited,” Pandu Nayak, the head of ranking at Google search, told CNBC in September. Google says it still personalizes for location and “immediate context from a prior search.”

    Regardless, Google has come under fire in the last couple of years for engaging in practices that, while not exactly clandestine, were certainly not widely known and likely kept out of the limelight to avoid too much public scrutiny.

    One notable example, surfaced by an investigation from The Wall Street Journal, was how app developers were often able to read your personal Gmail messages, which prompted concern from some users and a congressional inquiry. Google also felt the need to stop scanning Gmail messages to target advertisements last year because it was worried that the practice, when it came to light in contract negotiations, might scare away enterprise companies.

    Google has come under fire for not being transparent about how its products work

    Google also failed to disclose a flaw in a Google+ API that could have exposed hundreds of thousands of users’ personal information for roughly six months, an action that might result in a Federal Trade Commission investigation. And in September, Google was forced to reverse a controversial Chrome login feature that would help it better target ads by automatically logging users into the browser without their express permission.

    The common thread here is that Google’s products, while often free to use and usually quite well-designed, only make money by using a vast trove of personal information to sell targeted online advertisements. And to improve how much money it makes from each ad sale, and the total quantity of ads it sells, Google often resorts to tactics its users have little insight into or deep understanding about.

    Personalized search results aren’t necessarily nefarious on their own. But the ways Google’s algorithms function — especially those that are now aided by complex and often unexplainable artificial intelligence software — are and always will be outside the understanding of the common user. By definition, that means most people have no idea how they work, including lawmakers and members of privacy watchdogs and advocacy groups. That could be a problem for a company like Google as it tries to maneuver a consumer market and a legislative environment that’s not quite as forgiving or negligent as it used to be.

    Update 12/5, 2:05PM ET: Added statement from Google calling the study’s methodology flawed. The headline has been updated to reflect that Google’s personalization of search results is a claim being made by its competitor DuckDuckGo.

    Update 12/5, 3:34PM ET: Included public statements on the nature of personalization in Google search from Danny Sullivan, the company’s liaison for search, published on Twitter.

    Today’s Storystream

    Feed refreshed 12 minutes ago Midjourneys

    Richard Lawler12 minutes ago
    Everything looks better in slow motion.

    Apple’s Dynamic Island alert system isn’t sitting still around your iPhone 14’s front-facing camera array. We’ve been enjoying its contextual animations — and even an Android copycat — since it was unveiled, but take a look at it here, captured at 240fps, to see exactly how iOS applies animations that make it feel a bit more lively.

    External Link
    Russell BrandomAn hour ago
    Oracle will pay $23 million to settle foreign bribery charges.

    The SEC alleges that Oracle used a slush fund to bribe officials in India, Turkey and the United Arab Emirates.

    This behavior is sadly common among software companies doing business overseas, and it’s not unique to Oracle. In March, a former Microsoft executive claimed the company spent as much as $200 million a year in bribes for foreign officials.

    External Link
    Emma Roth3:16 PM UTC
    Celsius’ CEO is out.

    Alex Mashinsky, the head of the bankrupt crypto lending firm Celsius, announced his resignation today, but not after patting himself on the back for working “tirelessly to help the company.”

    In Mashinsky’s eyes, I guess that means designing “Unbankrupt yourself” t-shirts on Cafepress and then selling them to a user base that just had their funds vaporized.

    At least customers of the embattled Voyager Digital crypto firm are in slightly better shape, as the Sam Bankman-Fried-owned FTX just bought out the company’s assets.

    Mary Beth Griggs2:46 PM UTC
    NASA’s SLS rocket is secure as Hurricane Ian barrels towards Florida.

    The rocket — and the Orion spacecraft on top — are now back inside the massive Vehicle Assembly Building. Facing menacing forecasts, NASA decided to roll it away from the launchpad yesterday.

    External Link
    Andrew J. Hawkins1:30 PM UTC
    Harley-Davidson’s electric motorcycle brand is about to go public via SPAC

    LiveWire has completed its merger with a blank-check company and will make its debut on the New York Stock Exchange today. Harley-Davison CEO Jochen Zeitz called it “a proud and exciting milestone for LiveWire towards its ambition to become the most desirable electric motorcycle brand in the world.” Hopefully it also manages to avoid the cash crunch of other EV SPACs, like Canoo, Arrival, Faraday Future, and Lordstown.

    The Verge
    Andrew Webster1:06 PM UTC
    “There’s an endless array of drama going on surrounding Twitch right now.”

    That’s Ryan Morrison, CEO of Evolved Talent Agency, which represents some of the biggest streamers around. And he’s right — as you can read in this investigation from my colleague Ash Parrish, who looked into just what’s going on with Amazon’s livestreaming service.

    The Verge
    Richard Lawler12:59 PM UTC
    Green light.

    NASA’s spacecraft crashed, and everyone is very happy about it.

    Otherwise, Mitchell Clark is kicking off the day with a deeper look at Dish Network’s definitely-real 5G wireless service , and Walmart’s metaverse vision in Roblox is not looking good at all.

    External Link
    Jess Weatherbed11:49 AM UTC
    Won’t anyone think of the billionaires?

    Forbes reports that rising inflation and falling stock prices have collectively cost members of the Forbes 400 US rich list $500 billion in 2022 with tech tycoons suffering the biggest losses.

    Jeff Bezos (worth $151 billion) lost $50 billion, Google’s Larry Page and Sergey Brin (worth a collective $182b) lost almost $60b, Mark Zuckerberg (worth $57.7b) lost $76.8b, and Twitter co-founder Jack Dorsey (worth $4.5b) lost $10.4b. Former Microsoft CEO Steve Ballmer (worth $83b) lost $13.5b while his ex-boss Bill Gates (worth $106b) lost $28b, albeit $20b of that via charity donations.