Google left thousands of machines in businesses with broken Chrome browsers this week, following a silent experimental change. Business users accessing Chrome through virtual machine environments like Citrix kept seeing white screens on open Chrome tabs, blocking access to the browser and leaving it totally unresponsive. It left many IT admins confused over the problem, as businesses typically manage and control Chrome updates.
After complaints, Google was forced to reveal it had launched an “experiment” on stable versions of Chrome that had changed the browser’s behavior. The experiment was made silently, without IT admins or users being warned about Google’s changes. Google had simply flipped the switch on a flag to enable a new WebContents Occlusion feature that’s designed to suspend Chrome tabs when you move other apps on top of them and reduce resource usage when the browser isn’t in use.
“The experiment / flag has been on in beta for ~5 months,” explained David Bienvenu, a software engineer at Google, in a Chromium bug thread. “It was turned on for stable (e.g., m77, m78) via an experiment that was pushed to released Chrome Tuesday morning. Prior to that, it had been on for about one percent of M77 and M78 users for a month with no reports of issues, unfortunately.”
Google rolled back the change late on Thursday night, following multiple reports from businesses with thousands of users affected. “I’ll rollback the launch of this experiment and try to figure out how to deal with Citrix,” noted Bienvenu in the bug thread.
“This has had a huge impact for all our Call Center agents and not being able to chat with our members,” explained a Costco IT admin in the Chromium thread. “We spent the last day and a half trying to figure this out.”
One IT admin that alerted The Verge to the issue said “we felt that this is a shady thing that Google can update Chrome silently without announcing anything and can impact 100,000+ people on a whim.” Those concerns are mirrored by hundreds of replies on Google’s support forum, the bug tracker thread, and on Twitter and Reddit.
It has left IT admins angry that they’ve wasted valuable resources and time on trying to fix issues in their environment, and questions over why Google decided to make a silent change to Chrome in the first place. “I am stunned by your response,” said one IT admin in response to Bienvenu’s confirmation on the issues. “Do you see the impact you created for thousands of us without any warning or explanation? We are not your test subjects. We are running professional services for multi million dollar programs.”
Google confirmed the testing and rollout of this feature in a statement to The Verge, and apologized for the issues:
“After the rollout, we received reports that in some virtual environments, Chrome on Windows displays a blank page, which may be because Chrome mistakenly believes it’s covered by another window. As soon as we confirmed the reports, the feature was disabled.
If Chrome on Windows is displaying blank pages, restart Chrome. On the next start, this feature will be disabled.
We also want to provide an explanation of how this change was rolled out. For some features, Chrome uses a gradual rollout process that happens more slowly than the main rollout. This allows us to quickly revert a change if we discover a bug that wasn’t uncovered in prior testing.
Once we received reports of the problem, we were able to revert it immediately. We sincerely apologize for the disruption this caused.”
Update, November 15th 4PM ET: Article updated with a statement from Google.