Mac users couldn’t launch apps this afternoon after Apple verification server issue

Illustration by Alex Castro / The Verge

Many Mac users reported that their computers have been running slower than usual on Thursday — including a number of Verge staffers — with apps launching slowly or not at all, alongside other Apple service issues.

It appears that the problem is due to many people rushing to download macOS Big Sur, which was officially released today, which in turn seems to have crashed Apple’s OCSP (online certificate status protocol) service — which is used for several key aspects of macOS, including validating digital certificates for both Apple and third-party software on the Mac, as Ars Technica reports.

Apple’s status site notes that the company had resolved an issue earlier today that may have prevented users from downloading macOS software updates, although it hasn’t been confirmed that the Big Sur update was the cause of the outage. The company also reported issues with iMessage and full-blown outages with Maps routing and navigation as well as its traffic tracking, which may have been related to the OSCP failure, too.

Mac and iOS developer Panic reports corroborates the reports, noting that the downed service had disabled Apple’s Gatekeeper technology, which checks validity of apps when you try to launch them. Panic also reports that the issue appears to be resolved, but it’s not clear if things have totally cleared up for everyone yet.

Apple didn’t immediately reply to a request for comment.

Comments

Is what way is it ok for an online service to stop your 3k+ laptop from working?

It didnt stop working?….A server crashed resulting in slower load times. Resolved within hours. Not a huge deal.

Not a huge deal that a single service going offline stopped my apps from launching on my local computer and caused the system to become unresponsive?

Ok, sure. People complain non stop about Windows spying on them but Mac OS phones home way more.

People complain non stop about Windows spying

I think this is the important takeaway people are missing. This proves Apple is phoning back the hash of every single app you open. They’re tracking all the apps you use and when you use them. How the hell people praise them as "champions of privacy" is beyond me.

That’s not what it proves. It’s checking the validity of certificates to see if they’ve been revoked or not.
The server was timing out.

Apple checks the validity of a certificate as often as it can so that it can quickly quarantine the spread of malware, which has worked a few times in the past. The fix here is to either decrease how often a certificate is validated OR handle the exception better when a request times out.

This is no doubt a huge oversight on Apple’s part, but the solution is quite simple, even if it does involve a trade off. If OCSP times out a dozen times in a row the service can just suspend for an hour and that effectively solves the problem.

As a part of the validity check, the hash of the application binary is sent. And it’s trivial for Apple to have a database of known hashes, since they also require notarization (i.e. you must submit ALL of your binaries to get signed).

This proves Apple is phoning back the hash of every single app you open.

Not it doesn’t. Gatekeeper doesn’t check the hash of every app you open every time you open them. It checks files that are quarantined (for instance stuff you downloaded) before you run them. If the check pass, the code isn’t checked again unless it changes. People that had issues during this outage were probably downloading updates for their apps after the Big Sur update, which triggered new GateKeeper checks (that failed due to the outage).
Besides, I don’t think there is any reason to believe, or any proof, that any personal information is shared with Apple during a Gatekeeper check.

Admittedly, none of this is great and it is indeed a big deal. It’s never great for a company like Apple to suffer from such an outage. It’s also fair to think Apple’s policies over what you can or cannot run on macOS are overprotective and too restrictive. I’m not disputing that. But to claim there is an underlying privacy scandal behind it is just false.

For people who says Apple doesn’t phone back the hash of every single app you open, do you know what hashes are?

If the check pass, the code isn’t checked again unless it changes.

Only until reboot, I think.

Not it doesn’t. Gatekeeper doesn’t check the hash of every app you open every time you open them. It checks files that are quarantined (for instance stuff you downloaded) before you run them.

Actually, it turns out that macOS now caches the result for about 12 hours (or until reboot). Before that, it was using 5 minute cache time.

So it IS fair to say that macOS phones home "every time you open them".

I doesn’t stricly contradict their own documentation on the subject, but it makes it kind of misleading ("Before opening downloaded software for the first time, macOS requests your approval […]"). Also, might be a dumb question, but if the cached check result automatically disappear after 5 minutes, I don’t know how the system is supposed to launch apps when you’re offline. Do you have technical documentation on the topic ?

The OCSP check is a soft failure, it’s completely ignored while offline.

Here are details: https://lapcatsoftware.com/articles/ocsp.html

All of this is, of course, undocumented by Apple.

It was a pretty huge deal. You basically couldn’t do anything. It was a beach ball party. You could barely even open a window.

It wasn’t slower load times for me, Excel and Teams refused to open at all. That is not okay.

It is a huge deal because it made my computer unusable. I thought it was the SSD failing and was performing all kinds of tests, diagnostics and troubleshooting.

This is unacceptable.

Lol, tell your "not a big deal" to the hundreds of editors I was supporting this afternoon who had deadlines. Nothing worse than seeing a tidal wave of tickets and Slack messages and then throwing up a "shrug" emoji.

Seemed a huge deal when I couldn’t open Word for hours and have briefs due.

And while that server crashed, my Mac laptop was practically unusable for at least a good hour. Not just a ‘little’ slower, but ‘just walk away for an hour’ slower. So yes, it effectively stopped working.

Let me state that again, a downed Apple server rendered my laptop unusable in the middle of the work day. If you don’t understand why that’s a problem, I don’t really know what else to say.

It’s not okay. This was a HUGE !@#$ up. I was quite worried, and I was rattling my brain trying to figure out what was going on. The console was going crazy but I couldn’t make sense of it. I did notice that turning off WiFi seemed to temporarily fix the problem, but I never imagined what was causing it.

This is definitely a rare edge case, but also a failure for the Apple security model. In some ways I’m glad that this is all the problem was. I was worried for a bit. Definitely don’t want to see this happen again though.

This major security fumble, along with the no password root acces, are reasons why Apple is in need of a serious revision of their goodwill.

Their all out marketing security claims just sounds hollow.

This design implementation sounds like it kinda sucks..

Absolutely, though it’s a broad, longstanding problem with the online approach to certificate revocation in general, not just Apple’s software.

To be honest, I’m a little surprised they’re still doing OCSP to determine certificate validity. The standard went in and out of vogue about a decade ago, as security professionals quickly realized DDoS’ing the server would nullify its protections, in addition to delaying users. Last I heard, browsers had gone back the old certificate revocation lists, a low-tech and problematic solution, but one much less vulnerable.

Updates still aren’t working, at least for me.

Also nobody can download Big Sur, almost 6 hours after launch

It just works!

View All Comments
Back to top ↑