Apple delays controversial child protection features after privacy outcry

Apple delays controversial child protection features
Illustration by Alex Castro / The Verge

Apple is delaying its child protection features announced last month, including a controversial feature that would scan users’ photos for child sexual abuse material (CSAM), following intense criticism that the changes could diminish user privacy. The changes had been scheduled to roll out later this year.

“Last month we announced plans for features intended to help protect children from predators who use communication tools to recruit and exploit them, and limit the spread of Child Sexual Abuse Material,” Apple said in a statement to The Verge. “Based on feedback from customers, advocacy groups, researchers and others, we have decided to take additional time over the coming months to collect input and make improvements before releasing these critically important child safety features.”

Apple’s original press release about the changes, which were intended to reduce the proliferation of child sexual abuse material (CSAM), has a similar statement at the top of the page. That release detailed three major changes in the works. One change to Search and Siri would point to resources to prevent CSAM if a user searched for information related it.

The other two changes came under more significant scrutiny. One would alert parents when their kids were receiving or sending sexually explicit photos and would blur those images for kids. The other would have scanned images stored in a user’s iCloud Photos for CSAM and report them to Apple moderators, who could then refer the reports to the National Center for Missing and Exploited Children, or NCMEC.

Apple detailed the iCloud Photo scanning system at length to make the case that it didn’t weaken user privacy. In short, it scanned photos stored in iCloud Photos on your iOS device and would assess those photos alongside a database of known CSAM image hashes from NCMEC and other child safety organizations.

Still, many privacy and security experts heavily criticized the company for the new system, arguing that it could have created an on-device surveillance system and that it violated the trust users had put in Apple for protecting on-device privacy.

The Electronic Frontier Foundation said in an August 5th statement that the new system, however well-intended, would “break key promises of the messenger’s encryption itself and open the door to broader abuses.”

“Apple is compromising the phone that you and I own and operate,” said Ben Thompson at Stratechery in his own criticism, “without any of us having a say in the matter.”

Comments

Let’s hope it’ll get delayed just as long as AirPower.

Delayed until they "make improvements". No, there are no improvements. They either uphold their commitment to privacy and toss this entire project in the dumpster, or they will subjected to extreme criticism yet again when they try to roll it out with the new "improvements".

Snowden wrote a damn good article on this entire issue – https://edwardsnowden.substack.com/p/all-seeing-i

Apple’s insistence on rolling this out is completely shattering their entire "privacy" facade. They should be going in the opposite direction, like fully encrypting all iCloud documents and files.

Thanks for sharing that Snowden article. He really nails the problem.

The real problem if Apple does implement this on-device scanning, then there is nothing stopping Apple from adding other types of image scans at a later date, or other objects, like drugs, guns, other types of weapons, bombs, even certain faces. Plus Apple could target scanning for certain countries, and the list keeps going.

Also Apple would NEVER be able to use the word ‘Privacy’ any more, because it would only be a word, and that word would mean nothing when it comes to Apple.

This is not a real problem and does not make sense based on the way CSAM works.

CSAM is, in and of itself, illegal. You cannot legally own CSAM. It is also, because of that, commonly traded and distributed amongst a community of criminals. There is a known database of commonly traded illegal content.

The reason this works is because you can compare a known database against material criminals are holding. And that comparison, using hashes, is like a fingerprint. Hashes only match if they are exact and there is almost NO chance of even two almost identical images returning a match. Even a different photo from the same burst shot a fraction of a second later would not return a match if the prior photo WAS in the database. (And both were CSAM.)

So, now, take your list: drugs, guns, bombs, or faces.

What good would it do for Apple or ANY government (no matter how authoritarian) to demand Apple produce a list of hashes for those items? What database would they be comparing against? What would be gained by getting a match between identical images of two faces, or two guns? The pictures themselves aren’t illegal and if it is unrelated ACTIVITY they want to pursue — they already have the original photo or else they wouldn’t be pointlessly trying to make a hash comparison. Anything the government wants to do based on those types of topics would be far easier to do some other way, and attempting to use hash matching would be an incredible waste of time.

And, indeed, even IF some government decided to do that waste if time, Apple would have to agree to do it and IF a government tried to do it sneakily, Apple has a human review factor that would immediately flag the fact that the hash match was not of CSAM and then make no referal.

Your mind and view of this is completely limited and lacks true understanding of what they could do with this stuff.

This is the weakest possible retort you could have given.

It’s pretty clear 2 things: or you simply blindly trust Apple and whatever they talk, or you simply really doesn’t understand nothing about privacy and gouvernement surveillance and really think this is a cool feature…

Nice try but you merely tie for an equally weak retort.

Dude… just give up your weak replies already…

Once someone shows what it is about my comment that is incorrect — via an actual rebuttal rather than simply saying "you don’t understand any of this" — i’ll stop telling people they have weak replies.

I doubt anyone is able to do that, though, or else they’d have countered my opinion with one of their own.

If they understood this half as well as they claim to, it should be an easy task. And yet…

Your arguments aren’t good. They’re just a load of convoluted nonsense that is simply a pain in the arse to argue against and even when people do, you simply move the goal posts in the next reply.

People not wanting to engage with you isn’t because you’re the last bastion of truth on the internet, it’s because there is no point engaging you.

All you post is warped logic that supports Apple no matter the content, no matter the facts, no matter the policy.
Arguing with you is simply a waste of time.

You waited days to post that?

"Your arguments aren’t good."

One, false.

Two, at least they’re arguments instead of sneaking in days later to waste time not actually adding one.

Grow up.

Correct me if I’m wrong – I’ve gotten the impression on earlier occasions that you are some sort of moderator here. If so, and if so then doubly so, how would you review your own comments here with regards to proper and decent forum behaviour?

I am not a moderator here and given that moderators are labeled moderators, i’m not sure why you’d think otherwise.

With that said, I argue my opinions on the subjects or articles that interest me. That’s generally video games, mobile technology, and entertainment news. I have never ever started a response to someone with whom I disagree with anything other than a counter argument. If someone engages in ad hominem nonsense or attempts to discredit my opinions by claiming I’m a shill, I’ll call them out in kind or report them if they take things to a too personal level.

If I were a moderator, there would be a lot less lazy name-calling and off-topic whining about perceived motivation, that’s for sure.

Sadly, the best I can do is report it when I see it and hope it gets removed.

Just speaking as someone who’s been a moderator before in other online fora and in gaming communities (don’t even get me started on that!), so far I’ve not been seeing anything in brianericford’s posts that would suggest that he’s just here to be a shill or a troll or whatever. We may not always see eye-to-eye on things, but I think he’s been pretty chill and reasonable as far as commenters go (there’ve been some real, shall we say, pieces of work up in here over the years – shout out to my Verge old-timers – y’all know what I’m talking about!).

Don’t flatter yourself. I read the article days later. My life doesn’t revolve around your obnoxious comments.

Even now your argument is bad, both itemised points are the same point, you’ve just stretched it out for 10x the necessary word count.

Pathetic

Uh huh. I’m sure that’s exactly how it happened.

Why are you incapable of engaging in a discussion without making things personal? Do you behave this way to people who have different opinions in real life or does the keyboard make you feel powerful?

Uh huh. I’m sure that’s exactly how it happened.

I see yet again you’re just ignoring any argument against your empty words and have moved the goal posts again.

I’m simply sick of seeing your bullshit and based on the recs, a lot of other people are too.

You’ve yet to make an actual argument against my empty words. Three people have shown up to claim it’s a bad argument and thus beneath them. That’s not a rebuttal, it’s a copour.

You did your rage anger tough guy thing where you think cursing at people and belittling them will make them give up. I do hope it’s only an online thing.

I’ve told you, I have no interest in engaging your bad arguments.

Your arguments aren’t good. They’re just a load of convoluted nonsense that is simply a pain in the arse to argue against and even when people do, you simply move the goal posts in the next reply.

People not wanting to engage with you isn’t because you’re the last bastion of truth on the internet, it’s because there is no point engaging you.

All you post is warped logic that supports Apple no matter the content, no matter the facts, no matter the policy.
Arguing with you is simply a waste of time.

It’s not just me who feels this way about you babe.
If you’re wondering why people don’t call you back, this is why.

Alright. I accept that you forfeit. You could have done it quietly and without the anger. Thanks, though.

I’m sure you do

What would be gained by getting a match between identical images of two faces, or two guns? The pictures themselves aren’t illegal and if it is unrelated ACTIVITY they want to pursue — they already have the original photo or else they wouldn’t be pointlessly trying to make a hash comparison.

Remember when China banned pictures comparing Xi to Winnie the Pooh? I can certainly see the Communist Party putting this system to good use against such "offending" imagery.

even IF some government decided to do that waste if time, Apple would have to agree to do it

True, but we’ve seen Apple and others bending to government pressure time and again. It will be a choice between injecting a hash to find Xi photoshopped with Winnie the Pooh or abandoning the Chinese market. If I had to bet on which it would be, well, you know which one.

So why does Apple have to scan the device itself? I don’t think anyone cares that Apple scans photos or documents coming into iCloud itself, that much is a given, however, this is on-device scanning. The only reason to do that is to view private information not contained within iCloud.

You seem to be under the impression that once this is put in place then that’s where it stops; that’s where you’re wrong. What makes you think that on-device scanning has to be against a hash? Once we as consumers allow for this, it opens the floodgates for allowing any type of comparison matching.

View All Comments
Back to top ↑