Facebook is making AI that can identify offensive memes

Image: Facebook

Facebook’s moderators can’t possibly look through every single image that gets posted on the enormous platform, so Facebook is building AI to help them out. In a blog post today, Facebook describes a system it’s built called Rosetta that uses machine learning to identify text in images and videos and then transcribe it into something that’s machine readable. In particular, Facebook is finding this tool helpful for transcribing the text on memes.

Text transcription tools are nothing new, but Facebook faces different challenges because of the size of its platform and the variety of the images it sees. Rosetta is said to be live now, extracting text from 1 billion images and video frames per day across both Facebook and Instagram.

Right now, it isn’t entirely clear what Facebook is doing with the data. It’s useful for basic features like photo search and screen readers, the blog post points out. But it also sounds like Facebook is starting to put it toward much bigger goals, like figuring out what would be interesting to put in your News Feed, and, more importantly, figuring out which memes are just goofy memes and which are actually spreading hate speech or other offensive comments.

Facebook says text extraction and machine learning are being put to use to “automatically identify content that violates our hate-speech policy” and that it’s doing so in multiple languages. Given the company’s well-known moderation issues, a well-functioning system that can automatically flag potentially problematic images could be a real help.


Allow me to help: If it is a video of a still image it is almost certainly an offensive meme (and also inconsiderate of my data usage).

I think these should universally be banned by Facebook and for good reason: they are intentionally gaming Facebook algorithms for more engagement. Especially the ones that are split, half video and half picture. It is unfair to pages who post appropriately.

AGREED! They need to retroactively demonetize posts that intentionally game the system.

As in most (semi)underground electronic music on YouTube? Nothing wrong with vids like that.

They were referring to memes that are posted as video (to get around Facebook’s algorithm), not music posted in a video that has a still image.

I know, I was debating the "almost certainly" in the statement.

Well they have sound, should not be hard for Facebook to figure out which is which.

So how long do you think before they just add some rightsfree music. You don’t have to be a "hacker" to circumvent that.

They’re either going to leave things as is, which is kinda unfair to other pages that post memes but don’t post them as videos. Or they’re going to keep adding in different things videos need in order to not be classified as just a meme posted as a video. I don’t see the point of just giving up when Facebook has already decided to cut down on the number of memes we see.

I should have qualified it as without sound. Specifically annoyed by things that are a picture rendered as video for no reason.

That said, if Electronic music goes down with it, that’s a sacrifice I’m willing to make

Yeah, that’s pretty much how people are today. They don’t care about anything but themselves.

I sincerely believe FB to be the most evil entity to come out of the digital revolution.

Eh, Facebook has way more information on people who have never used it than you think. I heard about this and searched for my cousin. She’s never had a Facebook. Yet they have a shadow profile of her, and anyone else who doesn’t use it.

I wonder if this so-called AI will identify Facebook itself as an offensive "meme" and delete the entire scurrilous enterprise. One can dream.

So, basically, this just extracts text from an image and does classifying on it? I’m really not seeing what’s that revolutionary.

Although, it does seem like it’s time to start creating memes using Captchas…just to push their system a little…

Memes evolve.

Still waiting for a good explanation for why we need to eliminate things that people find offensive.

Rip free speech
…and hello thought police!

For real though, why can’t we just be big kids and utilize those block/hide/report buttons Facebook gave us? Are we so incompetent that they have to hold our hands and decide what is offensive for us? This is getting ridiculous.

Facebook would save a shit ton of money if they did just this, and I don’t know how many people would disagree with being quick at checking reported posts and just moderating those.

If something really offends people, they will report it and then Facebook can take a look.

View All Comments
Back to top ↑