Snapchat applies for patent to serve ads by recognizing objects in your snaps

Snapchat has filed a patent application for a system of advertising that uses object recognition to serve users sponsored filters. The technology outlined by the company would identify items in users' pictures, and then offer them image overlays from brands related to these objects. It's the visual equivalent of buying advertising space based on keywords in Google searches — but instead of looking for textual data in searches like "headphones" or "shoes," it's looking for the objects themselves.

The patent application was filed in January last year, published by the US patent office earlier this month, and first spotted by Business Insider. Although it includes details about the advertising system described above, its primary purpose is outlining a more general system of "object recognition based photo filters." These would complement Snapchat's current geofilters, which offer users custom image overlays based on location. Users can submit geofilter designs themselves for free (for a friend's wedding, for example), while advertisers have to pay if they want to create a filter to promote a brand or product.

An object recognition based filter for the Empire State Building from Snapchat's patent application.

In the patent application, Snapchat first describes how object recognition could provide a "compelling way for users to personalize, supplement and enhance" their snaps. The company gives the example of a user taking a picture of the Empire State Building. The app's object recognition would identify the landmark, and when the user swiped through their filters they might be offered an image overlay of King Kong, hanging off the top of the skyscraper swatting at planes.

For advertisers, the object recognition could be used to serve sponsored filters, but also more involved ads. In one example, an image that includes an object "recognized as a restaurant" would serve a filter of that restaurant's menu. In another, a filter associated with a certain dish from that restaurant may include "celebratory graphics" to "commemorate the user's achievements" if they order a particular item often enough. Snapchat even speculates that it could create an automated auction system for the filters, letting companies compete to buy the advertising rights for certain objects.

The application describes how companies might bid against one another to buy filters for popular items.

This patent application is, of course, just a patent application, and doesn't necessarily reflect concrete plans from Snapchat. However, using object recognition for fun and for advertising is certainly something that fits with the company's business model to date. You can compare this idea with Snapchat's use of facial recognition and image overlays (which it calls Lenses). In both cases, machine vision is used to create a feature that is first and foremost fun for users, with the tech then monetized by selling sponsored overlays to third parties.

A presentation deck for investors leaked from Snapchat earlier this year suggested that the company wants to dramatically boost its revenue, and aims to make between $500 million and $1 billion in the year 2017. (Compared to revenues in 2015 of $59 million.) Sponsored filters based on object recognition might just help the company reach that goal. After all, what's better for monetization than potentially turning every object in a photograph into an opportunity for ads.


Snapchat even speculates that it could create an automated auction system for the filters, letting companies compete to buy the advertising rights for certain objects.

The stuff nightmares are made of.

do Google has patent to serve ads by recognizing objects in photos?

Who knows, but I can’t imagine they haven’t already been doing this far before January 2015, since they’ve recognized objects far longer.

Still don’t get the logic behind this. If I take a picture of coffee, then in all likelihood I already have coffee, and don’t need more.

Google always does this: I recently searched for, and bought, a new grill. So now all websites I visit are full of ads for grills. Pointless. I bought it already, you are too late!

Now, if they showed me ads for a juicy steak for my new grill. Or perhaps a way to turn cold rainy days into hot summer days. Then we could talk.

Online ads run on BS promises. Views, engagement etc.

I already have that right now. If, let’s say, I do a search for running shoes and I buy them online, I’ll probably get bombarded with ads for… the same bloody running shoe for a month. Totally pointless

The online ad world is a mystery to me.

Cream, pull the refresh….dolla dolla bill yallllll

View All Comments
Back to top ↑