Skip to main content

Facebook’s AI now lets you search for photos by their content

Facebook’s AI now lets you search for photos by their content


No more relying on tags

Share this story

Facebook is updating its search feature today with the help of its computer vision platform. Now, you can search for photos on Facebook using any keyword that describes the contents of the image.

For example, you can search for “pictures of pizza” or “orange shirt photos” and the results will highlight posts from your friends that match the description. Facebook will show these results higher in the grid, followed by other relevant images that you may be interested in. The image-recognition feature is similar to what you may be familiar with on iOS 10 or Google Photos.

Keywords that are somewhat abstract also seem to work. I tried looking for photos that are “creepy” and got the following results.

Creepy-ish, I guess!
Creepy-ish, I guess!

For more subjective searches, Facebook appears to tie results with other context clues from the image, such as caption or comments. A search for “bad photos,” for example, yielded images from friends at the Women’s March a few weekends ago, in which a sign referring to “Bad Hombres” was visible. The results also showed another seemingly fine photo, but the image’s caption described the subjects as “unphotogenic.”

Note that the wording of the search doesn’t seem to totally tie in with Graph Search, which generates results from user-specific posts. So while you can look up “food photos,” you won’t get many relevant results if you tried “photos of me with food.” Unless you’re one to tag yourself in numerous food selfies — I don’t know your life!

Today’s photo search update, available to mobile and web users in the US, align with the company’s efforts to use computer vision in more consumer-facing products. Last year, it began using artificial intelligence to help visually impaired users identify various elements of an image. According to TechCrunch, the AI will eventually be applied to analyze context for video searches as well.