Skip to main content

How to start experimenting with Google Lens

How to start experimenting with Google Lens

/

This image recognition app can be a great way to research a photo or copy some text

Share this story

Illustration by Samar Haddad

This morning, I was idly looking through my Twitter feed and came across a thread discussing Starbucks’ recent hiring of a former Pinkerton intelligence analyst, which led to a discussion of the history of the Pinkertons as strikebreakers, which led to a 19th-century illustration of a crowd of women confronting uniformed men with guns. Curious as to the source of the illustration, I aimed my Pixel 6 phone at it and tapped on the Google Lens icon on the right side of my homepage’s Google search field.

After maybe a minute, I found a series of websites that had used the illustration, including one from Wikipedia that told me that the image was of an 1884 wood engraving made from a sketch by Joseph Becker illustrating “the striking miners’ reception of ‘Blackleg’ workmen when returning from their work escorted by a detachment of Pinkerton’s detectives.”

Lens and a woodcut.
To identify this woodcut, I pointed my phone camera at it and used Lens.
Old-time illustration with searched sources
The results were a number of sites that had used the illustration, which enabled me to find the source.

It’s easy to forget how useful Google Lens can be. This Android app has been slowly improving since it was introduced in 2017, and it probably doesn’t get as much attention as it deserves. Lens, an image recognition app, can not only help identify the source of an image but also can help you figure out which bird is in your friend’s photo or whether anyone is still selling that sweater you need to replace.

Here are a few of the things you can do with Google Lens. These were tested with a Pixel 6 running Android 12; since Android phones can differ (especially if you’ve got a Samsung device), your mileage may vary.

How to access Google Lens

Before we talk about what Google Lens can do, it’s probably a good idea to let you know how you can access it. There are several places you can find it in Android:

  • In the Google search field on your homescreen, Lens is the icon on the far right. (It looks like a circle surrounded by three multicolored lines and a dot.)
  • In the Google Photo app, swipe left on the buttons on the bottom of the screen until Modes is highlighted, and then select Lens.
  • In the Chrome app, tap the camera icon on the right side of the search field.
  • And of course, you can always just open the Lens app itself.
Home screen
To bring up Lens, you can just tap on the symbol to the right of the Google search field.
Photo app with Modes selected.
You can also select Modes > Lens in the Photo app.

How to use an image from your device

When you open the Lens app, images from your device will be listed below the square “Search with your camera” field at the top.

While you can’t search among those images in Lens (which is inconvenient, to say the least), you can narrow down your search. The section just beneath the search field will say something like “Screenshots” or “Downloads.” Tap on the arrow next to it and you can select from a variety of sources and apps that your image may be associated with.

An easier way to use Lens with an existing image is to go to your Photos app and use its search feature to find the specific image you want. Select the image, then tap on the Lens icon at the bottom of the screen.

How to identify an item or text using your camera

  • If you want to identify an object, some text, or anything else that you haven’t already photographed, then you can by choosing one of the methods above to start Lens on your phone. You’ll see that square field on top labeled “Search with your camera.” Tap on the camera icon in the center of that field, and it will open to take up your entire screen.
  • You’ll see four corner lines that will indicate what Lens is going to concentrate on. Move the camera so that the image you want is inside of those lines; you can also use your fingers to make the image larger or smaller. If you don’t feel you have enough light, tap the lightning bolt icon at the top left.
  • Tap “Search” at the bottom of the screen.

How to identify a specific part of an image

Once you’ve told Lens to start identifying an image, it will begin by trying to guess what in the photo you want to match and will show you what it’s concentrating on by surrounding the item with four “corner” lines. But while Lens can be very good at picking out what’s interesting in a photo, it can guess wrong. For example, when I tried it on a photo of a dog in the foreground with a person walking in the background, Lens centered in on the person.

If you feel that the wrong object is being identified, tap on the item in the photo that you do want it to concentrate on. (Sometimes a secondary object will already have a dot on it, as if to say, “Maybe it’s this instead?”)

Lens with photo showing dog
Lens thinks I want to ID the person in the background. Or maybe the dog instead?
Lens showing dog
By tapping on the dog, Lens knows what you want to ID.

If Lens has chosen the correct item but the outline is taking in too much or too little of the surrounding area, then just use your finger to adjust it.

What else can you do with Lens?

Once you’ve got the basics down, there are a variety of features that you can experiment with. If you go to the Lens app, you can see these features listed at the bottom of the screen below your image. Here are a few:

  • By tapping on Translate, you can translate text to one of several dozen languages.
  • By tapping on Text at the bottom of the Lens screen, you can copy the text to your clipboard or to your device, listen to it being read, or do a search.
  • Homework will bring up information about the highlighted text. For example, when I used Edna St. Vincent Millay’s poem “Dirge Without Music,” I got results from the Poetry Foundation and Poets.org among others.
Poem by Edna St. Vincent Millay
By pointing your camera at text, you can copy it, save it, or have it read aloud.
Poem by Edna St. Vincent Millay
You can also have the text immediately (although probably imperfectly) translated.
  • Shopping will help you find a product similar to the one you’ve taken a photo of (using a barcode will get you much more accurate results).
  • Places lets you point at a building or other object outdoors that will get you information on where you are or what you’re looking at.
  • Dining lets you take a photo of food or a menu and identify recipes or where the restaurant is located.

There are a variety of other things you can do with Lens — and the best way to find out how it works for you is to try it out.

Today’s Storystream

Feed refreshed 5:33 PM UTC Striking out

A
Youtube
Andrew Webster5:33 PM UTC
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.


A
The Verge
Andrew Webster4:28 PM UTC
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.


A
Andrew Webster1:05 PM UTC
Looking for something to do this weekend?

Why not hang out on the couch playing video games and watching TV. It’s a good time for it, with intriguing recent releases like Return to Monkey Island, Session: Skate Sim, and the Star Wars spinoff Andor. Or you could check out some of the new anime on Netflix, including Thermae Romae Novae (pictured below), which is my personal favorite time-traveling story about bathing.


A screenshot from the Netflix anime Thermae Romae Novae.
Thermae Romae Novae.
Image: Netflix
J
Twitter
Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.


T
Twitter
Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.


A
External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.


A
External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.


E
TikTok
Spain’s Transports Urbans de Sabadell has La Bussí.

Once again, the US has fallen behind in transportation — call it the Bussí gap. A hole in our infrastructure, if you will.


J
External Link
Jay PetersSep 23
Doing more with less (extravagant holiday parties).

Sundar Pichai addressed employees’ questions about Google’s spending changes at an all-hands this week, according to CNBC.

“Maybe you were planning on hiring six more people but maybe you are going to have to do with four and how are you going to make that happen?” Pichai sent a memo to workers in July about a hiring slowdown.

In the all-hands, Google’s head of finance also asked staff to try not to go “over the top” for holiday parties.