Skip to main content

Google says its new image search features are like ‘Ctrl+F for the world around you’

Google says its new image search features are like ‘Ctrl+F for the world around you’

/

You’ll now be able to use Google Lens to take a picture of an object and search for local results

Share this story

“Near me” lets you search for pictures of objects locally.
“Near me” lets you search for pictures of objects locally.
Image: Google

At Google I/O, Google senior vice president Prabhakar Raghavan announced new enhancements for its Lens multisearch tool, which lets you conduct a search with just an image and a couple of words.

A new mode, called “near me,” will let users take a photo of an object and then find results locally. As Raghavan explained, you’ll be able to take a photo of a dish and then search for restaurants that serve that specific food. Google will then display a list of relevant restaurants near you. To make this feature happen, Google scans relevant photos from websites, as well as those posted by reviewers, and then matches them to the one you uploaded. Near me will be available in English later this year and will expand to more languages “over time.”

This is like having a supercharged Ctrl+F for the world around you

Google is also rolling out something called scene exploration. This will allow users to pan their camera and then enter a search phrase about the objects in front of them. When explaining the feature, Raghavan used the example of trying to find a nut-free chocolate bar in a supermarket. You’ll be able to scan an entire shelf of chocolate bars and then see overlays that provide “helpful insights,” like reviews about each object. We think Raghavan’s description of the feature sums this up quite nicely: “This is like having a supercharged Ctrl+F for the world around you.”

Scene exploration lets you scan and learn about objects in your environment.
Scene exploration lets you scan and learn about objects in your environment.
Image: Google

The search giant first rolled out multisearch in April, but its purpose was primarily tied to shopping and searching for instructions. For instance, you can take a screenshot or picture of a dress you like and then type in the name of a color. Google’s multisearch will then show a list of results with similar-looking dresses in that color. Or you can take a photo of a specific type of plant and look up “care instructions” to find relevant information.

But, as Lou Wang, Google’s director of product management, hinted at during its release, Google’s multisearch feature can be used for a lot more, and it looks like that additional functionality is starting to arrive.

To use multisearch, open up the Google app on Android or iOS, and then hit the Lens icon on the right side of the search bar. From there, you can upload a photo or screenshot from your gallery — or snap a picture of an object in front of you — to begin your search. Then, swipe up on the screen and select “Add to your search” to search for the photo alongside a relevant word or phrase.

Update May 11th, 1:51PM ET: Updated to add additional context from Prabhakar Raghavan.

Update May 11th, 2:12PM ET: Updated to add Google’s relevant blog post and additional information related to the new features.


Related:

Today’s Storystream

Feed refreshed Two hours ago Striking out

A
Youtube
Andrew WebsterTwo hours ago
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.


A
The Verge
Andrew Webster4:28 PM UTC
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.


J
Twitter
Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.


T
Twitter
Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.


A
External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.


A
External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.


E
TikTok
Spain’s Transports Urbans de Sabadell has La Bussí.

Once again, the US has fallen behind in transportation — call it the Bussí gap. A hole in our infrastructure, if you will.


J
External Link
Jay PetersSep 23
Doing more with less (extravagant holiday parties).

Sundar Pichai addressed employees’ questions about Google’s spending changes at an all-hands this week, according to CNBC.

“Maybe you were planning on hiring six more people but maybe you are going to have to do with four and how are you going to make that happen?” Pichai sent a memo to workers in July about a hiring slowdown.

In the all-hands, Google’s head of finance also asked staff to try not to go “over the top” for holiday parties.


E
External Link
Insiders made the most money off of Helium’s “People’s Network.”

Remember Helium, which was touted by The New York Times in an article entitled “Maybe There’s a Use for Crypto After All?” Not only was the company misleading people about who used it — Salesforce and Lime weren’t using it, despite what Helium said on its site — Helium disproportionately enriched insiders, Forbes reports.