Google on Wednesday is rolling out a new update that lets users search what’s on their screen via a simple gesture when browsing within the Google Chrome or Google Search app on iOS. The feature is similar to Android’s built-in “Circle to Search” feature which also lets users search what’s on their screen using a variety of gestures.
At launch, iPhone users will also be able to search what’s on their screen by drawing, highlighting, or tapping something, via Google Lens.
For instance, if you’re reading an article and come across an image of an interesting art piece, you can now use Lens to quickly circle or tap on the image to learn more about it. Or, if you’re watching a video and see an item you like, you can use Lens to find something similar by circling it.
The idea behind the feature is to allow users to quickly perform a visual search while browsing without having to take a screenshot or open a new tab, but it also gives users another way to kick off a traditional web search — an area of its business that could be impacted by the adoption of AI technology over the long-term.
After you highlight or tap something on your mobile device’s screen, you’ll be shown visual matches and other related results. You can then tap the “Add to your search” option to refine by color, brand, or another detail. You can also ask a follow-up question to learn even more about a topic.
To access the new feature in the Chrome or Google app, you need to open the three-dot menu and select “Search Screen with Google Lens.” In the coming months, you will be able to access the feature through a new Lens icon in the address bar, Google says.
The update will continue to roll out this week and will be available globally on iOS.
In addition, Google announced that it’s expanding AI Overviews, which display a snapshot of information at the top of the results page, to more of its Google Lens search results.
In the past, Google displayed AI Overviews in Lens searches that included both images and text. With this latest update, users will begin to see AI Overviews without adding additional text or questions to their searches.
For instance, if you come across an interesting-looking car, you can take a picture of it and then get an AI Overview to help you quickly learn more about what you’re looking at, and get links to helpful resources on the web.
This update is rolling out this week for English-language users in countries where AI Overviews are available, starting with the Google app for Android and iOS.
The update will soon roll out to Chrome on desktop and mobile devices, the company says.
