AI-powered Google Lens feature that allows you to record and search any query

During the Google I/O 2024 event, Google unveiled a new Lens feature called Google Lens Ask With Video. It is powered by multi-model Gen AI technology and allows users to ask questions via video. Now, the tech giant is finally rolling out the feature to Android and iOS devices to make Google Lens more usable. According to Google, the “Ask with Video” feature in Lens will be very useful for troubleshooting gadget issues.

Google also provided a demonstration showing how you can use this feature to troubleshoot a broken old record player without having to go through a series of complicated internet searches.

READ ALSO | From 2FA to viewer education, here’s what every YouTuber should do to protect their channels from hackers

How does the new Google Lens feature work?

When users perform a Google search, they will now receive an AI-generated overview of potential answers, along with links to more detailed information. The “Ask with Video” feature in Google Lens is powered by cutting-edge speech models, advanced visual processing, and a custom Gemini AI model hosted in the cloud. This allows Google to analyze videos in real time, frame by frame, providing accurate search results within seconds. This feature improves the user experience by quickly answering complex queries that would typically take longer to complete a manual search.

Once a query is submitted, Gemini’s custom cloud-based model meticulously processes the video frame by frame to generate relevant search results. This AI-powered analysis takes place in seconds, providing a much faster alternative to manual searches that can take several minutes to achieve the same precision. In various test scenarios, Google Lens delivered results with extreme accuracy. This feature is especially useful for DIY enthusiasts as it simplifies troubleshooting or repairing gadgets by providing timely and detailed instructions based on video analysis.

Google Lens is available as a standalone app for Android devices, allowing users to directly experience visual search capabilities. However, iPhone users do not have access to the dedicated Lens app; instead, they can use this feature in the Google app. Meanwhile, Apple is rolling out its own competing technology called Visual Intelligence, part of Apple Intelligence. This feature allows users to ask questions about any topic by simply holding down the camera capture button, similar to Google Lens for visual queries.