Google Gemini introduces “Ask about this screen” feature on Android, note taking feature on Google Meet

While the bot’s initial response will be based on what the screenshot contains, the user can then ask Gemini to access the internet for more details

FILE PHOTO: Gemini has rolled out two new features for Android that were showcased earlier at Google I/O this year.  | Photo Credit: Reuters

Gemini has rolled out two new features for Android that were showcased earlier at Google I/O this year. Called ‘Ask about this screen,” the feature allows users to have queries about whatever is happening on the screen. 

There’s also an “Ask about this video” feature which is compatible with YouTube. 

The prompt appears on top of the Gemini window after the user has summoned the AI assistant. Once the user taps on the icon a temporary screenshot is captured and then the user can type in their query. 

While the bot’s initial response will be based on what the screenshot contains, the user can then ask Gemini to access the internet for more details or even summarise a news article.

The system does not save these screenshots. 

For the “Ask about this video” feature, the bot appears to look through the video captions to describe the contents of the video.

The features were spotted by Android Police on Gemini beta app version 1.0.662093464 and the Google beta app version 15.34.32.

Gemini in Workspace

Aside from Android devices, Gemini will also be integrated into Google Meet for their new “Take Notes for Me,” feature for summarising meetings. The feature is currently available just in English and will be rolled out to Google Workspace clients with Gemini Enterprise, Gemini Education Premium, AI Meetings and Messaging add-ons. 

The tool automatically makes notes in a Google Doc and will attach the file in the calendar event after the meeting is over. It can also share the file with anyone who turns the feature on and the meeting organiser.

If someone is late to a meeting, it will fill you in once you’ve joined via links of files along with the Google Doc.

Google said it will complete rolling the feature out by September 10. 

Gemini in WhatsApp

Separately, Android Authority reported seeing Gemini extensions for WhatsApp, Google Messages and Android system notifications, in the 15.34.32.29.arm64 beta. 

The bot could help users send and read messages and make calls on WhatsApp using voice commands according to the description of the feature.