Google is reportedly working on a powerful new feature for Gemini — its flagship AI assistant — that would let it not just respond to questions but actually control apps on your phone to perform actions for you, bypassing manual interaction. This capability is commonly referred to as “screen automation.”
Instead of merely offering suggestions, gemini could soon handle tasks such as:
📍 Booking a ride in your ride‑hailing app
🍔 Placing a food order in a delivery service
🛍️ Shopping or making selections within supported apps
…all just from your voice or prompt.
📱 How Screen Automation Works
The functionality originates from a Google app beta teardown, which revealed internal references to a new feature tentatively named “Get tasks done with Gemini” (codenamed “bonobo” internally).
Here’s how it’s expected to work:
🖥️ AI Takes Over Screen Actions
Gemini could interact with your phone just like a user would — tapping buttons, filling forms, navigating interfaces, and confirming actions to complete a task, all on behalf of the user. This type of capability goes beyond voice replies and steps into the territory of agents that act for you.
📲 Supported Via Android 16 QPR3
Early reports suggest this capability will first be supported on devices running Android 16 QPR3 and may initially be limited to select apps where automation makes sense — such as ride‑hailing or food delivery.
🧠 User Control & Safety
Because actions involve real commitments (like purchases or ride bookings), Google’s internal strings reportedly warn that:
Users are responsible for the actions gemini performs.
You can stop gemini mid‑task and take manual control if needed.
🌍 Why This Matters
🔄 Shift from Responses to Actions
Until now, gemini has primarily been a conversational AI — great at answering questions, drafting text, summarizing content, and offering suggestions. This move towards agentic behavior marks a significant leap, with gemini potentially handling many day‑to‑day tasks you currently do manually.
🚀 Automation Without Touching the Screen
According to multiple leaks and reports:
You may soon ask Gemini to handle chores that usually require opening specific apps.
This could significantly streamline workflows and save time.
📍 Initial Limitations
However, initial automation likely won’t work universally:
Only a subset of apps may be supported at first.
UI changes by developers could impact automation reliability.
🛡️ Concerns & Considerations
The feature’s advancement raises important questions:
🔒 Privacy & Responsibility
Gemini may have to temporarily capture screenshots or metadata while it operates apps — raising concerns about what data is processed and how it’s stored. Some reports note potential reviewer access for improvement purposes if users enable certain settings.
⚠️ User Control
Because gemini could automate real‑world actions (purchases/services), user oversight will be crucial — both for safety and privacy. You’ll likely have the option to stop a task at any time.
📊 Current Status
The feature is in development and spotted in beta code references — not yet officially rolled out to the public.
Google may offer it initially as an experimental Labs feature or in limited releases, similar to other gemini advancements.
Wider adoption will depend on app support, privacy safeguards, and user feedback.
📍 In Summary — What’s New With Gemini
🔹 Feature
⭐ Status / Details
Screen automation
In development; spotted in beta code
Tasks gemini could handle
Booking rides, ordering food, completing tasks in apps
User responsibility
You remain responsible for automated actions
Platform requirement
Android 16 QPR3 (initial rollout)
Safety controls
Ability to stop or override gemini mid‑task
This potential upgrade signals a major evolution in how AI can interact with smartphones — moving closer to hands‑free task execution, not just information retrieval. Many users and developers will be watching closely as this capability matures.
Disclaimer:
The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any agency, organization, employer, or company. All information provided is for general informational purposes only. While every effort has been made to ensure accuracy, we make no representations or warranties of any kind, express or implied, about the completeness, reliability, or suitability of the information contained herein. Readers are advised to verify facts and seek professional advice where necessary. Any reliance placed on such information is strictly at the reader’s own risk.
click and follow Indiaherald WhatsApp channel