Google is reportedly building a significant upgrade for its generative AI assistant Gemini that could shift it from being primarily a conversational helper to something closer to a real-life work agent. In a recent beta teardown of the Google app code by 9to5Google, developers uncovered strings pointing to a feature known internally as “screen automation”. It suggests that Gemini could soon take direct actions on your behalf inside certain Android apps, such as placing orders or booking rides, without requiring the user to manually tap through screens.
While Gemini already powers conversational tasks like drafting emails or generating research plans, this upgrade appears poised to let it literally interact with app interfaces, tapping buttons and navigating screens to finish tasks you’d typically do yourself. Early evidence from the beta suggests these capabilities will initially be limited to a handful of supported apps and will emphasize user supervision, with Google warning that “Gemini can make mistakes” and that users remain responsible for actions taken on their behalf.
How this upgrade moves AI from assistant to agent
The concept behind screen automation is a major step toward giving AI more autonomy in everyday digital workflows. Instead of just suggesting what you could do, Gemini may soon execute those choices directly inside apps for you. Early code strings from Google’s beta also indicate privacy precautions, such as advising users not to enter login or payment information into AI chats and warning that screenshots may be reviewed to improve the feature. Google already offers some agent capabilities through its Gemini Agent platform in Workspace and web, where AI can handle complex workflows and coordinate across services, but screen automation could bring those abilities directly into smartphones and daily app use.
If these features roll out widely, it could mark a shift in how people interact with mobile devices, from tapping and swiping themselves to giving AI tools permission to act on their behalf. That may make everyday routines easier, but it also raises questions around control, security, and oversight, especially when automation touches sensitive tasks like bookings or financial orders.
Google is reportedly positioning these upgrades as optional and supervised, letting users stop or override Gemini at any time. For now, though, the screen automation feature remains in development and has yet to arrive in stable releases.
