Skip to main content
Back

Google's Gemini Now Controls Your Mac in the Background While You Work

Google's Gemini app for Mac can now click, type, and navigate apps in a background window while you keep using your screen.

  • Gemini app for Mac adds computer use, driving desktop apps and websites on your behalf.
  • Runs in an isolated background window with live preview, keeping your cursor and screen free.
  • Rolling out in US English to AI Plus, Pro, and Ultra subscribers only.
  • Requires Apple Silicon Mac on macOS Sequoia 15.0 or later, age 18+.
  • Triggered with @Computer use prompt; new Blender and Autodesk Fusion connectors available.
  • Google warns the agent can make mistakes including deleting or sharing files incorrectly.

Google is rolling out Computer use in the native Gemini Mac app. The feature can operate local files, installed software, and websites to complete multi-step tasks described in plain language. Gemini performs the work in a dedicated background window and streams a live preview into the chat, leaving the main display and pointer available.

One prompt, several interfaces

Users invoke the agent by adding @Computer use to a prompt. Gemini can then click, type, navigate menus, fill out forms, and move information among approved apps, folders, and websites. New desktop connectors for Blender and Autodesk Fusion appear under Customize > Connectors, alongside the existing Git and Obsidian integrations.

Google’s demonstration starts with a child’s butterfly sketch and has Gemini build and rig a 3D model in Blender, including the internal structure needed for animation. The example shows how the agent can carry a task through several interface states without continuous mouse input from the user.

Eligibility is tightly scoped

Requirement Details
Hardware Apple Silicon Mac
Operating system macOS Sequoia 15.0 or later
Region and language United States, English only
Account Personal Google account belonging to a user aged 18 or older
Subscription Google AI Plus, Pro, or Ultra
Installation Gemini desktop page

Gemini works beside you

Gemini executes actions inside its own background window instead of moving the pointer across the primary workspace. The chat displays a live feed of that window, allowing users to monitor progress, interrupt the task, or continue working in another app.

The separation makes long-running jobs less disruptive during coding, calls, or document editing. It does not remove the risks attached to approved files and services, because the agent can still take consequential actions within the access it receives.

Access follows folders and apps

Gemini’s local-file access is limited to folders added through the Connected folders toolbar. Users also choose which applications the agent may operate and can stop an active task with one click. The announced safety boundaries rely on these access controls, visible progress, and confirmation steps.

  • Folder scope: Gemini can inspect only the local folders selected by the user.
  • App scope: Users approve the desktop applications available to the agent.
  • Sensitive steps: Gemini pauses for login, payment, and other consequential actions that require confirmation.
  • Interruption: An active task can be stopped from the Gemini interface.

Temporary copies created while Gemini edits computer files are deleted after 24 hours or when another query or task begins, whichever occurs first.

GUI gaps are the target

The feature targets workflows that span software without stable APIs or compatible automation systems. Gemini operates the rendered interface through mouse and keyboard actions, which gives it access to menus, forms, and controls designed for people.

  • Creative batch work: Color-grade a folder of images or stage multiple 3D scenes in Blender.
  • PDF-to-form entry: Extract dimensions, weights, and addresses from local documents and enter them into shipping quotes.
  • Dashboard reporting: collect regional metrics from online dashboards and assemble them into Google Slides.
  • Marketplace listings: Analyze clothing photos, research prices, and prepare resale listings.

Direct APIs remain faster, more deterministic, and easier to test when software exposes them. Computer use extends automation to closed desktop applications and cross-app handoffs where an API integration would be unavailable or expensive to build.

Errors remain consequential

Google warns that Gemini can make mistakes, including deleting or sharing the wrong file. Long workflows are especially vulnerable to changed layouts, unexpected dialogs, ambiguous labels, slow page loads, and stale application state.

The dedicated window leaves the main desktop usable, but approved files and accounts remain exposed to erroneous actions. Teams evaluating the feature can reduce that exposure by granting narrow folder access, using test accounts, keeping recoverable copies of important files, and requiring confirmation before irreversible steps.

Native packaging changes the product

OpenAI’s Operator and Anthropic’s computer-use tools established model-driven interface control through hosted environments and developer APIs. Google’s implementation brings that interaction loop into a native consumer Mac app, with app selection, folder permissions, live monitoring, interruption, and cleanup built into the experience.

Apple is also testing tighter controls for Mac applications that request broad disk access, partly in response to the risks posed by AI agents. A preview appeared in the macOS 26.3 beta. Those controls could affect how desktop agents request permissions and how much of the file system they can reach.

Google’s computer-use capability had already been available through its API, so the underlying model is not the main change. The Mac release packages the model around a parallel execution surface and explicit approval gates. For developers, that architecture provides a practical template for desktop agents: keep progress visible, scope access narrowly, make interruption immediate, and treat every external side effect as a permission decision.