AI is rapidly moving from “just talking and giving advice” to “actually doing the work for you.” On November 20, Google officially launched the “Computer Use” feature for the macOS version of the Gemini app, allowing AI to work across desktop apps, local files, and web environments to autonomously carry out complex, multi-step tasks.

Runs independently in the background without grabbing the mouse, with live preview and the ability to stop anytime

For users who are used to handling multiple tasks at the same time on a computer, the biggest concern about letting AI take over the computer is often that it will “grab the mouse and click around the screen,” interrupting what they are doing.

Gemini’s “Computer Use” feature is designed to avoid this issue: all tasks run in a separate background window, so the user’s everyday main screen and mouse cursor are not disturbed. At the same time, a live preview of the current operation is shown in the chat interface, and users can manually pause or stop the task at any time.

image.png

For specific use cases, Google listed several practical examples: batch processing and editing photos, extracting weight, dimensions, and address information from local PDF documents and automatically filling in shipping quotes, compiling data metrics and screenshots from regional dashboards into a PowerPoint presentation, and analyzing clothing photos in a local folder to help list items on a secondhand marketplace.

Fine-grained, controllable permissions, with macOS 27.3 enhancing security and transparency

To give users a greater sense of control when granting AI permission to operate the computer, the feature provides multiple layers of security and authorization mechanisms:

Self-selected authorization and key confirmations: Users can choose which apps Gemini is allowed to access, and the model will proactively ask for user approval before carrying out critical actions.

Scope limits and transparent indicators: Users can restrict access permissions to specific apps and folders and configure authorization on a per-request basis. The system also provides transparent indicators that clearly show whether one app is currently controlling another.

System version alignment: The feature launches just before the official rollout of macOS 27.3. Once macOS 27.3 is officially released, the new system will alert users to the risks of granting access to restricted data, further clarifying the boundaries of what AI agents can view and do; apps that are already installed and have been granted Mac permissions will not need to be reauthorized after upgrading to macOS 27.3, nor will they be forced to show a new explanation screen.

Availability and subscription requirements

In terms of how to get it and the support requirements, users can download the Gemini app for Mac from Google’s official website. Currently, the “Computer Use” feature is available to Google AI Plus, Pro, and Ultra subscribers in the United States. Subscription plans start at $4.99 per month (approximately 35.7 yuan), and the hardware requirement is a Mac with an Apple silicon chip.