Google has introduced Tool Retrieval in the Gemini API, a mechanism designed to reduce input token counts and latency for agents managing large tool libraries. Available in preview within the Interactions API, the feature lets developers defer loading parameter schemas until a model actually needs them.
Agents that connect to dozens or hundreds of tools traditionally send every full schema on every request. Tool Retrieval changes that by keeping the full library on standby while sending only names and descriptions to the model. When the model requires a specific tool, it requests it by name, and the API loads the full schema within the same request.
Developers enable the feature by adding a tool_search tool to their request and setting defer_loading: true on the functions they want to hold back. The model still sees the deferred tools in its context, but the parameter definitions stay out of the prompt until invoked. All three steps—search, schema load, and function call—appear in interaction.steps, which allows developers to inspect exactly which tools the model considered when debugging names or descriptions.
The system supports both server-side and client-side execution. Server-side retrieval requires no external index; Gemini runs the search on the server, and developers only need to adjust their tool definitions. Server-side retrieval also plays well with prefix caching. Deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn.
Client-side retrieval allows developers to maintain their own tool registries or apply per-user permission filters by declaring the search tool with "execution": "client". This approach adds one extra API round trip per search but enables dynamic tool exposure mid-conversation when a user connects an integration.
Google bills Tool Retrieval at standard input and output token rates. Loading a deferred schema counts as input tokens for the remainder of the conversation, identical to declaring a tool upfront. The feature saves tokens when the schemas a model never loads outweigh the ones it does, which is typical for long-tail tool libraries. It works with Gemini 3.6 and later, including all Gemini 4 models.
The Gemini Mac app now includes a background automation feature that executes multistep desktop tasks while the user continues working. Google describes the capability as an agent that clicks, types, and navigates menus and forms across local files, desktop applications, and websites.
Illustrative image: axonite / Pixabay
The feature is called Computer Use. It runs in an isolated background window with a live chat preview, keeping the main screen and cursor free for uninterrupted work. Users define which apps Gemini can access, monitor every step in real time, pause execution, and approve key actions before they happen. Google lists batch photo editing, filling shipping forms from local PDFs, and building presentations as example workflows.
Computer Use launches today in English for the US market. The source notes that OpenAI has recently focused on similar desktop automation and is pulling ahead with its Astra model in this specific category.
Google is integrating its telephony capabilities directly into the Tasks module of the Gemini App, moving the underlying Duplex architecture to Gemini 4 Hydrogen and Gemini 4 Neon. The update shifts the calling stack from a hybrid system to a unified stack built entirely on Gemini.
The new architecture addresses the specific constraints of live telephony. Google reports tuning the stack for sub-second latency in noisy acoustic environments, using custom acoustic and conversation cadence models to handle background noise and manage speaking turns. The system also relies on pragmatic grounding to interpret indirect refusals, predict task feasibility based on historical context, and keep calls brief.
Google emphasizes that the calling system must work for the business answering the phone as well as the user. Every call opens with a clear disclosure that an automated system from Google is calling. Merchants can opt out of automated calls through their Google Business Profile settings or by saying "Please stop calling my business" during a call. The system gathers details like happy hour specials or patio seating and feeds them back into Google Maps, reducing repetitive questions for businesses.
The integration extends beyond telephony. Tasks in Gemini now supports broader workflows, including analyzing utility bills, processing transactions via the Agent Payments Protocol, and coordinating with users over email and Rich Communication Services.
Sources
- research.google: How Google Duplex evolved to also call businesses for you in Gemini
- blog.google: Introducing Tool Retrieval in the Gemini API
- the-decoder.com: Google's Gemini can now run Mac apps and finish desktop chores in the background while you keep working
PNEUMETRON EDITORIAL
Pneumetron's pipeline pairs AI-assisted drafting with human editorial review before publishing — our goal is to make staying informed easier, not to replace real reporting.




