The Pneumetron
NEWSTOOLSINFRASTRUCTURE
Pneumetron.Google rolls out tool retrieval, desktop automation, and telephony updates
Share
Skip to article content
  1. Home
  2. ›
  3. News
  4. ›
  5. Technology
  6. ›
  7. Google rolls out tool retrieval, desktop automation, and telephony updates
Technology·November 22, 2026

Google rolls out tool retrieval, desktop automation, and telephony updates

BY PNEUMETRON|3 MIN READ · 721 WORDS3 MIN READ
Tools
Share
Illustrative photo: server, cloud, development, business
Illustrative image: ColossusCloud / Pixabay

Google is deploying three distinct agent infrastructure updates: an API optimization for large tool libraries, background desktop automation on Mac, and a telephony stack built on Gemini 4.

Key Takeaways

  • 01Tool Retrieval uses deferred loading to keep parameter schemas out of context until a model actually needs them, cutting input token bloat.
  • 02Client-side execution adds an extra API round trip, and loaded schemas are billed at standard input rates for the rest of the conversation.
  • 03The Gemini Mac app runs multistep desktop tasks in an isolated background window, though it is currently limited to US English.
  • 04The telephony stack now runs on Gemini 4 Hydrogen and Neon, prioritizing sub-second latency and pragmatic grounding for live calls.

Google has introduced Tool Retrieval in the Gemini API, a mechanism designed to reduce input token counts and latency for agents managing large tool libraries. Available in preview within the Interactions API, the feature lets developers defer loading parameter schemas until a model actually needs them.

Agents that connect to dozens or hundreds of tools traditionally send every full schema on every request. Tool Retrieval changes that by keeping the full library on standby while sending only names and descriptions to the model. When the model requires a specific tool, it requests it by name, and the API loads the full schema within the same request.

Developers enable the feature by adding a tool_search tool to their request and setting defer_loading: true on the functions they want to hold back. The model still sees the deferred tools in its context, but the parameter definitions stay out of the prompt until invoked. All three steps—search, schema load, and function call—appear in interaction.steps, which allows developers to inspect exactly which tools the model considered when debugging names or descriptions.

The system supports both server-side and client-side execution. Server-side retrieval requires no external index; Gemini runs the search on the server, and developers only need to adjust their tool definitions. Server-side retrieval also plays well with prefix caching. Deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn.

Client-side retrieval allows developers to maintain their own tool registries or apply per-user permission filters by declaring the search tool with "execution": "client". This approach adds one extra API round trip per search but enables dynamic tool exposure mid-conversation when a user connects an integration.

Google bills Tool Retrieval at standard input and output token rates. Loading a deferred schema counts as input tokens for the remainder of the conversation, identical to declaring a tool upfront. The feature saves tokens when the schemas a model never loads outweigh the ones it does, which is typical for long-tail tool libraries. It works with Gemini 3.6 and later, including all Gemini 4 models.

The Gemini Mac app now includes a background automation feature that executes multistep desktop tasks while the user continues working. Google describes the capability as an agent that clicks, types, and navigates menus and forms across local files, desktop applications, and websites.

Illustrative photo: cyber, security, network, internet Illustrative image: axonite / Pixabay

The feature is called Computer Use. It runs in an isolated background window with a live chat preview, keeping the main screen and cursor free for uninterrupted work. Users define which apps Gemini can access, monitor every step in real time, pause execution, and approve key actions before they happen. Google lists batch photo editing, filling shipping forms from local PDFs, and building presentations as example workflows.

Computer Use launches today in English for the US market. The source notes that OpenAI has recently focused on similar desktop automation and is pulling ahead with its Astra model in this specific category.

Google is integrating its telephony capabilities directly into the Tasks module of the Gemini App, moving the underlying Duplex architecture to Gemini 4 Hydrogen and Gemini 4 Neon. The update shifts the calling stack from a hybrid system to a unified stack built entirely on Gemini.

The new architecture addresses the specific constraints of live telephony. Google reports tuning the stack for sub-second latency in noisy acoustic environments, using custom acoustic and conversation cadence models to handle background noise and manage speaking turns. The system also relies on pragmatic grounding to interpret indirect refusals, predict task feasibility based on historical context, and keep calls brief.

Google emphasizes that the calling system must work for the business answering the phone as well as the user. Every call opens with a clear disclosure that an automated system from Google is calling. Merchants can opt out of automated calls through their Google Business Profile settings or by saying "Please stop calling my business" during a call. The system gathers details like happy hour specials or patio seating and feeds them back into Google Maps, reducing repetitive questions for businesses.

The integration extends beyond telephony. Tasks in Gemini now supports broader workflows, including analyzing utility bills, processing transactions via the Agent Payments Protocol, and coordinating with users over email and Rich Communication Services.

Sources

  • research.google: How Google Duplex evolved to also call businesses for you in Gemini
  • blog.google: Introducing Tool Retrieval in the Gemini API
  • the-decoder.com: Google's Gemini can now run Mac apps and finish desktop chores in the background while you keep working
#gemini-api#tool-retrieval#desktop-automation#telephony#google-ai
PE
PUBLISHED BY

PNEUMETRON EDITORIAL

Pneumetron's pipeline pairs AI-assisted drafting with human editorial review before publishing — our goal is to make staying informed easier, not to replace real reporting.

Source Material:research.google, blog.google, the-decoder.com ↗
Share this article
Share
Stay Informed

Never miss a signal.

Subscribe to the Pneumetron Intelligence Digest.

← Previous
AWS Quick and Hermes Agent standardize team workflows with new features
Next →
Nvidia Reports $115.8 Billion Q3 Revenue Amid Vera Rubin Ramp And China Export Halt

More from Technology

View All →
TechnologyJust now
Illustrative photo: server, space, the server room, dark

OpenAI cuts reasoning extraction success to 0.2 percent; Signal blocks mobile screenshots

OpenAI reports adaptive extraction attack success rates dropped to 0.2 percent on GPT-6.2 Astra following new API filters and model training. Signal restricts screenshots on Android and iOS to prevent OS-level AI agents from reading private chats, citing incomplete privacy guarantees.

BY PNEUMETRON3 MIN READ
Read more
Technology1 min ago
Illustrative photo: server, space, the server room, dark

Nvidia Reports $115.8 Billion Q3 Revenue Amid Vera Rubin Ramp And China Export Halt

Nvidia’s third-quarter revenue doubled to $115.8 billion, driven by a 110 percent jump in data center sales and a rapid Vera Rubin platform rollout. The company will halt H200 chip sales in China while navigating a leadership transition and an active board investigation.

BY PNEUMETRON3 MIN READ
Read more
Technology1 min ago
Illustrative photo: switch, it, server, server cabinet

AWS Quick and Hermes Agent standardize team workflows with new features

AWS Quick adds multi-agent orchestration and chat-built flows, while Nous Research’s Hermes Agent introduces a Microsoft Teams integration and reusable skill sharing. Both updates target enterprise teams moving beyond ad-hoc agent usage.

BY PNEUMETRON3 MIN READ
Read more
Technology1 min ago
Illustrative photo: spatialship, future, space, travel

AI hacking frameworks automate attacks at machine speed

AI penetration frameworks automate cyberattacks at machine speed, forcing defenders to adopt automated containment and zero-data-retention architectures. Open-source tools like Artex lower the skill barrier for attackers, enabling rapid system compromises. Defenders must implement automated detection and ephemeral processing to survive.

BY PNEUMETRON5 MIN READ
Read more

Comments

Most Read

01
Technology·3d ago
Nvidia Delays 10-Q Filing Amid SEC Probe and New AI Chip Export Controls
02
Technology·2d ago
Nvidia and AMD report record data center revenue as AI monitoring degrades
03
Technology·2d ago
Anthropic prices IPO at $83, opens trading at $111.50 amid safety disclosures
04
Technology·3d ago
Isomorphic Labs Secures $5B Series C to Advance AI Drug Design Platform
05
Technology·4d ago
California sues OpenAI over rogue AI agents and Hugging Face breach
Daily Digest

Get top AI & tech signals delivered to your inbox every morning.

Subscribe →
Follow Signals
X / TWITTERLINKEDININSTAGRAMYOUTUBETELEGRAM

THE PNEUMETRON

The publication of modern technology, AI, and infrastructure.

AI-assisted, human-reviewed.

NEWS CATEGORIES

  • AI RESEARCH
  • TECHNOLOGY
  • WORLD
  • BUSINESS
  • POLITICS
  • SCIENCE
  • HEALTH
  • SPORTS
  • ENTERTAINMENT

AI & RESEARCH

  • AI RESEARCH
  • INFRASTRUCTURE
  • TOOLS
  • DEEP DIVES
  • TECHNICAL DISPATCHES

RESOURCES

  • ABOUT US
  • CONTACT
  • CAREERS
  • ADVERTISE
Pneumetron.
© 2026 Pneumetron. AI-assisted, human-reviewed.
All rights reserved.
ABOUTTOOLSPRIVACYTERMSCONTACT