Gemini API Tool Retrieval: tool_search, defer_loading, and Billing
Enable Tool Retrieval in preview on the Gemini Interactions API (v1beta) using tool_search and defer_loading. Compare server-side and client-side modes, check the four documented models versus Google blog claims, handle remote MCP and stateless replay, resolve common errors, and evaluate token billing and implicit prefix caching.
Contents

Tool Retrieval is now in preview in the Gemini Interactions API (v1beta), letting agents load tool parameter schemas on demand with tool_search and defer_loading. It affects developers connecting dozens of custom functions or remote MCP servers to Gemini, and it is available only on the Interactions API (not generateContent). To try it as of 2026-11-20, call v1beta/interactions with one of the four documented models (gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, or gemini-3.6-flash), add {"type": "tool_search"}, and set defer_loading: true on tools outside your core set.
Choose the right API route first
Google announced Tool Retrieval on November 19, 2026 on the Google blog and published setup instructions in the AI Studio Tool retrieval documentation. Because the feature is locked to the Interactions API preview (v1beta), your existing endpoint determines whether you can adopt it immediately.
| Route | Account and region | Billing unit | Tool Retrieval support | When to choose |
|---|---|---|---|---|
Official Gemini Interactions API (v1beta) | Google AI Studio API key in a supported region; Paid Tier required for gemini-4-argon | Per 1M input, cached, and output tokens (Standard Free Tier available on gemini-4-hydrogen, gemini-3.8-flash, gemini-3.6-flash) | Supported in preview (tool_search, defer_loading, remote mcp_server) | You have a large tool library and can call v1beta/interactions directly |
Official Gemini generateContent API | Same Google AI Studio account and supported region | Per 1M input, cached, and output tokens | Not supported | You have fewer than ~20 stable tools, need explicit caching on supported Gemini 3 or 2.5 models, or cannot migrate yet |
| Pay-as-you-go API gateway (such as BetterToken) | BetterToken account and API key created in the console | Pay-as-you-go balance charged by each model’s current billing unit | Interactions API is not documented; Tool Retrieval support is unverified | You stay on documented native Gemini generateContent or OpenAI-compatible Chat Completions endpoints across multiple models (check per-model tool support in BetterToken docs) |
If you stay on standard Gemini generateContent or OpenAI-compatible Chat Completions instead of migrating to the Interactions API, you can check BetterToken’s Gemini API reference for documented endpoints, model IDs, and per-model tool support.
Supported models and the Gemini 4 400 error
Google’s November 19, 2026 announcement blog states that Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, and links to the Latest Gemini models page. That page lists four Gemini 4 models: gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, and gemini-4-argon.
As of 2026-11-20, however, the official Tool retrieval supported models list includes only four model IDs:
gemini-4-hydrogengemini-4-argongemini-3.8-flashgemini-3.6-flash
gemini-4-carbon and gemini-4-neon are not listed on the Tool Retrieval documentation page as of 2026-11-20. Sending defer_loading to an unsupported model returns an HTTP 400 error with the message deferred loading is not supported for this model.
Start your integration with gemini-4-hydrogen, which Google uses across the Tool Retrieval examples; see our Gemini 4 Hydrogen pricing and migration guide if you are switching from Flash models. If your workflow relies on gemini-4-carbon, verify live behavior on v1beta before enabling defer_loading, or review Gemini 4 Carbon API pricing and migration while keeping static tool loading.
Server-side vs. client-side retrieval
The Interactions API supports two mutually exclusive retrieval modes: server-side retrieval (the default) and client-side retrieval (execution: "client"). Both keep schemas out of the initial prompt, but they differ in latency, setup, and tool registration.
| Dimension | Static tool loading | Server-side retrieval | Client-side retrieval |
|---|---|---|---|
| How schemas reach the model | All sent upfront | Gemini loads deferred schemas on demand | Your app returns schemas in function_result |
| Retrieval code to write | None | None | Custom search, ranking, and filtering |
| Tools declared upfront | Yes | Yes | No |
| Remote MCP support | Yes | Yes (defer_loading: true on mcp_server) | Not applicable |
| Extra network round trips | None | None (runs inside one API call) | One round trip per search |
| Best fit | Fewer than ~20 stable tools | Large, fixed tool libraries or multiple MCP servers | Proprietary registries, per-user permissions, dynamic mid-session tools |
You can declare only one tool_search entry per request. Combining server-side and client-side tool_search entries in the same request returns the error Cannot specify both client-side and server-side tool_search tools..
Set up server-side retrieval and remote MCP
Server-side retrieval requires the v1beta API version. Although the snippet in Google’s blog post omits api_version, the official Tool retrieval guide states that preview Tool Retrieval requires v1beta: in Python, initialize genai.Client(http_options={"api_version": "v1beta"}); in JavaScript, initialize new GoogleGenAI({ httpOptions: { apiVersion: "v1beta" } }).
In Bash, load your Gemini API key without echoing it to the terminal, then send a POST request to https://generativelanguage.googleapis.com/v1beta/interactions. The example below pairs a deferred local function (convert_currency) with Google’s documented demo weather MCP server (https://gemini-api-demos.uc.r.appspot.com/mcp):
read -rs GEMINI_API_KEY && export GEMINI_API_KEY
curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
-H "x-goog-api-key: $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gemini-4-hydrogen",
"input": "Check the temperature in London and convert 100 USD to EUR.",
"tools": [
{"type": "tool_search"},
{
"type": "function",
"name": "convert_currency",
"description": "Convert an amount from one currency to another using live mid-market rates.",
"short_description": "Convert an amount between currencies.",
"defer_loading": true,
"parameters": {
"type": "object",
"properties": {
"amount": {"type": "number"},
"from": {"type": "string"},
"to": {"type": "string"}
},
"required": ["amount", "from", "to"]
}
},
{
"type": "mcp_server",
"name": "weather_service",
"url": "https://gemini-api-demos.uc.r.appspot.com/mcp",
"defer_loading": true
}
]
}'
Follow four rules when building the tools array:
- Leave
name,description, andparametersunset on a server-sidetool_searchobject. - Send the complete
parametersschema on every deferredfunction. Unloaded schemas are not billed, and a tool sent without parameters comes back from a search with an empty schema the model cannot call. - Use optional
short_descriptiononly alongsidedefer_loading: trueto replace a verbosedescriptionin the compact deferred list. - For remote MCP servers, connect Streamable HTTP endpoints only (SSE servers are not supported) and use
snake_caseserver names without-, as documented in Remote MCP. If your remote server fails handshake or schema validation, use our MCP server integration troubleshooting guide.
Verify the server-side response
A successful server-side retrieval response returns tool_search_call and tool_search_result steps inside steps in the same API call, followed by the actual tool call. For remote MCP tools such as get_weather on the demo server, the API also runs the remote call and returns mcp_server_tool_call and mcp_server_tool_result steps in the same response.
For your own functions, look for function_call items directly inside steps. Do not rely only on status == "requires_action", because Google notes that status is not always requires_action when a function_call step is present. Execute the function locally and send a function_result on the next turn.
Set up client-side retrieval
Use execution: "client" when your tool catalog depends on user permissions, internal namespaces, or integrations added after a session starts. Both name and description are required on a client-side tool_search definition (omitting either returns a 400 error), and you must not set defer_loading on any tool in the request.
The Python snippet below is a minimal protocol example that returns a fixed declaration rather than querying a live registry. Confirm that the round trip works when the script prints a function_call step for get_billing_history with {"user_id": 123} from followup.steps:
from google import genai
client = genai.Client(http_options={"api_version": "v1beta"})
tools = [
{
"type": "tool_search",
"execution": "client",
"name": "search_tool_registry",
"description": "Search internal tool namespaces such as billing and support when needed.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"},
"namespace": {"type": "string"},
},
"required": ["query"],
},
}
]
interaction = client.interactions.create(
model="gemini-4-hydrogen",
input="Look up the billing history for user 123.",
tools=tools,
)
for step in interaction.steps:
if step.type == "function_call" and step.name == "search_tool_registry":
# Minimal protocol example: replace with your own registry or permission filter.
matches = [
{
"type": "function",
"name": "get_billing_history",
"description": "Retrieve billing history for a given user.",
"parameters": {
"type": "object",
"properties": {"user_id": {"type": "integer"}},
"required": ["user_id"],
},
}
]
followup = client.interactions.create(
model="gemini-4-hydrogen",
previous_interaction_id=interaction.id,
tools=tools,
input=[
{
"type": "function_result",
"name": step.name,
"call_id": step.id,
"result": matches,
}
],
)
for next_step in followup.steps:
if next_step.type == "function_call":
print(next_step.name, next_step.arguments)
Return complete function declarations as JSON objects inside the function_result.result array, not as a serialized JSON string, and include the matching name and call_id from the search function_call. If your client-side search finds no match, return a short text result saying nothing matched and listing what the model can search for so it can adjust its query.
Multi-turn state, stateless replay, and common errors
Three preview constraints cause most integration failures after the first turn:
- Resend the full
toolsarray on every request. The Interactions API stores step history when you passprevious_interaction_id, but it does not store tool declarations between requests. Leaving tools out on follow-up turns causes the model to miss undeclared tools or emit internalretriever:calls in server-side mode. - Loaded tools cannot be unloaded, and server-side misses return text. Once the model loads a tool, its schema stays in context for the rest of the conversation. If a server-side query matches no deferred tool, the API returns a plain text response rather than a structured retrieval error, so do not rely on
tool_search_resultto detect a miss. - Circulate steps and signatures verbatim when
store=false. On each stateless follow-up, pass the full step history ininputand resend the fulltoolsarray. Append every response step (tool_search_call,tool_search_result,thought, andfunction_call) unchanged in its original order with itssignatureintact, followed by the newfunction_result. Sending onlyfunction_resultdrops the retrieval context and triggers a400error. Use an SDK version that recognizestool_search_callandtool_search_result, because older SDKs parse them as unknown steps and drop their fields. Keep any tool named in system instructions or tool output undeferred, or the model may call it without searching and fail onfunction_result.
Google documents HTTP 400 responses for unsupported models, missing server-side tool_search with defer_loading, short_description without defer_loading: true, omitted client-side name or description, missing circulated stateless steps, and stripped signature values. Use the table below to match each documented error message to its cause and fix:
| Documented error message | Cause | Fix |
|---|---|---|
deferred loading is not supported for this model | defer_loading sent on a model outside the supported list | Switch to gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, or gemini-3.6-flash |
tool_search tool must be specified when using defer_loading. | A function or mcp_server tool sets defer_loading: true without any tool_search tool in the request | Add {"type": "tool_search"} or remove defer_loading |
When defer_loading is set on a FunctionDeclaration, a Tool with tool_search must also be provided in the request. | A function sets defer_loading: true without a server-side tool_search (for example, only execution: "client" is declared or a follow-up request omitted tool_search) | Remove defer_loading in client-side mode, or resend the full tools array including {"type": "tool_search"} |
Cannot specify both client-side and server-side tool_search tools. | Request includes both a server-side and a client-side tool_search | Declare exactly one tool_search entry per request |
ToolSearch name must not be specified when execution mode is SERVER_SIDE. | A server-side tool_search sets name (the same error is returned for description and parameters) | Remove the field; if switching to execution: "client", also provide both name and description and remove defer_loading from all tools |
short_description can only be set if defer_loading is true | A function sets short_description without defer_loading: true | Add defer_loading: true or remove short_description |
Server side tool call context must be circulated when using deferred loading functions. | In a stateless request, the history includes a function_result for a deferred tool without the tool_search_call and tool_search_result steps that loaded it | Send back every tool_search_call and tool_search_result step unchanged, including signature |
Server side tool call context must be circulated ... Missing circulated context for functions: FUNCTION_NAME (even when all steps were sent back) | The model called a deferred function that no tool_search loaded, usually because its name appeared in system instructions or another tool’s output | Keep tools mentioned by name undeferred |
Tool call part is missing thought_signature. / Tool response part is missing thought_signature. | In a stateless request, a tool_search_call (call part) or tool_search_result (response part) step was sent back without its signature field | Send both retrieval steps back unchanged with their signature values intact |
Billing fields, prefix caching, and break-even math
Tool Retrieval has no separate feature fee. Every token is billed at the standard input, cached, and output rates on the Gemini API pricing page. As of 2026-11-20, gemini-4-hydrogen costs $1.50 per 1M input tokens, $0.075 per 1M cached input tokens, and $7.50 per 1M output tokens (including thinking tokens) on the Paid Tier. Standard Free Tier access is available for gemini-4-hydrogen, gemini-3.8-flash, and gemini-3.6-flash, while gemini-4-argon is Paid Tier only at $2.00 input, $0.10 cached, and $10.00 output per 1M tokens through November 30, 2026, increasing to $4.00 / $0.20 / $20.00 starting December 1, 2026.
Check five usage fields on each interaction response to track token counts:
total_input_tokens: the prompt of the first model call in the request, including your input, conversation history, system instructions, undeferred tool schemas, the small fixedtool_searchinstruction block, and one-line entries for deferred tools (nameplusshort_descriptionor the first sentence ofdescription, roughly 26 tokens per deferred tool in Google’s early testing).total_tool_use_tokens: the prompts of additional model calls the API runs inside the same request after a server-side tool executes, including each server-side tool search. Each call sends the full context again plus newly loaded schemas and is billed at the input rate; multiple searches in one turn add multiple passes.total_cached_tokens: the part of the prompt served from the implicit cache, billed at the cached rate instead of the input rate.total_output_tokens: generated response tokens, billed at the output rate.total_thought_tokens: generated thinking tokens, billed at the output rate.
Prefix caching threshold caveat
The Interactions API supports implicit caching only; explicit caching is not available on the Interactions API or on Gemini 4 models, per Context caching. Because retrieved schemas are added where the model loads them rather than at the start of the prompt, your system instructions, tool list, and earlier turns stay an unchanged prefix across turns. Once that prefix exceeds the model’s minimum cacheable size, later requests in the conversation, including the extra model call made by each server-side search, can be served from the implicit cache and reported in total_cached_tokens.
However, all four supported Tool Retrieval models require a minimum of 4,096 input tokens for implicit caching as of 2026-11-20. If your prompt only exceeded 4,096 tokens because of full tool schemas, deferring those tools can drop your request below the minimum cacheable size. On multi-turn and agent loops with Gemini 4 models, Gemini caches the full previous request only if the new request adds at least 2,050 tokens (model output, thought, and function results all count toward this total); for smaller additions, Gemini re-processes the last 2,050 tokens of the previous request at the standard input rate.
When Tool Retrieval saves tokens vs. costs more
Google’s documentation gives an illustrative two-request estimate (round numbers, excluding output tokens and caching) where the model loads two 400-token tools and returns a function_call on request 1 followed by function_result on request 2:
| Illustrative scenario (Google docs estimate) | Static loading (input tokens) | Tool retrieval (input + tool-use tokens) | Net result |
|---|---|---|---|
| 200 tools (400 tokens each), 2,000-token context | 2 × (2,000 + 80,000) = 164,000 | Request 1: 7,350 + search pass 8,150; Request 2: 8,150 = ~23,650 | Saves ~86% of input-side tokens |
| 10 tools (400 tokens each), 30,000-token context | 2 × (30,000 + 4,000) = 68,000 | Request 1: 30,410 + search pass 31,210; Request 2: 31,210 = ~92,830 | Costs ~37% more input-side tokens |
Deferring tools pays off when your library is large and most turns touch only a few tools. With a small toolset or a long conversation that searches on nearly every turn, the extra server-side search pass (total_tool_use_tokens) can outweigh the savings. Keep your highest-frequency tools loaded upfront, defer the long tail, and measure total_input_tokens, total_tool_use_tokens, and total_cached_tokens on your own traffic alongside actual billed cost before rolling out broadly.