Gemini API Tool Retrieval: tool_search, defer_loading, and Billing

Enable Tool Retrieval in preview on the Gemini Interactions API (v1beta) using tool_search and defer_loading. Compare server-side and client-side modes, check the four documented models versus Google blog claims, handle remote MCP and stateless replay, resolve common errors, and evaluate token billing and implicit prefix caching.

Contents
Gemini API Tool Retrieval: tool_search, defer_loading, and Billing

Tool Retrieval is now in preview in the Gemini Interactions API (v1beta), letting agents load tool parameter schemas on demand with tool_search and defer_loading. It affects developers connecting dozens of custom functions or remote MCP servers to Gemini, and it is available only on the Interactions API (not generateContent). To try it as of 2026-11-20, call v1beta/interactions with one of the four documented models (gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, or gemini-3.6-flash), add {"type": "tool_search"}, and set defer_loading: true on tools outside your core set.

Choose the right API route first

Google announced Tool Retrieval on November 19, 2026 on the Google blog and published setup instructions in the AI Studio Tool retrieval documentation. Because the feature is locked to the Interactions API preview (v1beta), your existing endpoint determines whether you can adopt it immediately.

RouteAccount and regionBilling unitTool Retrieval supportWhen to choose
Official Gemini Interactions API (v1beta)Google AI Studio API key in a supported region; Paid Tier required for gemini-4-argonPer 1M input, cached, and output tokens (Standard Free Tier available on gemini-4-hydrogen, gemini-3.8-flash, gemini-3.6-flash)Supported in preview (tool_search, defer_loading, remote mcp_server)You have a large tool library and can call v1beta/interactions directly
Official Gemini generateContent APISame Google AI Studio account and supported regionPer 1M input, cached, and output tokensNot supportedYou have fewer than ~20 stable tools, need explicit caching on supported Gemini 3 or 2.5 models, or cannot migrate yet
Pay-as-you-go API gateway (such as BetterToken)BetterToken account and API key created in the consolePay-as-you-go balance charged by each model’s current billing unitInteractions API is not documented; Tool Retrieval support is unverifiedYou stay on documented native Gemini generateContent or OpenAI-compatible Chat Completions endpoints across multiple models (check per-model tool support in BetterToken docs)

If you stay on standard Gemini generateContent or OpenAI-compatible Chat Completions instead of migrating to the Interactions API, you can check BetterToken’s Gemini API reference for documented endpoints, model IDs, and per-model tool support.

Supported models and the Gemini 4 400 error

Google’s November 19, 2026 announcement blog states that Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, and links to the Latest Gemini models page. That page lists four Gemini 4 models: gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, and gemini-4-argon.

As of 2026-11-20, however, the official Tool retrieval supported models list includes only four model IDs:

  • gemini-4-hydrogen
  • gemini-4-argon
  • gemini-3.8-flash
  • gemini-3.6-flash

gemini-4-carbon and gemini-4-neon are not listed on the Tool Retrieval documentation page as of 2026-11-20. Sending defer_loading to an unsupported model returns an HTTP 400 error with the message deferred loading is not supported for this model.

Start your integration with gemini-4-hydrogen, which Google uses across the Tool Retrieval examples; see our Gemini 4 Hydrogen pricing and migration guide if you are switching from Flash models. If your workflow relies on gemini-4-carbon, verify live behavior on v1beta before enabling defer_loading, or review Gemini 4 Carbon API pricing and migration while keeping static tool loading.

Server-side vs. client-side retrieval

The Interactions API supports two mutually exclusive retrieval modes: server-side retrieval (the default) and client-side retrieval (execution: "client"). Both keep schemas out of the initial prompt, but they differ in latency, setup, and tool registration.

DimensionStatic tool loadingServer-side retrievalClient-side retrieval
How schemas reach the modelAll sent upfrontGemini loads deferred schemas on demandYour app returns schemas in function_result
Retrieval code to writeNoneNoneCustom search, ranking, and filtering
Tools declared upfrontYesYesNo
Remote MCP supportYesYes (defer_loading: true on mcp_server)Not applicable
Extra network round tripsNoneNone (runs inside one API call)One round trip per search
Best fitFewer than ~20 stable toolsLarge, fixed tool libraries or multiple MCP serversProprietary registries, per-user permissions, dynamic mid-session tools

You can declare only one tool_search entry per request. Combining server-side and client-side tool_search entries in the same request returns the error Cannot specify both client-side and server-side tool_search tools..

Set up server-side retrieval and remote MCP

Server-side retrieval requires the v1beta API version. Although the snippet in Google’s blog post omits api_version, the official Tool retrieval guide states that preview Tool Retrieval requires v1beta: in Python, initialize genai.Client(http_options={"api_version": "v1beta"}); in JavaScript, initialize new GoogleGenAI({ httpOptions: { apiVersion: "v1beta" } }).

In Bash, load your Gemini API key without echoing it to the terminal, then send a POST request to https://generativelanguage.googleapis.com/v1beta/interactions. The example below pairs a deferred local function (convert_currency) with Google’s documented demo weather MCP server (https://gemini-api-demos.uc.r.appspot.com/mcp):

read -rs GEMINI_API_KEY && export GEMINI_API_KEY

curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-4-hydrogen",
    "input": "Check the temperature in London and convert 100 USD to EUR.",
    "tools": [
      {"type": "tool_search"},
      {
        "type": "function",
        "name": "convert_currency",
        "description": "Convert an amount from one currency to another using live mid-market rates.",
        "short_description": "Convert an amount between currencies.",
        "defer_loading": true,
        "parameters": {
          "type": "object",
          "properties": {
            "amount": {"type": "number"},
            "from": {"type": "string"},
            "to": {"type": "string"}
          },
          "required": ["amount", "from", "to"]
        }
      },
      {
        "type": "mcp_server",
        "name": "weather_service",
        "url": "https://gemini-api-demos.uc.r.appspot.com/mcp",
        "defer_loading": true
      }
    ]
  }'

Follow four rules when building the tools array:

  • Leave name, description, and parameters unset on a server-side tool_search object.
  • Send the complete parameters schema on every deferred function. Unloaded schemas are not billed, and a tool sent without parameters comes back from a search with an empty schema the model cannot call.
  • Use optional short_description only alongside defer_loading: true to replace a verbose description in the compact deferred list.
  • For remote MCP servers, connect Streamable HTTP endpoints only (SSE servers are not supported) and use snake_case server names without -, as documented in Remote MCP. If your remote server fails handshake or schema validation, use our MCP server integration troubleshooting guide.

Verify the server-side response

A successful server-side retrieval response returns tool_search_call and tool_search_result steps inside steps in the same API call, followed by the actual tool call. For remote MCP tools such as get_weather on the demo server, the API also runs the remote call and returns mcp_server_tool_call and mcp_server_tool_result steps in the same response.

For your own functions, look for function_call items directly inside steps. Do not rely only on status == "requires_action", because Google notes that status is not always requires_action when a function_call step is present. Execute the function locally and send a function_result on the next turn.

Set up client-side retrieval

Use execution: "client" when your tool catalog depends on user permissions, internal namespaces, or integrations added after a session starts. Both name and description are required on a client-side tool_search definition (omitting either returns a 400 error), and you must not set defer_loading on any tool in the request.

The Python snippet below is a minimal protocol example that returns a fixed declaration rather than querying a live registry. Confirm that the round trip works when the script prints a function_call step for get_billing_history with {"user_id": 123} from followup.steps:

from google import genai

client = genai.Client(http_options={"api_version": "v1beta"})

tools = [
    {
        "type": "tool_search",
        "execution": "client",
        "name": "search_tool_registry",
        "description": "Search internal tool namespaces such as billing and support when needed.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "namespace": {"type": "string"},
            },
            "required": ["query"],
        },
    }
]

interaction = client.interactions.create(
    model="gemini-4-hydrogen",
    input="Look up the billing history for user 123.",
    tools=tools,
)

for step in interaction.steps:
    if step.type == "function_call" and step.name == "search_tool_registry":
        # Minimal protocol example: replace with your own registry or permission filter.
        matches = [
            {
                "type": "function",
                "name": "get_billing_history",
                "description": "Retrieve billing history for a given user.",
                "parameters": {
                    "type": "object",
                    "properties": {"user_id": {"type": "integer"}},
                    "required": ["user_id"],
                },
            }
        ]
        followup = client.interactions.create(
            model="gemini-4-hydrogen",
            previous_interaction_id=interaction.id,
            tools=tools,
            input=[
                {
                    "type": "function_result",
                    "name": step.name,
                    "call_id": step.id,
                    "result": matches,
                }
            ],
        )
        for next_step in followup.steps:
            if next_step.type == "function_call":
                print(next_step.name, next_step.arguments)

Return complete function declarations as JSON objects inside the function_result.result array, not as a serialized JSON string, and include the matching name and call_id from the search function_call. If your client-side search finds no match, return a short text result saying nothing matched and listing what the model can search for so it can adjust its query.

Multi-turn state, stateless replay, and common errors

Three preview constraints cause most integration failures after the first turn:

  • Resend the full tools array on every request. The Interactions API stores step history when you pass previous_interaction_id, but it does not store tool declarations between requests. Leaving tools out on follow-up turns causes the model to miss undeclared tools or emit internal retriever: calls in server-side mode.
  • Loaded tools cannot be unloaded, and server-side misses return text. Once the model loads a tool, its schema stays in context for the rest of the conversation. If a server-side query matches no deferred tool, the API returns a plain text response rather than a structured retrieval error, so do not rely on tool_search_result to detect a miss.
  • Circulate steps and signatures verbatim when store=false. On each stateless follow-up, pass the full step history in input and resend the full tools array. Append every response step (tool_search_call, tool_search_result, thought, and function_call) unchanged in its original order with its signature intact, followed by the new function_result. Sending only function_result drops the retrieval context and triggers a 400 error. Use an SDK version that recognizes tool_search_call and tool_search_result, because older SDKs parse them as unknown steps and drop their fields. Keep any tool named in system instructions or tool output undeferred, or the model may call it without searching and fail on function_result.

Google documents HTTP 400 responses for unsupported models, missing server-side tool_search with defer_loading, short_description without defer_loading: true, omitted client-side name or description, missing circulated stateless steps, and stripped signature values. Use the table below to match each documented error message to its cause and fix:

Documented error messageCauseFix
deferred loading is not supported for this modeldefer_loading sent on a model outside the supported listSwitch to gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, or gemini-3.6-flash
tool_search tool must be specified when using defer_loading.A function or mcp_server tool sets defer_loading: true without any tool_search tool in the requestAdd {"type": "tool_search"} or remove defer_loading
When defer_loading is set on a FunctionDeclaration, a Tool with tool_search must also be provided in the request.A function sets defer_loading: true without a server-side tool_search (for example, only execution: "client" is declared or a follow-up request omitted tool_search)Remove defer_loading in client-side mode, or resend the full tools array including {"type": "tool_search"}
Cannot specify both client-side and server-side tool_search tools.Request includes both a server-side and a client-side tool_searchDeclare exactly one tool_search entry per request
ToolSearch name must not be specified when execution mode is SERVER_SIDE.A server-side tool_search sets name (the same error is returned for description and parameters)Remove the field; if switching to execution: "client", also provide both name and description and remove defer_loading from all tools
short_description can only be set if defer_loading is trueA function sets short_description without defer_loading: trueAdd defer_loading: true or remove short_description
Server side tool call context must be circulated when using deferred loading functions.In a stateless request, the history includes a function_result for a deferred tool without the tool_search_call and tool_search_result steps that loaded itSend back every tool_search_call and tool_search_result step unchanged, including signature
Server side tool call context must be circulated ... Missing circulated context for functions: FUNCTION_NAME (even when all steps were sent back)The model called a deferred function that no tool_search loaded, usually because its name appeared in system instructions or another tool’s outputKeep tools mentioned by name undeferred
Tool call part is missing thought_signature. / Tool response part is missing thought_signature.In a stateless request, a tool_search_call (call part) or tool_search_result (response part) step was sent back without its signature fieldSend both retrieval steps back unchanged with their signature values intact

Billing fields, prefix caching, and break-even math

Tool Retrieval has no separate feature fee. Every token is billed at the standard input, cached, and output rates on the Gemini API pricing page. As of 2026-11-20, gemini-4-hydrogen costs $1.50 per 1M input tokens, $0.075 per 1M cached input tokens, and $7.50 per 1M output tokens (including thinking tokens) on the Paid Tier. Standard Free Tier access is available for gemini-4-hydrogen, gemini-3.8-flash, and gemini-3.6-flash, while gemini-4-argon is Paid Tier only at $2.00 input, $0.10 cached, and $10.00 output per 1M tokens through November 30, 2026, increasing to $4.00 / $0.20 / $20.00 starting December 1, 2026.

Check five usage fields on each interaction response to track token counts:

  • total_input_tokens: the prompt of the first model call in the request, including your input, conversation history, system instructions, undeferred tool schemas, the small fixed tool_search instruction block, and one-line entries for deferred tools (name plus short_description or the first sentence of description, roughly 26 tokens per deferred tool in Google’s early testing).
  • total_tool_use_tokens: the prompts of additional model calls the API runs inside the same request after a server-side tool executes, including each server-side tool search. Each call sends the full context again plus newly loaded schemas and is billed at the input rate; multiple searches in one turn add multiple passes.
  • total_cached_tokens: the part of the prompt served from the implicit cache, billed at the cached rate instead of the input rate.
  • total_output_tokens: generated response tokens, billed at the output rate.
  • total_thought_tokens: generated thinking tokens, billed at the output rate.

Prefix caching threshold caveat

The Interactions API supports implicit caching only; explicit caching is not available on the Interactions API or on Gemini 4 models, per Context caching. Because retrieved schemas are added where the model loads them rather than at the start of the prompt, your system instructions, tool list, and earlier turns stay an unchanged prefix across turns. Once that prefix exceeds the model’s minimum cacheable size, later requests in the conversation, including the extra model call made by each server-side search, can be served from the implicit cache and reported in total_cached_tokens.

However, all four supported Tool Retrieval models require a minimum of 4,096 input tokens for implicit caching as of 2026-11-20. If your prompt only exceeded 4,096 tokens because of full tool schemas, deferring those tools can drop your request below the minimum cacheable size. On multi-turn and agent loops with Gemini 4 models, Gemini caches the full previous request only if the new request adds at least 2,050 tokens (model output, thought, and function results all count toward this total); for smaller additions, Gemini re-processes the last 2,050 tokens of the previous request at the standard input rate.

When Tool Retrieval saves tokens vs. costs more

Google’s documentation gives an illustrative two-request estimate (round numbers, excluding output tokens and caching) where the model loads two 400-token tools and returns a function_call on request 1 followed by function_result on request 2:

Illustrative scenario (Google docs estimate)Static loading (input tokens)Tool retrieval (input + tool-use tokens)Net result
200 tools (400 tokens each), 2,000-token context2 × (2,000 + 80,000) = 164,000Request 1: 7,350 + search pass 8,150; Request 2: 8,150 = ~23,650Saves ~86% of input-side tokens
10 tools (400 tokens each), 30,000-token context2 × (30,000 + 4,000) = 68,000Request 1: 30,410 + search pass 31,210; Request 2: 31,210 = ~92,830Costs ~37% more input-side tokens

Deferring tools pays off when your library is large and most turns touch only a few tools. With a small toolset or a long conversation that searches on nearly every turn, the extra server-side search pass (total_tool_use_tokens) can outweigh the savings. Keep your highest-frequency tools loaded upfront, defer the long tail, and measure total_input_tokens, total_tool_use_tokens, and total_cached_tokens on your own traffic alongside actual billed cost before rolling out broadly.

Ready to optimize your LLM workflow?

Join thousands of developers building faster, smarter, and more cost-effective AI applications with BetterToken.

Get Started for Free