Prompt Cache: Cost of the First and Repeated Requests

Measure Prompt Cache cost with a series of controlled requests: the first creates or prepares a reusable prefix, repeated requests attempt to read it, and a control request changes the prefix to produce a miss. Compare usage categories and the actual charge for one model. A fixed savings percentage means little without the Model ID, TTL, prefix length, and current prices.

What the Experiment Measures

Prompt Cache reduces repeated processing of an unchanged input prefix. That prefix may contain a system prompt, instructions, a large document, or stable conversation history. Put the changing question after the common prefix.

The experiment needs three states:

A. first request: stable prefix + question 1
B. cache hit:     stable prefix + question 2
C. cache miss:    changed prefix + question 3

Requests A and B use the same model, settings, and cache policy. Request C changes one character in the cached region or runs after a confirmed TTL expiry. If you also change the model, output length, and prompt, the result cannot be attributed to cache behavior.

Want to test the formula against your own usage? You can create a BetterToken account and API key, take the current rates from the pricing page, and send a first and repeated request with the same prefix. Then compare input, output, applicable cache tokens, and cost in the Dashboard, after checking cache and TTL rules in the API reference and the provider documentation.

OpenAI and Anthropic Cache Differently

The word cache does not imply the same mechanism.

OpenAI prompt caching

For supported OpenAI APIs and models, caching is applied automatically to eligible prefixes. Usage reports cached tokens in input details. The application normally does not create a separate cache object, but it must keep the shared prefix stable. Check current thresholds, retention, and rates in the official Prompt Caching guide.

Anthropic prompt caching

Anthropic Messages can mark a cache boundary with cache_control. Usage may separate cache creation from cache reads. Minimum size, TTL, block order, and pricing depend on the current contract and model; verify them in the Anthropic prompt caching documentation.

Do not copy usage field names or price multipliers between protocols. Record the categories returned by the endpoint you actually called.

Build a Stable Prefix

Split the input into two parts:

STABLE_PREFIX
  system instructions
  tool definitions, when required
  unchanged reference document

DYNAMIC_SUFFIX
  current user question

For the first experiment, remove tools and streaming. They do not necessarily prevent caching, but they introduce more variables in usage and output.

The prefix must satisfy the current model's minimum size. If it is shorter, a missing cache hit may be expected. Do not pad production prompts with meaningless text; use a real document that the workload already repeats.

Store a hash of the cached part before the call:

import hashlib

prefix_hash = hashlib.sha256(STABLE_PREFIX.encode("utf-8")).hexdigest()
print(prefix_hash)

The hash proves that A and B used an identical prefix without disclosing its content.

Record These Fields

For each request, preserve:

  • timestamp and request ID;
  • Model ID and protocol;
  • prefix_hash;
  • ordinary input tokens;
  • cache creation or write tokens when the contract exposes them;
  • cache read or cached tokens when exposed;
  • output tokens;
  • actual charge;
  • status, with latency only as a diagnostic field.

Latency does not prove cost. A fast response may be a cache miss, while a cache hit may wait in a queue. Draw cost conclusions from usage and pricing.

Formula for the First Request

Use these variables:

I  — ordinary input tokens
W  — cache write / creation tokens
R  — cache read / cached tokens
O  — output tokens
Pi — ordinary input price per 1,000,000 tokens
Pw — cache write price per 1,000,000 tokens
Pr — cache read price per 1,000,000 tokens
Po — output price per 1,000,000 tokens

For an endpoint that separates the four categories:

cost = I / 1_000_000 × Pi
     + W / 1_000_000 × Pw
     + R / 1_000_000 × Pr
     + O / 1_000_000 × Po

The first request may have W > 0 and R = 0. An automatically cached API may expose a different shape; use its uncached and cached input values rather than inventing a cache-write field.

The first cached request can cost more than an uncached request when cache creation has a separate rate. That is not necessarily an error. Savings appear only after enough cache reads.

Repeated Requests and Break-Even

Let:

C0 — cost of the first request with cache creation
Ch — cost of one request with a cache hit
Cu — cost of one comparable uncached request
n  — total number of requests

A series with one creation and n - 1 hits costs:

C_cached(n) = C0 + (n - 1) × Ch
C_uncached(n) = n × Cu

The cache breaks even at the first integer n for which:

C_cached(n) < C_uncached(n)

Do not use prices from another model. If Ch >= Cu, the current setup provides no cost reduction; inspect the hit, prefix length, and rate categories.

Run a Control Cache Miss

After A and B, execute C. Change only the cached prefix while preserving the model and expected output length. The cache-read category should shrink or disappear according to the contract, while ordinary processing or cache creation changes.

Unexpected misses commonly come from:

  • a changed character or space inside the prefix;
  • tool definitions in a different order;
  • a moved system block;
  • a different model or endpoint;
  • a request outside the TTL;
  • a prefix below the minimum size;
  • a client serializing the same data in a different order.

Changing the question after the stable prefix is normally expected. Changing content inside the prefix creates a different cache identity.

Why This Article Does Not Publish a Dollar Result

This run had no access to an account API Key or usage record, so it does not present a calculation as a completed test. Prices, models, and cache rules change. A made-up number would turn a reproducible method into stale advertising.

To produce your own result:

  1. select one model and protocol;
  2. open the current BetterToken pricing page;
  3. run A, B, and C;
  4. copy usage and cost from the Dashboard;
  5. calculate C0, Ch, Cu, and break-even;
  6. save the verification date and prefix_hash.

FAQ

Why can the first cached request cost more?

Some protocols price cache creation separately. The initial premium is recovered only through repeated cache reads. Use the current rate for the exact model.

Why did a repeated request miss the cache?

Check prefix length and stability, block order, model, endpoint, TTL, and minimum thresholds. Compare prefix_hash values.

Can OpenAI and Anthropic be compared with one usage field?

No. Their mechanisms, configuration, and usage categories differ. Normalize values into your own I, W, R, and O fields while retaining the original provider fields.

Does cache always reduce cost?

No. A short prefix, infrequent reuse, frequent edits, or a low hit rate may not recover cache creation cost.

Where do I verify a BetterToken charge?

Match the request by time, model, and status in the Dashboard. Take the rate from the pricing page and cache rules from the relevant protocol documentation.