Prompt Caching: First vs Repeat Request Costs

What the experiment measures

Prompt Cache reduces reprocessing of the unchanged portion of the input. This could be a system prompt, a set of instructions, a large document, or a stable story. The question being modified is placed after the general prefix.

The experiment requires three states:

A. first request: stable prefix + question 1
B. cache hit:     stable prefix + question 2
C. cache miss:    changed prefix + question 3

Requests A and B use the same model, the same settings and the same cache policy. C changes one character in the cached area or is executed after a confirmed TTL has expired. If you change the model, output and prompt length at the same time, the result cannot be explained by the cache alone.

Want to check the formula on your own usage? You can create a BetterToken account and API Key, take the current rates from the pricing page and perform the first and repeated requests with the same prefix. Then match the input, output, applicable cache Token and consumption in the Dashboard, and first check the cache and TTL rules with API reference and the provider documentation.

OpenAI and Anthropic count cache differently

The same word cache does not mean the same mechanism.

OpenAI prompt caching

In OpenAI supported APIs and models, caching is applied automatically to the appropriate prefix. Usage shows cached tokens inside input details. The code usually does not create a separate cache object, but should keep the common prefix intact. Exact thresholds, retention and discounts are checked on the official Prompt Caching page.

Anthropic prompt caching

Anthropic Messages allows you to mark the cache boundary via cache_control. Usage can separately show cache creation and reading. Minimum size, TTL, block order and cost vary by current contract and model; they need to be verified in Anthropic official documentation.

Do not transfer usage field names or coefficients between two protocols. In the experiment table, write down exactly those categories that were returned by the current endpoint.

Preparing a stable prefix

Collect input from two parts:

STABLE_PREFIX
  system instructions
  tool definitions, if they are needed
  unchanged reference document

DYNAMIC_SUFFIX
  current user question

For the first experience, it is better to remove tools and streaming. They don't interfere with cache automatically, but they do add variables to usage and output.

Prefix must be long enough according to the rules of the selected model. If it is shorter than the minimum threshold, no cache hit will be the expected result. Don't lengthen it with meaningless text in production; For the experiment, use a real document, which is already repeated in the problem.

Before calling, save the hash of the cached part:

import hashlib

prefix_hash = hashlib.sha256(STABLE_PREFIX.encode("utf-8")).hexdigest()
print(prefix_hash)

Hash confirms that A and B received the same prefix without publishing its contents.

Which fields to write down

For each request, store:

• timestamp and request ID;
• Model ID and protocol;
• prefix_hash;
• regular input tokens;
• cache creation/write tokens, if the contract allocates them;
• cache read/cached tokens, if the contract allocates them;
• output tokens;
• actual consumption;
• status and latency only as diagnostic fields.

Latency is not price proof. A quick answer may be a cache miss, and a cache hit may mean waiting in line. The conclusion about the cost is made based on usage and tariff.

First query formula

Let's denote:

I  — regular input tokens
W  — cache write / creation tokens
R  — cache read / cached tokens
O  — output tokens
Pi — regular input price per 1,000,000 tokens
Pw — cache write price per 1,000,000 tokens
Pr — cache read price per 1,000,000 tokens
Po — output price per 1,000,000 tokens

Then the calculation for the endpoint that separates these categories is:

cost = I / 1_000_000 × Pi
     + W / 1_000_000 × Pw
     + R / 1_000_000 × Pr
     + O / 1_000_000 × Po

In the first request, W can be greater than zero, and R can be greater than zero. For automatic caching, the set of fields may be different: use uncached and cached input from the actual usage, do not create a non-existent category.

The price of the first request is sometimes higher than a request without cache, if creating a cache is charged separately. This is not a mistake in itself. The payback only comes after enough readings.

Repeat request formula and payback point

Let:

C0 — cost of the first request that creates the cache
Ch — cost of one request with a cache hit
Cu — cost of one comparable request without cache
n  — total number of requests

Series with one creation and n - 1 hits:

C_cached(n) = C0 + (n - 1) × Ch
C_uncached(n) = n × Cu

The minimum n at which the cache pays off is the first integer with the condition:

C_cached(n) < C_uncached(n)

Do not substitute prices of another model into the formula. If Ch >= Cu, the current configuration does not provide savings; check cache hit, prefix size and tariff categories.

Control cache miss

After A and B, do C. Change only the cached prefix, keeping the model and length of the expected response. The cache-read category must be reduced or disappeared according to the contract, and normal processing or cache creation must change.

Reasons for unexpected miss:

• the symbol or space inside the prefix has changed;
• tool definitions came in a different order;
• system block has moved;
• the model or endpoint has changed;
• the request fell outside the TTL;
• prefix turned out to be shorter than the minimum threshold;
• the client serializes the same data in a different order.

Changing the question after a stable prefix is expected. A change inside the prefix creates a different cache identity.

Why don't we publish "result in dollars"

This article does not have access to the API Key and usage of a specific account, so it does not provide a calculated example for the test performed. Prices, models and caching rules change. Publishing a random number would quickly render a reproducible experiment an obsolete advertisement.

To get your own result:

1. Choose one model and one protocol;
2. open the current BetterToken pricing page;
3. do A, B and C;
4. rewrite usage and consumption from Dashboard;
5. calculate C0, Ch, Cu and the payback point;
   6.Save the inspection date and prefix_hash.

FAQ

Why can the first request with cache cost more?

Some protocols charge cache creation/write separately. The initial surcharge is compensated only by repeated cache reads. See the current price of a specific model.

Why didn't the repeated request get a cache hit?

Check prefix length and immutability, block order, model, endpoint, TTL and minimum threshold. Compare prefix_hash.

Is it possible to compare OpenAI and Anthropic with one usage field?

No. The mechanisms, configurations, and category names differ. Normalize the values into your own I, W, R, and O fields while retaining the original fields.

Does Cache Always Reduce Cost?

No. A short prefix, rare repetitions, frequent changes and a low cache hit rate may not pay for creating a cache.

Where can I check the actual debit of BetterToken?

In Dashboard by time, model and request status. Take the tariff from pricing page, and caching rules from the documentation of the corresponding protocol.