Cached input pricing and what an agent costs

What a cached input token is

When two requests begin with the same tokens, the serving stack can reuse the attention state it already computed for that prefix instead of recomputing it. The tokens still count -- they are still in the context the model attends to -- but the compute behind them has already been paid for, so they are billed at a reduced rate.

The mechanism is prefix matching, and that word does the work. The cache hits from the first token forward and stops at the first divergence. Everything after a changed token is a cache miss even if it is byte-identical to a previous request, which makes prompt ordering an actual cost decision rather than a style preference.

The rates

Every text model here prices cached input at a fifth of its uncached input rate, which is also the ratio each model publisher uses on their own official card. The published figures are rounded to something printable, so GLM-5.2's $0.16 sits just under a strict fifth of $0.164.

Cache is stored as a ratio, not as a second price

This is an implementation detail with a consequence worth publishing. The gateway does not hold a cached-input price. It holds a cache ratio, which our sync script derives as the cached rate divided by the input rate and writes to eight decimal places -- 0.19512195 for GLM-5.2, a flat 0.2 for Kimi K3.

Two things follow. A cached rate here structurally cannot exceed the uncached rate, because it is defined as a fraction of it. And repricing input reprices cache in the same write, so the two numbers cannot drift apart -- which is the failure mode of every rate card that stores two independent absolute figures and updates one of them.

It also means the ratio is auditable from the published card. Divide the cached rate by the input rate on any row above and you have the number the billing system is actually using.

The agent-loop arithmetic

The workload where this dominates is the one most people building agents are running: a large fixed system prompt -- tool definitions, schemas, a style guide, a retrieved corpus -- resent on every turn of a conversation. The variable part is a few hundred tokens. The fixed part is tens of thousands.

Take a 40,000-token fixed prefix over 50 turns. Without caching that is 2,000,000 input tokens. With caching the first turn pays full rate and the remaining 49 pay a fifth. The output side is unchanged; this is the input bill only.

How deep the discount goes varies enormously across the market

Our discount is a flat 5x on every model. That is shallower than several competitors advertise on their large models, and the honest reason is arithmetic rather than modesty: a deep cache discount off a high uncached rate and a shallow one off a low uncached rate can land in the same place, so the ratio on its own tells you nothing about what you pay.

The published cache discounts below are from each vendor's own rate card for their own DeepSeek listings, so the absolute prices reflect different SKUs and should not be read as a like-for-like comparison. The ratio column is the point: on the Pro tier the depth ranges from 10x to 30x, and on the Flash tier from 4x to 31x, for models that are nominally the same family.

Structuring a prompt so the cache actually hits

Put everything stable first and everything volatile last. Tool definitions, system instructions and retrieved documents belong at the front; the user turn, a timestamp, a request id or a session variable belong at the end. A timestamp injected into the first line of a system prompt voids the entire prefix on every single request, which is the single most common way a caching discount is engineered away by accident.

Keep the prefix byte-stable. Re-serialising a JSON tool definition with different key ordering produces different tokens and therefore a cache miss, even though the semantic content is identical. Serialise once and reuse the string.

Output is never cached. Caching reduces the cost of what the model reads, not what it writes, so on a model with a high output multiple -- Kimi K3 writes at 4.9x its input rate -- caching cannot rescue a workload whose bill is dominated by generation length. That is a different lever: shorter completions, or a cheaper model for the generating step.

Measuring it rather than assuming it

Every response carries usage.cost_usd, which is the actual charge rather than an estimate, so the way to confirm a caching change worked is to compare that field before and after on the same prompt shape. A prefix cache is a serving-side behaviour, and behaviours you have not measured are behaviours you are guessing about.

If you stream, you must ask for usage explicitly. Without stream_options.include_usage a streaming caller never learns what the call cost at all -- the usage block only exists on the blocking response, or on the one extra frame include_usage adds before the terminator.

stream = client.chat.completions.create(
    model="z-ai/glm-5.2",
    messages=messages,
    stream=True,
    stream_options={"include_usage": True},   # without this, no cost is reported
)

for chunk in stream:
    if chunk.usage:                # populated on exactly one frame
        print(chunk.usage.cost_usd)

When caching is not worth engineering for

If your prefix is small, the discount is small in absolute terms and the restructuring is not worth the risk of breaking a working prompt. At a 2,000-token prefix on DeepSeek V4 Flash, fifty turns of caching saves about a hundredth of a cent.

If your traffic is one-shot rather than conversational -- classification, extraction, a batch job over distinct documents -- there is no repeated prefix to hit, and the cached rate is irrelevant to you. Optimise the uncached input rate and the model choice instead.

And if your bill is mostly output, caching is the wrong lever entirely. Check the output multiple on your model first: on a 2x model like DeepSeek V4 Flash the input side is a real share of the bill, but on a 4.9x model most of what you are paying for is the writing.