Prompt caching
Prompt caching is the reuse of an already-processed prompt prefix, so that repeated leading tokens — a system prompt, a tool schema, a fixed document — are billed at a reduced cached-input rate instead of the full input rate.
When a model reads a prompt it builds an internal key-value cache over that sequence. If the next request begins with exactly the same tokens, that work has already been done and the cached state can be reused rather than recomputed. The saving is real compute, which is why providers pass a lower rate on rather than treating it as a discount.
The match is on an exact prefix, from the first token. A single changed character near the start invalidates everything after it, so prompt layout decides whether caching happens at all: fixed content first, variable content last. Putting a timestamp or a user id at the top of a system prompt is enough to defeat the cache on every request.
Caches also expire. A prefix that is not reused within the provider's retention window is recomputed at the full rate, so caching helps sustained traffic against a stable prompt far more than it helps sporadic traffic.
Every text model in this catalogue publishes its cached-input rate in the same table as its standard rate, in its own column. A provider that does not publish one is not cheaper; it is a provider whose largest line item on a repeated-prefix workload is unpriced.
What it costs you
On agent and chat workloads the repeated prefix is most of every request, so the cached-input rate rather than the headline input rate is what the monthly bill is largely made of. If four fifths of your input is a stable prefix billed at roughly a fifth of the standard rate, effective input cost falls by about two thirds — a larger swing than the gap between most models' published rates. It is also the only saving here that requires no change to model choice, only to prompt ordering.
Worked from today’s rates
GLM-5.2 bills cached input at $0.16/M against a standard input rate of $0.82/M — about 20% of the uncached price. A 30,000-token system prompt costs $0.0246 the first time and $0.00480 on each cached repeat.
Related terms
- Context windowA context window is the maximum number of tokens a model can hold in one request, counting the system prompt, the conversation history, any retrieved documents and the reply being generated together.
- Input vs output tokensInput tokens are the text you send a model and output tokens are the text it writes back; they are billed at separate rates, and the output rate is almost always several times the input rate.
- Time to first token (TTFT)Time to first token, or TTFT, is the delay between sending a request and receiving the first token of the response, and it is the part of latency a person waiting on a streamed answer actually perceives.
Go deeper
- Cached input pricing, and why it decides what an agent costs
- Coding agents: where cached input dominates
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.