Does caching happen automatically, or do you have to design for it?

The cached rate applies automatically whenever a request's prefix exactly matches one already sent, but getting the benefit takes application-level discipline: keep the system prompt and tool schema first and unchanged, and append rather than edit earlier context [3].

How much of a real bill can caching remove?

Independent research on repeated-prefix workloads puts the range at 41-80% depending on how much of the prompt actually repeats and how consistently it's structured [1]; a workload with little repeated context has correspondingly less for caching to remove.

Resources · AI Token Router

Cost optimizationPricingOpen-weight

Prompt Caching: Why It's the Biggest Lever in Your LLM Bill

Prompt caching is an exact-prefix match: send the same system prompt, tool schema and transcript prefix as an earlier request, in the same order, with nothing changed ahead of it, and that portion bills at a discounted cached rate instead of the standard one. An academic study of 500+ agent sessions found this alone cut API cost 41-80% and improved time-to-first-token 13-31% [1], and one documented case took a support-agent bill from $4,200/mo to $680/mo after a single afternoon of restructuring [2]. On this catalogue, caching is why the agent-workload discount runs 65–68% against a 36–42% rate-card discount for the same models -- the mechanism, not a different set of prices.

September 8, 2026

Put your own numbers in before you take ours on trust.

Model your own cache hit rate

The mechanism, and where it breaks

Caching only applies to an exact-prefix match: the cached portion of a request has to be byte-identical to a prefix already sent, in the same order, with nothing inserted before it. Reorder the tool schema, edit a line early in a long system prompt, or restructure how context gets assembled, and the cache for that prefix invalidates -- the next request is billed at the standard rate as if nothing had ever been cached [3].

That is a real constraint on how an agent's prompt gets built, not a footnote: appending new turns to the end of a transcript is cache-safe, and editing anything upstream of the append point is not. A system that mutates its own system prompt mid-session -- to inject a dynamically updated tool list, for instance -- is quietly paying the standard rate on every request regardless of what the rate card's cached column says.

What independent research says it's worth

An academic study covering 500+ agent sessions with 10,000-token system prompts found caching cut API cost by 41-80% and improved time-to-first-token by 13-31%, independent of which underlying model was used [1]. A separate documented case describes a support-agent bill falling from $4,200/mo to $680/mo after one afternoon of prompt restructuring -- no model change, just moving the static parts of the prompt ahead of the parts that change on every request [2].

Provider support for some form of caching is close to universal by September 2026. Every text model in this catalogue publishes a cached input rate as its own column in the rate table, not a negotiated or hidden discount.

Agent-workload discount, callable text models

Caching is not a marginal tweak on any one of these -- it widens the discount by a comparable amount across very different price points, because the mechanism is the same regardless of what the base rate is. Kimi K2.6 carries the deepest discount in this set at 68%, against 65% for DeepSeek V4 Flash. Figures assume 400M input tokens a month with a 60% repeated prefix, and 8M output.Computed from our published rates against the official rate, on the stated workload.

What that's worth on a single model's bill

Turning that into dollars on one model puts a number on the base-rate gap the caching discount builds on top of. Priced at the standard rate on both sides -- our rate against the model publisher's own -- at the volume this site's own agent workload is stated on (400M input tokens a month with a 60% repeated prefix, and 8M output), the figure below is the floor caching works from; the ranking above already shows how much further the cached rate on the repeated prefix pulls the same model's bill down.

One model, the agent workload, before caching's own extra discount

Official rate

$412

Our rate, Kimi K2.6

$238

Pricing the same monthly workload on a single model turns the ranking above into a dollar figure -- the rate-card gap caching then widens further. At 400M input tokens a month with a 60% repeated prefix, and 8M output on Kimi K2.6, that is $238 a month against $412 at the official rate, a difference of $174.Computed from this site's published rates and the model publisher's own.

Sources

[1] "Don't Break the Cache", arXiv 2601.06007 (2026) — arXiv. 500+ agent sessions; caching cut API cost 41-80% and improved time-to-first-token 13-31%.
[2] Prompt Caching in 2026: Cut Your LLM API Costs by Up to 90% — DevToolLab. The $4,200/mo to $680/mo support-agent case after one afternoon of prompt restructuring.
[3] LLM Prompt Caching: Cost Savings, Invalidation & Workload Design — IntuitionLabs. The exact-prefix-match mechanism and what invalidates a cached prefix.

Questions this raises

Does caching happen automatically, or do you have to design for it?: The cached rate applies automatically whenever a request's prefix exactly matches one already sent, but getting the benefit takes application-level discipline: keep the system prompt and tool schema first and unchanged, and append rather than edit earlier context [3].
How much of a real bill can caching remove?: Independent research on repeated-prefix workloads puts the range at 41-80% depending on how much of the prompt actually repeats and how consistently it's structured [1]; a workload with little repeated context has correspondingly less for caching to remove.

AI Token Router is an OpenAI-compatible gateway for open-weight models, priced below each publisher’s own rate on every row.

Model your own cache hit rate Get $5 in free credits

What a Fixed Monthly AI Budget Actually Buys in 2026
Three realistic budget tiers, worked by hand against this catalogue's own rate table, at one stated request shape -- how many requests and tokens $10, $50 and $250 a month actually buys on three callable models.
When a Closed Frontier Model Is Still the Right Call
Closed frontier models measurably lead reasoning-heavy benchmarks as of September 2026. Where that lead and a simpler operational model are worth the higher price -- and why our catalogue is not the answer for that reader.
Self-Hosting vs a Managed Open-Weight API: When Each Wins
Where the self-host breakeven actually sits, what self-hosting really costs once engineering time is priced in, and the honest cases where self-hosting wins -- this is not a blanket argument for a managed API.