Most of an agent's bill is a prompt it already sent

Why agent bills look nothing like chat bills

A chat request sends a short prompt and reads back a short answer. An agent turn sends the system prompt, the tool schema, the file contents it has already read, the transcript so far, and then reads back a few hundred tokens of decision. Then it does it again. By the tenth tool call the request body is enormous and almost entirely identical to the ninth.

That repetition is the defining economic property of the workload, and it cuts both ways. It is why an agent can burn through a budget faster than anyone estimates from watching it work — the visible output is tiny and the invisible input is not. It is also why an agent is the workload where prompt caching does the most good, because the repeated portion is billed at a fraction of the standard input rate.

The reference workload this page quotes is 400M input tokens a month with a 60% repeated prefix, and 8M output. That cached share is deliberately conservative: a real coding agent with a fixed system prompt and a stable tool schema repeats considerably more than that, and the calculator further down lets you push the assumption up and watch the figure move.

What an agent month is made of

Priced on GLM-5.2 at the reference volume, the bill comes to $190 — and the biggest slice of it is the prefix, not the output. The repeated portion is billed at $0.16/M instead of $0.82/M, which is the single largest lever available on this shape of work.

The practical consequence is that agent cost optimisation is mostly prompt-structure work. Keep the stable material at the front of the request — system prompt first, tool schema next, then retrieved context, then the turn-by-turn transcript — so that the longest possible prefix stays byte-identical between turns. A cache entry lives roughly five minutes after its last use, which a busy loop keeps warm without trying.

Cached input applies automatically when a prefix repeats. There is no header to set, no flag to pass and no plan to be on — the rate is published per model and billed as published.

Move the cache slider

This is the calculator to spend real time in, because agent workloads are the case where the default assumptions are most wrong. Enter your monthly input and output volume, then set the cache slider to the share of input that repeats between turns.

At a cache share of zero the discount is the rate-card figure, 36–42%. Push it toward what a real agent loop repeats and it climbs to 65–68% and beyond. Both ends of that range come from the same catalogue and the same arithmetic; only the shape of the workload changes.

Pointing an agent at this endpoint

Agent frameworks are built on the OpenAI-compatible surface, so the loop itself does not change. Tool definitions, tool_choice and the assistant message carrying tool calls all have the shape your framework already emits.

1. 1

   Swap the base URL in the agent's client

   One constructor argument. The loop keeps its retries, its tool registry and its transcript handling, because none of those touch the transport.

2. 2

   Choose the model for the loop, not for the benchmark

   Kimi K3 holds reasoning together across long tool-use chains and carries the largest context window in the catalogue; DeepSeek V4 Flash is the one to reach for on the high-frequency, low-stakes steps. Mixing them across steps of the same loop is normal and is often the largest saving available after caching.

3. 3

   Put the stable material first and leave it alone

   Order the request so the invariant prefix is byte-identical between turns: system prompt, tool schema, retrieved context, then the transcript. Anything that mutates near the front of the request — a timestamp, a shuffled tool list, a re-serialised schema — silently invalidates the cache for everything after it.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://router.xark.io/api/v1"   # ← the only line that changes
)

response = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=[{"role": "user", "content": "Hello"}]
)

Your prepaid balance is a hard ceiling on every request, which is the property that matters most when the thing spending the money is a loop rather than a person. A runaway agent stops at what you have already paid.

Per-model, on the rate card and on an agent loop

The two numbers below are the same catalogue measured against two different shapes of work. The first assumes every input token is billed fresh, which is true of a one-shot request. The second applies the reference agent workload, where most of the input is a prefix that has already been sent.

Kimi K2.6

42% below the publisher's rate on the rate card, 68% on the reference agent workload. Cached input is $0.11/M against a standard input rate of $0.55/M.

Kimi K3

40% below the publisher's rate on the rate card, 65% on the reference agent workload. Cached input is $0.37/M against a standard input rate of $1.85/M.

GLM-5.2

42% below the publisher's rate on the rate card, 68% on the reference agent workload. Cached input is $0.16/M against a standard input rate of $0.82/M.

DeepSeek V4 Pro

37% below the publisher's rate on the rate card, 66% on the reference agent workload. Cached input is $0.055/M against a standard input rate of $0.28/M.

DeepSeek V4 Flash

36% below the publisher's rate on the rate card, 65% on the reference agent workload. Cached input is $0.018/M against a standard input rate of $0.090/M.

The failure modes we will not paper over

Agent workloads fail in ways chat workloads do not, and the honest list is short enough to print.

No cross-model fallback

If an upstream fails, the request fails with its status and a machine-readable code. We do not retry against a different model, because in an agent loop a silently substituted model produces a plausible-looking trajectory that is wrong in a way the loop cannot detect.

Cache misses are silent by nature

A prefix that changes by one byte is a new prefix. If a bill comes in higher than the calculator predicted, the first thing to check is whether something near the front of the request is mutating between turns.

Availability is published, not assumed

7 of 24 catalogued models are callable today. Building a loop against a model marked unavailable produces an error at call time, so the marker is on every pricing row and every model page.

Per-key spending limits are not shipped

The account balance is the ceiling today, and it is a hard one. Per-key limits are on the roadmap; we would rather say so than let you design around a control that does not exist yet.

What agent teams say

Reserved for quotes from teams running production agent loops who have agreed to describe what happened to their bill.

No customer quote below has been approved for publication yet. Every card in this section is sample copy held in place until a real one replaces it, and none of it is offered as an endorsement.

• Sample — pending customer approval

  Sample placeholder text, awaiting an approved customer quote about running a coding agent against a repeated system prompt.

  Founding engineer, developer tools company

• Sample — pending customer approval

  Sample placeholder text, awaiting an approved customer quote about the cached-input line on the invoice.

  Platform engineer, e-commerce company

A loop, and its cost, side by side

A recording of an agent running a multi-step task with the per-request cost visible next to it, so the cached line can be watched rather than described.

Agent and caching questions

The mechanics, the limits, and the parts that are easy to get wrong in a loop that runs unattended.