System prompt
A system prompt is the instruction block placed first in a request, under the system or developer role, that sets a model's persona, rules and output format for that request — and because the API keeps no state between calls, it is resent and re-billed on every single one.
There is no privileged channel. A system prompt is ordinary tokens in the same sequence as everything else; models are trained to weight instructions arriving under that role more heavily, but the mechanism is training, not transport. It occupies the same context window and is billed at the same input rate as the user's message.
System prompts grow by accretion. Each incident adds a rule, each edge case adds a clause, and nothing is ever removed because nobody can prove which line is holding the behaviour together. A production agent prompt of several thousand tokens is unremarkable, and almost none of it is read by a human again after it is written.
Position is the part that has a price attached. A cache matches on an exact prefix from the first token, so fixed instructions belong at the very top and anything variable — a user id, a timestamp, today's date — belongs after them. A single interpolated value near the start invalidates the prefix on every request.
It is also not a security boundary. Instructions arriving later in the sequence can contradict it, and a model asked to ignore its instructions sometimes will. Anything that must not happen belongs in code around the model, not in a sentence inside it.
What it costs you
A system prompt is the most-repeated text an application owns: it does no per-request work and yet scales exactly with traffic, so it is usually the largest line item nobody has looked at. The two fixes are unglamorous and both large — delete the clauses that no longer earn their tokens, and put what remains first so it is billed at the cached rate rather than the standard one. Neither requires changing model.
Worked from today’s rates
A 1,200-token system prompt sent on 100,000 requests a month is 120M input tokens of pure instruction. On GLM-5.2 that is $98.40 at the standard $0.82/M, or $19.20 once the prefix is stable enough to be cached at $0.16/M.
Related terms
- Prompt cachingPrompt caching is the reuse of an already-processed prompt prefix, so that repeated leading tokens — a system prompt, a tool schema, a fixed document — are billed at a reduced cached-input rate instead of the full input rate.
- Token budgetA token budget is the ceiling an application deliberately puts on how many tokens one request may consume: the output cap sent as max_completion_tokens, and the input cap set by how much history and retrieved context the client is willing to include.
- Context windowA context window is the maximum number of tokens a model can hold in one request, counting the system prompt, the conversation history, any retrieved documents and the reply being generated together.
Go deeper
- Cached input pricing, and why it decides what an agent costs
- Coding agents: where the repeated prefix dominates
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.