Token budget
A token budget is the ceiling an application deliberately puts on how many tokens one request may consume: the output cap sent as max_completion_tokens, and the input cap set by how much history and retrieved context the client is willing to include.
The two halves are enforced in different places. The output half is a parameter — max_completion_tokens, or the deprecated max_tokens spelling, which this API still honours — and the server enforces it. The input half is enforced by nobody but you: nothing stops a client sending a context window's worth of history on every turn.
An output cap truncates, it does not summarise. A response that hits the ceiling stops mid-sentence and comes back with finish_reason of length, so a budget is only useful alongside code that notices that and does something sensible. Setting a cap and ignoring the flag converts a cost problem into a correctness one.
The input half is where the money usually is. Retrieved chunks, tool results and conversation history all arrive as input on every request, and a budget expressed as a number of chunks is not a budget at all unless the chunks themselves are bounded.
Budget the task, not the request. A user asks one question; the application may answer it with fourteen model calls. A per-request cap that looks modest can sit under a per-task cost nobody has ever measured.
What it costs you
An application with no token budget has no upper bound on the cost of a single request — only the model's context window, which is a capacity limit and not a spending limit. That is the difference between a cost you can forecast and a cost you can only observe. A cap costs nothing to set, turns an unbounded worst case into a known one, and is the only cost control that keeps working when a prompt, a model or a user behaves in a way nobody anticipated.
Worked from today’s rates
An uncapped reply that runs to 4,000 tokens costs $0.0102 in output on GLM-5.2; the same request capped at 500 costs $0.00127. Across 200,000 requests a month that is $2,040.00 against $255.00, from one request parameter.
Related terms
- Context windowA context window is the maximum number of tokens a model can hold in one request, counting the system prompt, the conversation history, any retrieved documents and the reply being generated together.
- Input vs output tokensInput tokens are the text you send a model and output tokens are the text it writes back; they are billed at separate rates, and the output rate is almost always several times the input rate.
- Prepaid balanceA prepaid balance is credit bought before it is spent and drawn down by each request, so the amount already loaded is a hard ceiling on what an API can cost you — the opposite of postpaid billing, where usage accrues freely and is invoiced after the fact.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.