Temperature
Temperature is a sampling parameter that controls how much randomness a model uses when choosing each next token: at 0 it takes the most likely token every time, and higher values raise the chance of less likely tokens being picked.
The model always produces a probability distribution over its whole vocabulary. Temperature reshapes that distribution before a token is drawn from it — pushing it towards its peak at low values and flattening it at high ones. It does not make the model more or less capable, only more or less willing to leave the most probable path.
OpenAI-compatible APIs accept a range from 0 to 2 with a default of 1. Values above roughly 1.2 degrade coherence quickly on most models, which is a much narrower useful band than the range suggests.
Temperature 0 is near-deterministic rather than deterministic. Batching, floating-point non-associativity on GPUs and routing between hardware all mean identical requests can still produce different outputs. Any test suite asserting exact string equality on model output will eventually fail for reasons unrelated to the change under test.
top_p is an alternative control that truncates the distribution to its most probable mass rather than reshaping it. Adjusting both at once makes behaviour hard to reason about, so the usual advice is to move one and leave the other at its default.
What it costs you
Temperature has no rate of its own, but it sets the retry rate, and a retry is billed in full. A high temperature on a structured-output or tool-calling task produces malformed responses that are paid for and then discarded, so the true cost of a usable answer is the price of every attempt it took. On any task with a machine-checkable output format, the correct temperature is the lowest one that works, and that is a cost decision as much as a quality one.
Worked from today’s rates
A 2,000-token reply from DeepSeek V4 Pro costs $0.00110 in output. If it comes back malformed and has to be retried, the usable answer cost $0.00220 — the failed attempt is billed exactly like the good one.
Related terms
- TokenA token is the unit of text a language model reads and writes — roughly four characters of English, or about three quarters of a word — and it is also the unit that every language model API bills by.
- Input vs output tokensInput tokens are the text you send a model and output tokens are the text it writes back; they are billed at separate rates, and the output rate is almost always several times the input rate.
- OpenAI-compatible APIAn OpenAI-compatible API is an endpoint that accepts the same request and response shapes as OpenAI's own API, so an existing client can be pointed at it by changing only the base URL, the API key and the model name.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.