Idempotency key
An idempotency key is a client-generated identifier attached to a request so a server can recognise a retry of that exact request and return the original result instead of performing — and charging for — the work a second time.
It exists for the ambiguous failure. A timeout, a reset connection or a dead load balancer leaves the client unable to tell whether the work happened, and without a key both available responses are wrong: retry and risk doing it twice, or give up and risk having lost a result you already paid for.
Money paths generally accept one and generation paths generally do not. A credit top-up here carries an idempotency key precisely so a redelivered payment webhook cannot credit an account twice. The OpenAI-compatible inference endpoints accept no such header, here or in the contract they implement, so a retried completion is a second generation and a second charge.
The reason is a genuine mismatch rather than an oversight. Honouring a key means storing the original response for the retry window, and a completion is both expensive to store and not deterministic — so the thing you would be handed on a retry is one particular past answer, kept at a cost, to a question the model would now answer differently.
What replaces it is discipline at your own layer: record a request id before sending, bound retries rather than looping on them, and treat a timeout as a question to resolve against your own records instead of an instruction to send again.
What it costs you
Because no key is available on an inference call, every retry is a full second charge for work that may already have succeeded. The dangerous version is silent: a client with an aggressive timeout and automatic retries can pay twice for a large share of its traffic while its own logs show no errors at all, because both attempts succeeded and only one of them arrived in time to be recorded. Long-context requests make it worse, since the duplicated part is the expensive prefix rather than the short answer.
Worked from today’s rates
A request carrying 40,000 tokens of context and writing 1,200 costs $0.0248 on Kimi K2.6. A client timeout firing just before the answer lands, followed by an automatic retry, makes that same answer cost $0.0495 — and both charges are legitimate, because both requests were served.
Related terms
- Rate limitA rate limit is a cap on how many requests or how many tokens an API will accept from one key in a given window, enforced by rejecting further calls with HTTP 429 until the window resets.
- Prepaid balanceA prepaid balance is credit bought before it is spent and drawn down by each request, so the amount already loaded is a hard ceiling on what an API can cost you — the opposite of postpaid billing, where usage accrues freely and is invoiced after the fact.
- Streaming (SSE)Streaming is the delivery of a model's answer as a sequence of server-sent events, each carrying the next fragment of text as it is generated, instead of as one JSON response returned after the whole answer is finished.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.