Streaming (SSE)
Streaming is the delivery of a model's answer as a sequence of server-sent events, each carrying the next fragment of text as it is generated, instead of as one JSON response returned after the whole answer is finished.
Setting stream to true changes the transport and nothing else. The response arrives as text/event-stream, a series of chunk objects terminated by a sentinel, and the token counts and the charge are identical to the same request made without it. What changes is when the first character appears, which on a long answer is the difference between an interface that feels alive and one that looks broken.
A streamed response carries no usage object by default. Sending stream_options with include_usage set to true appends a final chunk that has one, and a client that omits it has no per-request token count at all — only a monthly total to reconcile against a guess.
Disconnects are the case worth designing for. Cancellation has to propagate: a caller who closes the tab or times out their HTTP client must actually stop the generation upstream, or the remaining tokens are produced and billed for output nobody will read. On this API the client's own abort signal is forwarded, so generation stops — but everything already produced up to that point is billed, because it was already produced.
Anything that buffers a response defeats it. A proxy, a serverless wrapper or a framework helper that collects the body before returning it turns a stream back into a single slow response while keeping all of the extra complexity.
What it costs you
Streaming does not change what an answer costs; it changes how much of an answer you end up paying for, and how well you can see it. Both directions are real money. An abandoned stream is billed for every token generated before the abort, which on an interface where users skim the first two lines and move on is a recurring charge for text nobody read. And a client that never sets include_usage cannot attribute cost to a request, a customer or a feature, which is how cost control stops at the invoice.
Worked from today’s rates
A 2,000-token answer from Kimi K3 would cost $0.0180 in output. Abandoned after 500 tokens, it costs $0.00450 — the generated quarter is billed, the rest is not, and only if the cancellation reached the upstream.
Related terms
- Time to first token (TTFT)Time to first token, or TTFT, is the delay between sending a request and receiving the first token of the response, and it is the part of latency a person waiting on a streamed answer actually perceives.
- Token budgetA token budget is the ceiling an application deliberately puts on how many tokens one request may consume: the output cap sent as max_completion_tokens, and the input cap set by how much history and retrieved context the client is willing to include.
- Input vs output tokensInput tokens are the text you send a model and output tokens are the text it writes back; they are billed at separate rates, and the output rate is almost always several times the input rate.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.