Use case
The best API for chatbots and support
Chat costs grow faster than traffic, because each turn resends the whole conversation so far. A 20-turn conversation sends roughly ten times the tokens of a 2-turn one for the same number of replies — which is why per-conversation cost, not per-message cost, is the number worth modelling.
Every turn resends the history. Costs scale with the square of conversation length, not linearly with messages, and a support bot that averages 12 turns costs far more than one averaging 4 even at identical volume.
The two effective levers are cached input on the fixed system prompt and a summarisation or truncation policy on older turns. Choosing a marginally cheaper model is usually a smaller win than capping how much history each turn carries.
What actually matters here
- Cached-input rate
- The system prompt and persona are identical on every turn of every conversation — the clearest cache win of any workload.
- Output rate
- Unlike agents, chat produces substantial output. Here the output rate genuinely matters.
- Latency
- A human is waiting. Time to first token is felt directly, in a way it is not for a background job.
What it costs, at 50,000 conversations/month, ~8 turns each
Worked from this catalogue’s published rates at 600M input and 40M output tokens a month. Your figure will differ; the arithmetic will not.
| Model | Input | Output | Per month | At official rates |
|---|---|---|---|---|
| DeepSeek V4 Flash36% | $0.090/M | $0.18/M | $61.20 | $95.20 |
| GLM-5.242% | $0.82/M | $2.55/M | $594.00 | $1,016.00 |
| Kimi K2.642% | $0.55/M | $2.30/M | $422.00 | $730.00 |
Excludes cached-input savings, which on a repeated-prefix workload typically reduce the input column substantially. Cached rates are published per model on the pricing page.
Questions
- Does truncating history hurt quality?
- Less than most teams expect for support workloads, where the useful context is usually the last few turns plus a summary. It is worth measuring on your own transcripts, because the cost difference is large and the quality difference often is not.