Choosing for a workload
Which model suits which shape of work, and why the answer changes with the input-to-output ratio rather than with the benchmark.
24 questions
coding agents
In context →- Does cached input really change the total that much?
- On an agent workload it is usually the largest single factor. If 80% of your input is a repeated prefix billed at roughly a fifth of the standard rate, the effective input cost falls by about two thirds — a bigger swing than the difference between most models' headline rates.
- Can I pin an agent to one model?
- Yes. Send the exact model id and no substitution happens. Fallback only applies if you ask for it, which matters for an agent whose prompts were tuned against one model's behaviour.
RAG and retrieval
In context →- Should I re-embed when a better model appears?
- Only if retrieval quality is measurably failing. Re-embedding means re-indexing the whole corpus and invalidating every stored vector, and embedding rates are low enough that the switching cost is usually higher than the saving.
chatbots and support
In context →- Does truncating history hurt quality?
- Less than most teams expect for support workloads, where the useful context is usually the last few turns plus a summary. It is worth measuring on your own transcripts, because the cost difference is large and the quality difference often is not.
document summarisation
In context →- Is a bigger context window worth paying for?
- Only when your documents actually exceed the smaller window. Below that line you are paying a much higher input rate for capacity you never use; above it you are paying a lower rate to read the same document twice. Measure your document length distribution before choosing — the answer is usually that a small tail of long documents should be routed to the long-context model and everything else should not.
- Does cached input help here?
- Less than on agent or chat workloads. Your instruction block repeats and can be cached, but it is small next to the document, and the document itself is different every time — so the cacheable share of input is the part that was already cheap.
- Should I summarise in one pass or map-reduce?
- One pass whenever the document fits, because map-reduce bills the source text once per chunk and then bills the intermediate summaries again in the reduce step. Map-reduce is a workaround for a context limit, not an optimisation.
- Can I run this asynchronously?
- Yes, and you should. Nothing about summarisation needs a low time to first token, so you can queue the work, run it at whatever concurrency your rate limit allows, and pick the model on price alone rather than on latency.
structured data extraction
In context →- How do I know whether a cheaper model is actually cheaper?
- Run both over a few hundred labelled documents and record the first-pass validation rate, not just the price. Divide the expensive model's cost per run by the cheap model's, and if the cheap model's failure rate is worse than that ratio implies, it is losing money on retries. This is a half-day experiment that decides a recurring bill.
- Should I validate before or after storing?
- Before, always, and with the same schema the model was constrained to. Validating downstream means a bad record is discovered by whatever consumes it, which is far more expensive to unpick than a retry at extraction time.
- Does a two-stage pipeline help?
- Often. A cheap model does the first pass and a validator routes only the failures to a stronger one, so you pay the higher rate on the minority of documents that need it. It is worth building once your volume makes the difference between the two models material.
- Do cached-input rates apply to extraction?
- To the schema and instruction prefix, yes — that block is identical on every call and is exactly what the cache is for. The document itself is unique per request and is billed at the standard input rate.
semantic search and embeddings
In context →- Why is only one model listed here?
- Because embeddings are a separate model class. A chat model cannot return a vector, so comparing this workload against generation models is comparing two different products. The models on the RAG page are the ones that read the retrieved chunks afterwards, which is a different page for a reason.
- Why is there no output column?
- An embedding model has no output side to bill. It returns a fixed-width vector, which is the same size regardless of what you sent, so the entire cost is the text you embedded. That is why the monthly figure here moves with corpus size and hardly at all with query traffic.
- Should I use 4096 dimensions or fewer?
- Start narrower than the native width and measure recall on your own queries. Qwen3-Embedding-8B is trained for Matryoshka truncation, so a shorter prefix of the vector remains usable — and since storage and search scale with width, the narrower vector is cheaper every day the index exists, while the embedding call costs the same either way.
- Do I have to re-embed the whole corpus each month?
- No — the volume above is deliberately pessimistic. Most corpora only need new and changed documents embedded, so the recurring bill is far smaller than the initial index. The figure is priced as a full re-index because that is the worst case, and a worst case you can afford needs no further modelling.
translation and localisation
In context →- Why does the strongest general model cost so much more here?
- Because it charges a much larger multiple for output than for input, and this workload produces almost as many output tokens as input tokens. On an agent workload that multiple is applied to a rounding error; on translation it is applied to half the volume. Compare the two rate columns in the table above and the gap explains itself.
- Should I translate string by string or document by document?
- Document by document wherever the format allows, because a string in isolation loses the context that decides gender, register and pronoun choice. It also amortises the instruction prefix across more content rather than resending it per string.
- Does a glossary help the bill or just the quality?
- Both, indirectly. A glossary in the system prompt is cacheable and cheap, and it removes the most common reason to rerun a translation — inconsistent terminology — which is a rerun you would otherwise pay for at full rate.
- Can I mix models across locales?
- Yes, and it is often the right call. Pin the pairs where quality is the constraint to a stronger model and run the rest on the cheapest one that passes review, sending an explicit model id per call so nothing is substituted underneath you.
content generation
In context →- Why does the input column barely move the total here?
- Because the brief and style guide are a fraction of what comes back. Look at the table: the input side of this workload is a small share of each row, which is why a model with a very cheap input rate and an ordinary output rate saves you almost nothing on generation.
- Is the cheapest output rate always the right answer?
- Only if the drafts are usable. Multiply the listed rate by the number of attempts a piece actually takes and compare that, because a model needing a second pass has doubled its own price while a more expensive model that lands first time has not.
- Does a longer, more detailed brief cost more?
- Technically yes, and it is almost always worth it. Input is the cheap side of this workload, so spending more tokens on the brief to avoid one regeneration is trading the cheap column for the expensive one.
- Should I generate an outline first?
- For anything long, yes. An outline pass is short output and gives you a cheap place to reject a bad direction before paying for the full piece. Rejecting at the outline stage costs a fraction of rejecting a finished draft.
Check it against your own numbers
$5 in free credits, no card required.
Every rate quoted above is the rate the API bills.