Use case
The best API for RAG and retrieval
RAG has two bills with different shapes: a one-off embedding cost to index the corpus, then a recurring generation cost per query. Embedding a million-document corpus with Qwen3-Embedding-8B costs about $0.03 per million tokens of text — usually a rounding error next to the generation that follows it.
Indexing is a one-time cost proportional to corpus size. Querying is a recurring cost proportional to traffic, and each query carries the retrieved chunks as input — so retrieval quality shows up directly in the bill. Returning ten chunks where three would do triples the input tokens on every request, forever.
This is the workload where people most often optimise the wrong number. Embedding rates are already so low that halving them saves little; the leverage is in retrieving fewer, better chunks, which cuts the generation input that dominates the total.
What actually matters here
- Embedding dimensions
- Wider vectors cost more to store and search, and past a point add little recall. Matching width to corpus size is a real cost decision.
- Generation input rate
- Retrieved context is input on every single query. At scale this is the whole bill, not the embeddings.
- Cached input
- A fixed system prompt and instruction block repeat on every query and can be served from cache.
What it costs, at 100,000 queries/month, 4K context each
Worked from this catalogue’s published rates at 400M input and 30M output tokens a month. Your figure will differ; the arithmetic will not.
| Model | Input | Output | Per month | At official rates |
|---|---|---|---|---|
| Qwen3 Embedding 8B40% | $0.030/M | — | $12.00 | $20.00 |
| GLM-5.242% | $0.82/M | $2.55/M | $404.50 | $692.00 |
| DeepSeek V4 Flash36% | $0.090/M | $0.18/M | $41.40 | $64.40 |
Excludes cached-input savings, which on a repeated-prefix workload typically reduce the input column substantially. Cached rates are published per model on the pricing page.
Questions
- Should I re-embed when a better model appears?
- Only if retrieval quality is measurably failing. Re-embedding means re-indexing the whole corpus and invalidating every stored vector, and embedding rates are low enough that the switching cost is usually higher than the saving.