Embeddings: dimensions and corpus cost

One embedding model is callable today

Two are catalogued and one is served. Qwen3 Embedding 8B is callable now; BGE-M3 is in the catalogue at a published rate but has no configured upstream, and the models endpoint marks it as such rather than letting a copy-runnable snippet fail at the moment of use. If you want 1,024-dimension MIT-licensed embeddings today, we are not the provider -- DeepInfra publishes BGE-M3 at $0.015 per million tokens and actually serves it.

Embedding models have one rate and no output side. There is no completion price, no cached-input price and no output multiple to reason about; the usage block carries prompt_tokens and total_tokens and nothing else. That makes them the easiest thing on any rate card to forecast and the easiest to underestimate, because the cost that bites is storage rather than inference.

What Matryoshka truncation actually buys you

Qwen3 Embedding 8B is trained for Matryoshka truncation, and the endpoint implements it the way the property requires rather than the way that is easier. Ask for fewer dimensions and the vector is drawn at the model's native 4,096 width, then sliced to the width you asked for, then renormalised. It is not a differently-seeded narrow vector.

That preserves the one property a caller is likely to be relying on without checking: the 1,024-wide vector is the prefix of the 4,096-wide vector for the same input. An index built at one width stays comparable with a query at that width, and you can decide the width after you have already embedded, provided you kept the wide originals.

Asking for more than the native width is a 400 rather than a pad. The message names the ceiling, the model and the value you sent, and says explicitly that smaller values are supported. Silently padding to a requested width would produce vectors that look fine, index fine and retrieve badly.

wide   = client.embeddings.create(model="qwen/qwen3-embedding-8b",
                                 input="hello", dimensions=4096)
narrow = client.embeddings.create(model="qwen/qwen3-embedding-8b",
                                  input="hello", dimensions=1024)

# The narrow vector is the renormalised prefix of the wide one.
# Direction is preserved; only the norm is restored after the slice.
# dimensions=8192 -> 400, param "dimensions", naming 4096 as the ceiling.

Normalisation, and why cosine and dot product agree

Every vector returned is L2-normalised, as real embedding models return them. That has one direct consequence for a vector database: cosine similarity and a plain dot product give the same ranking, so you can pick whichever your index implements fastest without changing your results.

It also means you should not normalise again on the way in. Some vector stores normalise on insert by default. Doing it twice is harmless mathematically and wasteful in practice, and more importantly it hides the case where an unnormalised vector from some other source has been mixed into the same collection -- which is the actual bug that produces retrieval that is subtly, unexplainably worse.

Values arrive rounded to six decimal places. That is below float32's own precision, so nothing is lost, and it keeps a 2,048-input batch of 4,096-wide vectors from being needlessly enormous on the wire.

Storage is the number nobody prices

A float32 vector is four bytes per dimension. That single fact decides most of the real cost of a retrieval system, and it is why the dimensions parameter is a budget decision rather than a quality knob.

At the native 4,096 dimensions each vector is 16 KiB before any index overhead. Ten million chunks -- a mid-sized document corpus, not an ambitious one -- is 163.84 GB of raw vectors. The same corpus at 1,024 dimensions is 40.96 GB, and because of the prefix property that reduction costs nothing but a re-slice of vectors you already hold.

What it costs to embed a corpus

The inference side is straightforward arithmetic against one rate. Our published figure is $0.03 per million tokens against Alibaba's official $0.05, a 40% reduction, and the competitor columns below are each vendor's published rate for the same model.

Note where the honest comparison lands. On this model our rate is the lowest of the four, but the gap to DeepInfra is $0.005 per million tokens, which on a 100-million-token corpus is $0.50 in total. Nobody should move providers for that. The reason to choose on price here is if you are embedding at a scale where the storage table above already frightened you -- and at that scale you should be choosing on dimensions, not on rate.

Batching rules that are contracts, not suggestions

Up to 2,048 inputs per request, which is the upstream contract's own ceiling. Exceeding it returns a 400 that reports how many you sent and tells you to split the batch, rather than truncating silently to the limit.

Results come back in the order the inputs were sent. The index field is echoed as well, but order is the guarantee, because order is what every batching client actually relies on when it zips results back onto its own rows. Sorting by index is defensive and free; assuming the array is a set is not.

Token-array inputs are accepted here, unlike on the completions endpoint where they are refused. They cannot be detokenised, so they are keyed on their own comma-joined form and counted as one token per id -- which is exactly right rather than merely close, and is the one place in this API where the token count is not an estimate.

With encoding_format: base64 the payload is little-endian float32, which is what the OpenAI clients decode with numpy.frombuffer at dtype float32. Endianness is not negotiable: get it wrong and the vectors decode to plausible-looking garbage rather than to an error, which is the worst failure mode available in a retrieval system.

resp = client.embeddings.create(
    model="qwen/qwen3-embedding-8b",
    input=chunks[:2048],          # 2,048 is the hard ceiling per request
    dimensions=1024,              # a quarter of the storage, same prefix
)

# Order is the contract. Index is a belt-and-braces check, not the mechanism.
vectors = [d.embedding for d in resp.data]
assert [d.index for d in resp.data] == list(range(len(chunks[:2048])))
print(resp.usage.prompt_tokens, resp.usage.cost_usd)   # no completion side

Re-embedding: cheap in tokens, expensive in everything else

Changing dimensions on the same model is free if you kept the wide vectors, because the narrow vector is a prefix. Changing model is not free in any sense, and the token cost is the smallest part of it.

The tokens are easy: a 100-million-token corpus is another $3.00. The parts that are not on any rate card are that vectors from two different models are not comparable at all, so there is no incremental migration -- you cannot search a half-migrated index and get sensible results. You need the storage for both indexes simultaneously, which on 10 million chunks at 4,096 dimensions means 327.68 GB rather than 163.84 GB during the cutover. And every downstream threshold you tuned -- similarity cut-offs, reranking scores, top-k -- was tuned against the old model's distribution and has to be retuned.

The practical rule that follows: choose the model once and choose the width late. The width is reversible and the model is not.

Wall clock, and the limit you will not hit

Embeddings are limited to 300 requests per minute per key, on a counter independent of the chat endpoints. Combined with the 2,048-input ceiling that is 614,400 texts a minute, which for almost any corpus means the rate limit is not your constraint.

The 100-million-token corpus above, chunked at 512 tokens, is 195,313 chunks, which is 96 requests. At 300 requests a minute the whole corpus goes through in about twenty seconds of API time. Whatever takes hours in a corpus ingestion job is the reading, chunking and writing, not us.

The 32,768-token context is the other number to plan against. It is a ceiling on a single chunk, not a target -- retrieval quality generally falls off long before it, and a chunk that large defeats the point of chunking. It matters mainly because it means an oversized chunk is an error you can catch rather than a truncation you will not notice.