Inference provider
An inference provider is a company that runs open-weight model weights on its own or rented GPUs and sells access to them over an API, as distinct from the lab that trained and published the model.
Two different roles are often described with the same word. A lab trains a model and publishes both the weights and an official rate for its own hosted endpoint. A provider serves those published weights and sets its own rate. Only the first can change what the model is; only the second competes on price.
Because the weights are identical, providers differ on the things the weights do not determine: precision served, achieved throughput, time to first token, context limits actually enforced, reliability, and how failures are reported. Those differences are invisible in a price table and are most of what distinguishes one endpoint from another.
A gateway is a further layer again — one API surface in front of several providers, routing between them. That adds redundancy and a single billing relationship, and it introduces the question of which underlying provider actually served a given request, which a gateway should be able to answer.
This service pins a request to the model you asked for. Sending an exact model id gets that model; substitution only happens if you explicitly request fallback, which matters when prompts have been tuned against one model's behaviour.
What it costs you
The lab's official rate is a ceiling rather than a market price, because the lab is pricing research recovery as well as compute while a provider is pricing compute alone. That gap is the whole opportunity, and it is also why the honest comparison is against the model publisher's official rate rather than against another reseller's promotional number — the official rate is published, stable, and verifiable, which a competitor's current price is not.
Worked from today’s rates
Kimi K2.6 was trained and published by Moonshot AI, whose official rate is $0.95/M for input. The identical weights are served here at $0.55/M, with Moonshot AI's official figure printed alongside ours on every page that quotes it.
Related terms
- Open-weight modelAn open-weight model is one whose trained parameters are published under a licence permitting others to download, run and serve them, which is what makes it possible for anyone other than the original lab to offer that model as an API.
- OpenAI-compatible APIAn OpenAI-compatible API is an endpoint that accepts the same request and response shapes as OpenAI's own API, so an existing client can be pointed at it by changing only the base URL, the API key and the model name.
- QuantizationQuantization is the compression of a model's weights from higher-precision numbers to lower-precision ones — typically 16-bit down to 8-bit or 4-bit — which reduces the memory and compute needed to serve it, at some cost to output quality.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.