Engineering blog
What running an inference gateway actually costs, what breaks, and the numbers behind both. Every post carries at least one figure that comes from operating this service rather than from a vendor’s marketing page.
- open-weightsprocurementlicensing
Open-weight or frontier: how to decide
Four questions that settle it, seven cases where a closed frontier model is the right answer and we are not, and the one property no price cut can replicate.
- evaluationcost-controlpricing
What it costs to run an evaluation suite
One pass over a 1,000-item benchmark is $1.68 on GLM-5.2. The suite you actually run is forty-five passes, and the judge nearly doubles it.
- embeddingspricingcost-control
Embeddings: dimensions and corpus cost
Matryoshka truncation is a free quarter of your storage bill; changing model is not. Worked corpus costs, and the storage table nobody prints.
- rate-limitsapi-contractagents
Rate limits, concurrency and backoff
Five independent per-endpoint buckets, a fixed window that allows 120 requests in two seconds, and the field that says whether retrying will help.
- securityapi-contractcredits
What a leaked API key can actually do
Keys are shown once because the plaintext is fetched once. What rotation really has to do, and why a leaked key's blast radius is your balance plus auto-recharge.
- streamingapi-contractsdk
Streaming: who pays when nobody listens
The SSE frames in order, the one frame that carries usage, and what actually happens to the bill when a client hangs up mid-completion.
- billingpricingcredits
Prepaid credits vs postpaid invoicing
A prepaid balance is a hard ceiling a runaway agent cannot exceed. What that protects you from, what it costs at procurement, and the fees across the market.
- pricingvideoopen-weights
What open-source video generation costs
Per-second rates for six open-weight video models, the three incompatible billing units this market uses, and why our calls bill a fixed five seconds.
- pricingcachingagents
Cached input pricing and what an agent costs
Cached input is a fifth of the uncached rate here. On a 50-turn agent loop with a 40,000-token system prompt that is $1.64 against $0.35.
- licensingopen-weightscompliance
Open-weight licences and what you can ship
Can you use Llama commercially? Yes, with conditions Apache 2.0 does not impose. Four licence families across 24 models, and the ones we refuse to resell.
- migrationopenai-compatibleapi-contract
What OpenAI compatibility does not change
Changing base_url is the whole migration. Three things still behave differently, and each one fails on the unhappy path in production rather than during the swap.
- pricingtokensbilling
How LLM API pricing is calculated
Input, cached input and output are three separate rates, not one. A worked example on a real request, and the integer money grid underneath the bill.