Inside the request path

One hop, and why it exists at all

There are three parties in a request here and only one of them is this API. Your client speaks to a route handler on router.xark.io; the route handler forwards to the inference gateway that meters and bills; the gateway forwards to whichever upstream actually serves the weights. The middle one is the only piece described on this page.

The reason it exists is not technical. Publishing the gateway's own hostname would make its hosting a permanent, customer-visible dependency, and moving it later would break every integration in the field. One hop buys the freedom to move the gateway without telling anyone -- which is a product decision paid for in latency, so it is worth stating exactly what that hop does.

It is also the reason the base URL you paste is ours rather than somebody else's. Everything below is what that one hop costs and what it changes.

Eight headers removed on the way out

Every inbound header is copied to the upstream request except eight, and each exclusion has a specific failure it prevents rather than being general hygiene. Note what is not on the list: authorization is forwarded untouched, which is what makes the gateway rather than this API the thing that authenticates you.

The one header we set rather than forward

x-forwarded-for is stripped and then written again from the connection the request actually arrived on. That looks like the same thing and is not, and the difference is the whole point: the gateway rate-limits on that address, so passing through whatever a caller sent would let anyone forge the origin the limiter counts against.

The practical reading for anyone running behind their own proxy is that the address the gateway sees is the one it observes, not one you can influence. If your fleet shares an egress address, it shares the gateway's per-address limit as well.

The body is never read, with one exception

The request body is handed to the upstream fetch as a stream rather than parsed and re-serialised. That is why a streamed completion arrives as it is produced rather than landing all at once at the end, and it is also why nothing in this hop can inspect, rewrite or validate what you sent. A relay that parses is a relay that can silently change your request.

The exception is POST /v1/video/generations, and it is a deliberate one. The gateway bills video through a flat price per call that does not scale with any request parameter, and the price on file is set for exactly five seconds. So that route reads the body, confirms the requested duration matches the duration the price corresponds to, refuses anything else with code fixed_duration_required, and then re-wraps the raw text into a fresh request -- because a body already read as a stream cannot be supplied a second time.

Everything else -- tools, response_format, stream_options, your messages -- reaches the gateway byte for byte as you sent it.

# The only request this hop looks inside, and only at one field.
curl https://router.xark.io/api/v1/video/generations \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"alibaba/wan-2.2-t2v-a14b","prompt":"rain on a window","duration":8}'

# 400  error.code = "fixed_duration_required"

Five headers removed on the way back, two added on a stream

connection, keep-alive and transfer-encoding are hop-by-hop and meaningless to copy onto a different connection. content-encoding and content-length go for a sharper reason: both describe a body that has already been re-framed by the time it reaches you, and a content-length that no longer matches the bytes is a hang rather than an error.

On any response whose content-type is text/event-stream, two headers are then set explicitly: cache-control: no-cache, no-transform and x-accel-buffering: no. A single buffering intermediary anywhere in the path destroys streaming completely, and the failure is invisible in a status check -- the request succeeds, the bytes are correct, and a human watches a cursor sit still for eight seconds and then dump a paragraph.

Errors are buffered. Success never is

A response with a status of 400 or above is read into memory and normalised. A response below 400 is passed through as a stream and never touched. The asymmetry is deliberate and cheap: an error body is a few hundred bytes, so buffering one costs nothing, while buffering a successful one would cost the only property this hop is trying not to break.

That condition used to also require a JSON content-type, and the extra clause was a real defect. The gateway answers a stream: true request that runs out of credit with a 403 whose content-type is still text/event-stream -- so the single most important error a customer can hit skipped normalisation on the single most common way to call the API, and arrived as a raw Chinese-language 403 instead of the documented 429. The content-type check bought nothing, because the normaliser already hands back anything that is not the JSON it expects, untouched.

Exactly two error codes are rewritten

The rewrite table has two entries. Both map an upstream out-of-credit condition onto the 429 with code insufficient_credits that this API publishes, and both restate the message in English. Anything not in that table passes through with its status and its body intact, because inventing a translation for an error nobody has seen before is worse than showing the original.

Two entries is a small number and it is the honest one. It means an unfamiliar upstream failure reaches you looking like an unfamiliar upstream failure rather than like something we understood and chose to describe. Parse defensively; the error envelope is guaranteed on the paths below and on everything this API generates itself, not on every byte the upstream can emit.

Two ways a request ends early, and both reach the upstream

The outbound fetch carries a combined abort signal: your own connection's signal, and a 300-second timeout. The timeout is generous because a long video render or a slow completion is normal here and cutting one off mid-stream is worse than waiting.

Your signal is the one that used to be missing. Without it, a caller who closed a streamed completion left the upstream generating to the end, and those tokens were metered against a balance for output nobody would ever read. Cancellation has to propagate or it is not cancellation, it is a disconnected socket. One caveat stated plainly: at the time of writing no streamable model is callable on this account's credit type, so propagation has been verified by inspection rather than by watching a stream stop.

Cancellation is still not a refund. It lands late, and everything generated before it lands is billed.

What the relay does not do, which is most of it

There is no key lookup in this hop, no balance check and no counter increment. Your authorization header is forwarded and the gateway authenticates it, decrements the balance and applies its own per-address limiter. That is why an invalid key's 401 comes from upstream, and it is precisely why the out-of-credit error needed rewriting at all -- the check that produces it does not happen here.

The published per-endpoint limits -- 60 a minute on chat, responses and completions, 300 on embeddings, 10 on video generations -- are the contract to build against, and they are enforced by the handler that answers when no gateway is configured. On the relayed path the limiter you actually meet is the gateway's, keyed on the connection's address rather than on your key. As with the in-process edge limiter, you may observe more succeeding than the published numbers permit; do not build on the surplus, because a client designed around it breaks under exactly the load that made it worth building.

One more absence worth naming: when no gateway is configured at all, five of the eight relayed endpoints fall back to a documented local mock and three refuse outright. Images and speech have no fallback because a fabricated image or audio file is not a degraded answer, it is a wrong one.

What we will not claim about latency

We do not publish a measured overhead figure for this hop, and we are not going to invent one. What can be said precisely is what it does: one additional TLS connection and one set of header operations, no body parse on any endpoint but video, and no buffering on a successful response.

A provider who publishes their number is easier to evaluate than one who does not, and OpenRouter documents roughly 25 to 40 milliseconds of routing overhead for their own router. That is a real disclosure and we have no equivalent. If time to first token is the number your product lives on, Groq's custom silicon is an architectural difference rather than a tuning claim, and neither we nor a comparison table can talk our way around it.

The honest way to find out what this hop costs you is to measure it against your own workload, which is one line more than measuring anything else.

A short checklist for anyone debugging through the hop

Branch on error.code, never on the status alone. Two upstream conditions become 429 here and a third meaning of 429 is a rate limit; the code is the only thing that separates them.

Treat a stream that ends without its terminator as a failure, not a short answer. [DONE] on chat and completions, response.completed or response.incomplete on responses.

Do not send an x-forwarded-for you expect to survive. It is replaced, on purpose.

If you get a body that is not the published error envelope, you have found an upstream failure outside the two-entry rewrite table. That is worth reporting, because the table is meant to grow only with errors we have actually seen.

Read usage.cost_usd on the response rather than reconstructing it. The relay does not compute it; the gateway does, from the same rates the pricing page prints.