Tool calling (function calling)
Tool calling is the mechanism by which a model, given a set of function schemas alongside the prompt, replies with a structured request to invoke one of them instead of with prose — your own code runs the function and sends the result back as a further message for the model to continue from.
The model never executes anything. It emits a function name and a JSON argument object; everything after that is your code, and the result returns as a message with the tool role. One tool call is therefore a minimum of two round trips, each carrying the full conversation so far.
Tool schemas are tokens. Every definition you pass is serialised into the request on every call, whether the model uses it or not, so a catalogue of thirty tools is paid for on the requests that call none of them. tool_choice narrows the behaviour — none, auto, required, or a named tool — but it does not narrow the bill; only sending fewer schemas does that.
The loop compounds. Each step resends the system prompt, the schemas, the original question and every prior call and result, so an eight-step task sends the fixed part eight times behind a tail that grows at every step. That shape, not the model's rate, is what makes agents expensive.
Arguments still need validating. A schema tells the model what shape to emit and is not a guarantee that it did, and a tool invoked with plausible but wrong arguments fails in a way that costs another turn to discover.
What it costs you
An agent's bill is almost entirely input, and almost all of that input is the same schemas and the same transcript sent again. Two consequences follow that nothing on a rate card will tell you: adding one more tool raises the price of every request in the application, including the ones that never touch it, and keeping the schema block fixed and first is what lets the largest repeated part of each request be billed at the cached rate. Pruning the tool list per request is usually a bigger saving than changing model.
Worked from today’s rates
A 2,500-token tool schema carried through a 12-step loop is sent twelve times per task. Across 10,000 tasks a month that is 300M input tokens before a single user word: $165.00 on Kimi K2.6 at $0.55/M, or $33.00 if the schema sits in a cached prefix.
Related terms
- Structured output (JSON mode)Structured output is a request setting — response_format on an OpenAI-compatible API — that constrains a model to answer with valid JSON, either free-form or conforming to a schema you supply, rather than with prose your code then has to parse out.
- System promptA system prompt is the instruction block placed first in a request, under the system or developer role, that sets a model's persona, rules and output format for that request — and because the API keeps no state between calls, it is resent and re-billed on every single one.
- Prompt cachingPrompt caching is the reuse of an already-processed prompt prefix, so that repeated leading tokens — a system prompt, a tool schema, a fixed document — are billed at a reduced cached-input rate instead of the full input rate.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.