--- title: "Call a model from a tool" description: "Use sampling or a gateway token and report outside service cost." canonical_url: "https://docs.oz0.ai/build/parts/tools/model-calls" --- # Call a model from a tool > Use sampling or a gateway token and report outside service cost. For: plugin authors. Use sampling or a gateway token and report outside service cost. ## Tools that need a model A tool that needs a model (to summarize what it fetched, to classify, to embed a text) asks the platform for one and holds no provider key of its own. Declare it under `requires.models`: ```yaml requires: models: sampling: default # the alias your server's sampling requests are answered with gateway: [openai-embeddings] # what sampling cannot express, with a token per call ``` Either one needs the tenant's grant at install (`plugin install --accept-model`). Every model call your tools make this way is a model call of the job that called the tool: its worst case is reserved first, it counts toward the job tree's budget, the tenant's and the quotas, the tenant's data rules apply, and it is on the tool call's step of the job's trace, in the audit trail (`run.model_call` with the plugin) and in the usage export's `plugin` column. A plugin OZ verifies never asks for a provider's key (house rule [OZ-S017](https://docs.oz0.ai/build/plugins/structure/lint-rules#oz-s017)). ### MCP sampling With `sampling: `, the runtime offers your servers MCP's `sampling` capability and answers their `sampling/createMessage` requests while one of their calls runs. The model is the alias you name: `default`, `fast` or `smart`, which every tenant has, or one of your own that the operator maps to a model at install. The request's `modelPreferences` choose only between your alias and `fast` (speed or cost first) or `smart` (intelligence first); a hint that names a provider's model chooses nothing. ```python from mcp import types from mcp.server import MCPServer from mcp.server.mcpserver import Context server = MCPServer("notes") @server.tool() async def summarize(text: str, ctx: Context) -> str | types.InputRequiredResult: """Summarize meeting notes.""" message = types.SamplingMessage(role="user", content=types.TextContent(type="text", text=text)) params = types.CreateMessageRequestParams(messages=[message], max_tokens=500, system_prompt="Summarize.") if (ctx.protocol_version or "") >= "2026-07-28": # The newer protocol: the request rides in the call's result, and the call comes again with the answer. answer = (ctx.input_responses or {}).get("summary") if answer is None: return types.InputRequiredResult(input_requests={"summary": types.CreateMessageRequest(params=params)}) else: answer = await ctx.session.send_request(types.CreateMessageRequest(params=params), types.CreateMessageResult) return answer.content.text ``` What the runtime answers: - Text only: a system prompt and user and assistant messages; at most 100 messages and 400,000 characters, 25 requests per call, `max_tokens` up to 16,384 (more is cut to that), a temperature from 0 to 2. - No tools: a request that carries `tools` or `toolChoice`, or tool calls and their results, is refused: a model that called tools through your tool would go around the tenant's tool policy. - Tool policy: the tenant's rules match your tool's requests as `.sampling` (`notes.sampling`), so a tenant can refuse them or put them behind approval: `oz0 policy ask notes.sampling`. Your own `requires_approval` may name `.sampling` too. A person then sees what the model would read (the system prompt and the messages) and decides while your call waits: give the tool a `timeout` long enough for that, since no answer by the end of the call's time is a refusal. - Labels: what your tool read goes with the request. Say it in a message's `_meta` as `oz0/labels`, as a result declares [labels](https://docs.oz0.ai/build/parts/tools/policy#labels); an open-world function that says nothing counts as having read outside input. The tenant's data rules apply to that, and the call's result carries it back into the job. - Refusals: a refused request is an error with code `-1` (a malformed one `-32600`). When a budget or a quota refused the model call, the error's `data` holds `ai.oz0/refusal`: the refusal's error object, `{"type", "message", "oz0"}`, with the [refusal's numbers](https://docs.oz0.ai/reference/spend-refusals) under `oz0` (so `error.data["ai.oz0/refusal"]["oz0"]["scope"]`). On MCP's newer protocol a refused request ends your call, and the agent hears why, with the same object as the tool error's structured content, `{"error": ...}`. #### Where Sampling works in the tool's own server, a job's sandboxed server and a remote tool's session per call. On MCP's newer protocol (2026-07-28) a request rides in its call, so it is that call's even when the server runs several calls at once (a remote tool, or `sandbox: own`). A server on the older protocol sends its requests on its own, and one sent while its server runs several calls at once, or none, cannot be told apart and gets a refusal. A request is answered only within a tool call the edge bound to its job; an agent that keeps its data on the node (`data: local`) gets none of its tools' requests answered. ### A gateway token for what sampling cannot express Embeddings, or a provider's own request format, are not sampling. With `gateway: []`, each call of your plugin's local tools gets, in its MCP request's `_meta` under `oz0/gateway`:
Key What it is
url , openai_base_url , anthropic_base_url The node's gateway path, for your SDK's base URL
token Your SDK's API key for this call only, while the call runs
formats What the token may call: openai-embeddings , openai-chat , openai-responses , anthropic-messages or google-generate-content , as you listed them
expires_at The latest it can work: the end of the call's time limit. It stops working sooner, when your call returns
```python import json import urllib.request @server.tool() def embed(text: str, ctx: Context) -> list[float]: """Embed a text.""" gate = (ctx.request_context.meta or {})["oz0/gateway"] body = json.dumps({"model": "text-embedding-3-small", "input": text}).encode() req = urllib.request.Request( gate["openai_base_url"] + "/embeddings", data=body, method="POST", headers={"Authorization": "Bearer " + gate["token"], "Content-Type": "application/json"}, ) with urllib.request.urlopen(req) as resp: return json.loads(resp.read())["data"][0]["embedding"] ``` The token is good for its call's job only, while that call runs (and never past `expires_at`), and only for the formats you listed: a use after your call returned, or a call in another format, is refused by the node (`403 oz0_gateway_token`), and a call your plugin does not declare is refused when it reaches the edge. Every call gets a token of its own, even two calls of one job at once, so one call's end never stops another's. Use it within the call and drop it after. A remote tool cannot reach the node's gateway path, so a remote server uses sampling instead. OpenAI's embeddings are metered on their input tokens. ## Report what your own services cost When a function calls a paid service of your own, such as a classifier, a search API or a small hosted model, put what it cost in the result's `_meta` under `oz0/usage`. The cost lands on the job that made the call, next to its model calls, and counts toward the job's [budget](https://docs.oz0.ai/build/parts/contracts-and-limits/budgets#budgets): ```python import json from mcp.server import MCPServer from mcp.types import CallToolResult, TextContent server = MCPServer("jev") @server.tool() def decide(state: str, questions: dict) -> CallToolResult: """Answer typed questions about a state, with a confidence for each answer.""" answers = ask_the_service(state, questions) usage = {"service": "jev", "unit": "decision", "units": 1, "cost_usd": 0.0004} return CallToolResult( content=[TextContent(type="text", text=json.dumps(answers))], structured_content=answers, _meta={"oz0/usage": usage}, ) ``` `service` and `cost_usd` are required; `unit` and `units` say what you paid for. A list reports several services at once, up to 20. One report may claim at most $100 and 10^9 units, and names are letters, digits and `. _ / : -`, at most 64 characters. Hooks report the same way. `job get`, `usage` and the console show these costs as reported by your plugin, and leave them out of model calls and tokens. The platform takes your plugin's word for what they cost, but not for whose they are: the node's runtime sends the report with the call binding of the tool call that returned it, and the edge charges it to that call's job, under that call's plugin and step, whatever the report names. A report that names another job is refused. ## Sitemap See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.