Translating providers

For: tenant administrators and cost owners.

Use a separate translation gateway when an agent's API format differs from the provider's. Review which information the translation loses before routing production jobs through it.

The LLM gateway preserves the provider's API format: a call in a format the provider does not speak is refused, never translated. The gateway can still change request settings to enforce tenant controls, such as a reasoning-effort ceiling. Sometimes you still want it, such as an agent written for OpenAI's chat completions that should run on Claude. For that, run a translation gateway as a process of your own, and add it as a provider record with translator.

Translation loses what one format has and the other does not: thinking and its signatures, prompt-cache breakpoints, server tools, citations. So every call through a translating provider is marked lossy: in the provider list, in the cost log, and in a job's cost per branch. Prefer a provider that speaks the agent's format, or an alias that picks one.

Two gateways are supported: agentgateway and Agent Router. LiteLLM is not, after its supply-chain incident.

Run agentgateway next to the edge

agentgateway is one binary. Download a release and check it against the release's SHA-256 file:

v=v1.6.0
curl -fLO https://github.com/agentgateway/agentgateway/releases/download/$v/agentgateway-linux-amd64
curl -fLO https://github.com/agentgateway/agentgateway/releases/download/$v/agentgateway-linux-amd64.sha256
sha256sum -c agentgateway-linux-amd64.sha256 && chmod +x agentgateway-linux-amd64

Give it the upstream API's key and tell it which models go where. This one takes OpenAI chat completions on port 4000 and sends them to Anthropic's Messages API:

llm:
  port: 4000
  models:
  - name: "*"
    provider: anthropic
    params:
      apiKey: $ANTHROPIC_API_KEY

ANTHROPIC_API_KEY=... ./agentgateway-linux-amd64 -f agentgateway.yaml

Run it on the edge's machines, or anywhere only the edges reach: it holds the upstream key, so it is a secret of its own, outside the tenant's secrets. Bind it to loopback or a private network; nothing but the edge's gateway should call it.

Add it as a provider

A cluster admin adds a shared record, which may reach a private address:

oz0 provider set claude-via-agw --shared --translator agentgateway \
  --base-url http://127.0.0.1:4000 --format openai-chat --price-provider anthropic --retains-data=true \
  --description "Claude through agentgateway"
oz0 provider model claude-via-agw claude-haiku-4-5 --shared --capabilities tools --context-window 200k
oz0 model alias set fast claude-via-agw/claude-haiku-4-5

• --format is what the translator takes from the edge: openai-chat here. The runtime calls the model in that format, and the gateway meters the call in it, from the usage the translator returns.
• --price-provider prices the calls as the upstream's models: genai-prices knows claude-haiku-4-5 as Anthropic's.
• The key is the translator's, so the record names no key secret. If your translator wants one from the edge, give the record --auth-header and --key-secret as for any provider.
• A tenant's own record reaches only public addresses unless a cluster admin marks it --internal.

provider list shows it as enabled, lossy (agentgateway), and the console's Models and providers page as lossy · agentgateway.

What lossy looks like

$ oz0 usage --job job-ce70999d-f8f8-430a-aeb7-01d30cd4174b
JOB                                       AGENT  NODE               MODEL                                                  CALLS  INPUT  CACHED  OUTPUT  COST
job-ce70999d-f8f8-430a-aeb7-01d30cd4174b  relay  node-f39357320992  claude-via-agw/claude-haiku-4-5 (lossy: agentgateway)  1      64     0       26      $0.0002

The mark is recorded with each call, so a record changed or removed later does not rewrite what earlier calls were. run and job get print it in the cost per branch, and the console's Cost page and job pages show a lossy badge.