---
title: "Connect a model provider"
description: "Add a provider record and store its key, then choose an alias or model for the agent."
canonical_url: "https://docs.oz0.ai/spend/models-and-keys"
---
# Connect a model provider

> Add a provider record and store its key, then choose an alias or model for the agent.

For: tenant administrators and cost owners.

Agents and tools that use the node's model proxy send cloud model calls through the LLM gateway in the edge. The gateway looks up the tenant's provider record, checks the tenant's data rules, adds the tenant's API key and records tokens and cost, including streamed calls. It preserves the provider's API format while enforcing controls such as the tenant's reasoning-effort ceiling. It never translates between formats; a [translating provider](https://docs.oz0.ai/spend/translating-providers) does, as a process of its own, and its calls are marked lossy.

[Remote harnesses](https://docs.oz0.ai/build/apps/remote-agents/sessions-and-limits#where-the-keys-are) call their provider directly and hold their provider credentials on the node. Their model calls do not pass through this gateway.

Add a provider record and store its key, then choose an alias or model for the agent.

## Choose a task

<table>
<thead>
  <tr>
    <th>
      Task
    </th>
    
    <th>
      Guide
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Choose model aliases
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/aliases">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Map a plugin's models
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/mapping">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Set reasoning effort and cascades
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/effort-and-cascades">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Authenticate model calls
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/authentication">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Control what model calls can reach
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/data-rules">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Pool model deployments
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/deployment-pools">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Configure caches and server tools
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/cache-and-tools">
        Open guide
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      Check model changes and retirement
    </td>
    
    <td>
      <a href="https://docs.oz0.ai/spend/models-and-keys/canaries-and-retirement">
        Open guide
      </a>
    </td>
  </tr>
</tbody>
</table>

## Providers are records

A provider is a record: its base URL, the wire formats it speaks, the header its key goes in and the tenant secret that holds the key, its models' prices, context windows and capabilities, its region, and whether it keeps data after a call.

<table>
<thead>
  <tr>
    <th>
      Built in
    </th>
    
    <th>
      Formats
    </th>
    
    <th>
      Key secret
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        anthropic
      </code>
    </td>
    
    <td>
      <code>
        anthropic-messages
      </code>
      
      : Anthropic's Messages API
    </td>
    
    <td>
      <code>
        ANTHROPIC_API_KEY
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        openai
      </code>
    </td>
    
    <td>
      <code>
        openai-chat
      </code>
      
       and <code>
        openai-responses
      </code>
      
      : chat completions and the Responses API
    </td>
    
    <td>
      <code>
        OPENAI_API_KEY
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        local
      </code>
    </td>
    
    <td>
      <code>
        openai-chat
      </code>
      
      , on the job's own node: see <a href="https://docs.oz0.ai/fleet/local-models">
        Local models
      </a>
    </td>
    
    <td>
      none
    </td>
  </tr>
</tbody>
</table>

Presets for other providers ship with the server: OpenRouter, Mistral, Groq, DeepSeek, Together, xAI, Azure OpenAI (with a key, or with an Entra app), AI Foundry with an Entra app, Vertex AI, Bedrock (with an access key, or a Bedrock API key), Google's Gemini API (`google`, in the `google-generate-content` format, for harness agents such as Gemini CLI and agents made from the [Google ADK template](https://docs.oz0.ai/build/parts/harnesses/custom-harnesses#frameworks-as-harnesses), which need it; key secret `GEMINI_API_KEY`) and a shared vLLM server. A tenant adds one, or any endpoint of its own:

```bash [Terminal]
oz0 provider list --presets
oz0 provider set mistral --preset mistral
printf %s "$MISTRAL_API_KEY" | oz0 secret set MISTRAL_API_KEY
oz0 provider model mistral mistral-large --capabilities tools,structured_output --context-window 128k \
  --max-output 32k --input 2 --output 6
oz0 provider set vllm --preset vllm --base-url http://vllm.internal:8000 --region eu --internal
# An endpoint with no preset: say whether it keeps data, since the no-retention rule trusts it.
printf %s "$ACME_API_KEY" | oz0 secret set ACME_API_KEY --host llm.acme.example
oz0 provider set acme --base-url https://llm.acme.example --format openai-chat --key-secret ACME_API_KEY \
  --region eu --retains-data=false
```

A provider's key goes only to a host it is bound to. The built-in providers' keys (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`)
go to their own provider's address as the edge's operator set it, and each preset's key to the preset's own address
(`MISTRAL_API_KEY` to `api.mistral.ai`, `GOOGLE_SERVICE_ACCOUNT_KEY` to `oauth2.googleapis.com`), without a binding of
their own.

Every other key, and a preset's key at another address, needs one: `oz0 secret set NAME --host HOST` with
the value, from a holder of `secrets.write`.

The edge checks it at every call, for a header key and a signature at the
record's (or the pool member's) address.

A key that mints tokens is checked at the token URL and at the record's
address, where its tokens go, so it is bound to both: a preset whose address the record names (`vertex`, the Entra
presets) needs that address bound with its key.

A record that names another secret of the tenant, or points a key or
its tokens at another host, gets `412 oz0_key_not_bound` before anything is sent or minted, and its address sees
nothing; `provider set` refuses such a record at once when it can tell.

Someone who may manage models but not list the
tenant's secrets (`secrets.read`) may set a record only once its key is set and bound to the record's address, and is
refused with the same answer whether the key is missing, a reference to your own store or bound elsewhere; whether a
record's key is set (`oz0 provider list`'s KEY column, the console's first-run steps) is shown only with
`secrets.read`, and says "not shown" to anyone else.

A secret a plugin asks for is no provider's key: a record may not read one, and no plugin may ask for a
provider's key.

A key's test before it is set (`secret set` without `--no-test`) calls only the records at an address
the key will be bound to.

A key, and a token it mints, go only over `https`, or plain `http` to the same machine (a
local server on `127.0.0.1`): `provider set` refuses a record that would send one over plain `http` to another host,
and the edge refuses the call.

A shared server over plain `http`, such as the `vllm` record above, takes no key.

Agents then name `mistral/mistral-large`, or an alias points at it. The runtime calls each model in the first format its record lists, through the node's proxy at `/llm/<provider>/...`.

<table>
<thead>
  <tr>
    <th>
      Field
    </th>
    
    <th>
      What it is
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        base_url
      </code>
    </td>
    
    <td>
      What a call's path, such as <code>
        v1/chat/completions
      </code>
      
      , is appended to
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        formats
      </code>
    </td>
    
    <td>
      <code>
        anthropic-messages
      </code>
      
      , <code>
        openai-chat
      </code>
      
      , <code>
        openai-responses
      </code>
      
      , <code>
        google-generate-content
      </code>
      
       (Google's Gemini API, metered from its <code>
        usageMetadata
      </code>
      
      ; Pydantic AI agents cannot call it yet, harness agents such as Gemini CLI can), the default first. A call in another format gets <code>
        400 oz0_wrong_format
      </code>
      
      : the gateway does not translate.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        auth.header
      </code>
      
      , <code>
        auth.prefix
      </code>
      
      , <code>
        key_secret
      </code>
    </td>
    
    <td>
      The key goes in this header, after the prefix (such as <code>
        Bearer
      </code>
      
      ), from this tenant secret. The record holds the secret's name, never the key.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        auth.kind
      </code>
      
       and the rest of <code>
        auth
      </code>
    </td>
    
    <td>
      How calls are signed besides a header key: an OAuth2 token or SigV4. See <a href="https://docs.oz0.ai/spend/models-and-keys/authentication#how-calls-are-signed">
        How calls are signed
      </a>
      
      .
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        models
      </code>
    </td>
    
    <td>
      Per model: <code>
        context
      </code>
      
      , <code>
        capabilities
      </code>
      
      , <code>
        max_output_tokens
      </code>
      
       (its largest output, which the gateway <a href="https://docs.oz0.ai/build/parts/contracts-and-limits/budgets#each-call-reserves-its-worst-case">
        reserves
      </a>
      
       for a call that sets no <code>
        max_tokens
      </code>
      
      ; 128,000 when not given), and prices in dollars per million tokens, or per second
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        price_source
      </code>
    </td>
    
    <td>
      <code>
        genai-prices
      </code>
      
       (the default, as <code>
        price_provider
      </code>
      
       when its ID differs), <code>
        reported
      </code>
      
       (the provider's own <code>
        usage.cost
      </code>
      
      , as OpenRouter reports it) or <code>
        none
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        region
      </code>
      
      , <code>
        retains_data
      </code>
    </td>
    
    <td>
      Where it processes data, such as <code>
        eu
      </code>
      
       or <code>
        us
      </code>
      
      , and whether it keeps prompts or answers. Data rules check both. A new record without a preset must give <code>
        --retains-data=true
      </code>
      
       or <code>
        --retains-data=false
      </code>
      
      ; the console starts a new record as one that keeps data.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        model_in
      </code>
    </td>
    
    <td>
      <code>
        body
      </code>
      
       (the default) or <code>
        path
      </code>
      
      , for APIs that name the model in the URL
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        translator
      </code>
    </td>
    
    <td>
      <code>
        agentgateway
      </code>
      
       or <code>
        agent-router
      </code>
      
       when the record is a translation gateway you run; its calls are lossy. See <a href="https://docs.oz0.ai/spend/translating-providers">
        Translating providers
      </a>
      
      .
    </td>
  </tr>
</tbody>
</table>

A tenant's own record replaces a shared one of the same ID for that tenant only; `--shared` changes the record every tenant sees, which only cluster admins may, and starts from that shared record, never from the tenant's own copy. Edges pick up changes within two seconds.

A tenant's own record reaches only public addresses: the gateway refuses to connect to loopback, private networks and link-local addresses such as cloud metadata services, after resolving the name, so a tenant cannot reach the edge's own network through it.

When the edge reaches providers through an HTTP proxy (`HTTPS_PROXY`), the gateway checks the target before handing it to the proxy: the name must resolve, on the edge, to public addresses only.

A server on that network, such as a shared vLLM, needs a cluster admin to mark the record `--internal`, or to make it a shared record. Only a cluster admin may turn `internal` on or off; a tenant admin may change everything else in such a record, and the flag stays as it is. The console has the same under Spend, Models and providers.

## Store the key once

```bash [Terminal]
printf %s "$ANTHROPIC_API_KEY" | oz0 secret set ANTHROPIC_API_KEY
```

The key is sealed with the master key and unsealed only in the edge. If a tenant has no key for a provider, the gateway answers `412` with the command that sets it. An agent with a fallback model on another provider falls back to it; when every model was refused, the job fails at once with `MissingKey` and that command, without retrying.

A key that is there but cannot be read now (the secrets store is away, or the value does not open with the edge's sealing key) is not a missing key: the call gets `503 oz0_credential_unavailable`, which says not to set the key again and which the call's activity retries, and the edge logs why. `oz0 doctor` reports such a key as one that cannot be read, not as missing.

## Point a built-in provider somewhere else

```bash [Terminal]
oz0 server start --llm-anthropic-url https://llm-proxy.internal
```

`--llm-anthropic-url` and `--llm-openai-url` point the shared `anthropic` and `openai` records at a corporate proxy, a mirror or a test server, on that server only. To route a tenant elsewhere for good, change the record: `oz0 provider set anthropic --shared --base-url ...`, or give the tenant a record of its own.

## Previous section links

<span id="name-the-model-or-ask-for-an-alias">

Name the model, or ask for an alias

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/aliases#name-the-model-or-ask-for-an-alias).

<span id="the-standard-aliases">

The standard aliases

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/aliases#the-standard-aliases).

<span id="map-an-alias-at-install">

Map an alias at install

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/aliases#map-an-alias-at-install).

<span id="map-a-plugins-models">

Map a plugin's models

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/mapping#map-a-plugins-models).

<span id="naming-presets">

Naming presets

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/mapping#naming-presets).

<span id="verified-plugins-keep-their-status">

Verified plugins keep their status

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/mapping#verified-plugins-keep-their-status).

<span id="a-judge-and-the-agent-it-judges">

A judge and the agent it judges

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/mapping#a-judge-and-the-agent-it-judges).

<span id="no-model-is-replaced-silently">

No model is replaced silently

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/aliases#no-model-is-replaced-silently).

<span id="reasoning-effort">

Reasoning effort

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/effort-and-cascades#reasoning-effort).

<span id="a-cascade-the-next-model-only-when-a-round-fails">

A cascade: the next model only when a round fails

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/effort-and-cascades#a-cascade-the-next-model-only-when-a-round-fails).

<span id="how-calls-are-signed">

How calls are signed

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/authentication#how-calls-are-signed).

<span id="two-tenants-one-key">

Two tenants, one key

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/authentication#two-tenants-one-key).

<span id="data-rules">

Data rules

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/data-rules#data-rules).

<span id="what-a-call-may-make-the-provider-read-or-fetch">

What a call may make the provider read or fetch

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/data-rules#what-a-call-may-make-the-provider-read-or-fetch).

<span id="pools-of-deployments">

Pools of deployments

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/deployment-pools#pools-of-deployments).

<span id="why-nodes-never-see-keys">

Why nodes never see keys

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/authentication#why-nodes-never-see-keys).

<span id="provenance-and-vetting">

Provenance and vetting

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/data-rules#provenance-and-vetting).

<span id="cost">

Cost

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/cache-and-tools#cost).

<span id="the-prompt-cache">

The prompt cache

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/cache-and-tools#the-prompt-cache).

<span id="server-tools">

Server tools

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/cache-and-tools#server-tools).

<span id="what-a-call-may-cost-besides-tokens">

What a call may cost besides tokens

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/cache-and-tools#what-a-call-may-cost-besides-tokens).

<span id="model-canaries">

Model canaries

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/canaries-and-retirement#model-canaries).

<span id="when-a-model-retires">

When a model retires

</span>

: [open guide](https://docs.oz0.ai/spend/models-and-keys/canaries-and-retirement#when-a-model-retires).


## Sitemap

See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.