---
title: "Cloud federation"
description: "Reach AWS, Google Cloud and Azure with no stored key: each cloud trusts the tenant's issuer, and the edge exchanges the agent's own token for a credential that lives an hour."
canonical_url: "https://docs.oz0.ai/fleet/cloud-federation"
---
# Cloud federation

> Reach AWS, Google Cloud and Azure with no stored key: each cloud trusts the tenant's issuer, and the edge exchanges the agent's own token for a credential that lives an hour.

For: cluster operators.

Configure the cloud's trust before using a federated provider or tool. The edge exchanges the calling agent's identity for a short-lived cloud credential.

AWS, Google Cloud and Microsoft Entra can trust an outside OIDC issuer instead of a key you store. Each tenant has one on
the edge ([identity for outside services](https://docs.oz0.ai/concepts/delegation-and-identity#identity-for-outside-services)), so a
cloud can trust the tenant's agents directly:

1. A model call or a tool call needs the cloud. The edge mints the calling agent's workload token for the cloud's
token service (five minutes, one audience, the agent as `sub`, never a person).
2. The edge exchanges it at the cloud: AWS STS `AssumeRoleWithWebIdentity`, Google Cloud's security token service (and,
if you name one, the service account it impersonates), or Entra's token endpoint with the token as the client
assertion of an app registration's federated credential.
3. The cloud checks the token through the issuer's discovery document and key set, against the trust you set up below,
and answers a credential that lives an hour.
4. For a model call, the edge signs the call with it and nothing else sees it. For a tool call, the credential goes to
the node sealed to it, and only the node's egress proxy opens it: it signs the tool's AWS requests or puts the
Google or Entra token on, while the tool holds a placeholder.

Nothing is stored: the edge keeps a credential in memory for the job's next calls until a few minutes before it ends.
A credential lives as long as the cloud says (`expires_in`, or AWS's `Expiration`), at most an hour: an answer that
gives no lifetime, or one that has already ended, is an error, never a credential. When the edge's memory for
credentials is full of live ones, a new call's credential serves that call alone, and no other job's is dropped for it.

A cloud's refusal (a trust policy, an audience: a 4xx answer) is told apart from an outage (no answer, a 5xx, 408 or
429), which may pass: the refusal names the cloud's own error code, and an outage is audited as a failed exchange
(`token.exchange_refused`, marked `unavailable`) with its cause.

The audit log records each exchange as `token.exchanged` (the cloud, the role, pool or application it reached, the
agent and its subject, the job), never a token or a credential.

Only the agent's own plugin's tools get its cloud
credentials: a tool of another plugin that the agent was granted gets none.

Nothing is sent to a cloud until a tenant admin sets up a federated
provider or installs a plugin with a federated connection.

## What the clouds see

<table>
<thead>
  <tr>
    <th>
      
    </th>
    
    <th>
      
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Issuer (<code>
        iss
      </code>
      
      )
    </td>
    
    <td>
      <code>
        https://&lt;edge>/oidc/&lt;tenant>
      </code>
      
      , from <code>
        oz0 tenant oidc show &lt;tenant>
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Subject (<code>
        sub
      </code>
      
      )
    </td>
    
    <td>
      <code>
        oz0:agent:&lt;tenant>:&lt;agent>:&lt;install generation>
      </code>
      
      : it survives upgrades of the plugin and changes when the plugin is removed and installed again
    </td>
  </tr>
  
  <tr>
    <td>
      Audience (<code>
        aud
      </code>
      
      )
    </td>
    
    <td>
      AWS: <code>
        https://sts.amazonaws.com
      </code>
      
      ; Google: <code>
        https://iam.googleapis.com/
      </code>
      
       and the pool provider's name; Entra: <code>
        api://AzureADTokenExchange
      </code>
      
      . A provider or connection may name another (<code>
        audience
      </code>
      
      )
    </td>
  </tr>
  
  <tr>
    <td>
      AWS session tag
    </td>
    
    <td>
      <code>
        oz0-agent
      </code>
      
      : the agent's name, in the token's <code>
        https://aws.amazon.com/tags
      </code>
      
       claim
    </td>
  </tr>
  
  <tr>
    <td>
      AWS session name
    </td>
    
    <td>
      <code>
        oz0-
      </code>
      
       and the job's ID, so CloudTrail names the job
    </td>
  </tr>
</tbody>
</table>

The clouds' endpoints are fixed: `sts.<region>.amazonaws.com`, `sts.googleapis.com` and
`iamcredentials.googleapis.com`, `login.microsoftonline.com`. A manifest or a tenant names the role, the pool or the
application, never where the agent's token goes. An operator can replace an endpoint, for a sovereign cloud, with the
server's `--cloud-endpoint aws-sts=<URL>` (also `gcp-sts`, `gcp-iam`, `entra`).

## AWS

1. An IAM OIDC identity provider for the issuer: in IAM, add an OpenID Connect provider with the provider URL set to
the tenant's issuer and the audience `https://sts.amazonaws.com`. AWS fetches the discovery document over the
internet; for a private cluster, [publish it](https://docs.oz0.ai/fleet/cloud-federation#private-clusters) first.
2. A role whose trust policy names the agent, by `aud` and `sub`, and allows the session tag:```json [trust-policy.json]
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Federated": "arn:aws:iam::123456789012:oidc-provider/oz0-edge.acme.example/oidc/acme"},
    "Action": ["sts:AssumeRoleWithWebIdentity", "sts:TagSession"],
    "Condition": {
      "StringEquals": {
        "oz0-edge.acme.example/oidc/acme:aud": "https://sts.amazonaws.com",
        "oz0-edge.acme.example/oidc/acme:sub": "oz0:agent:acme:triage:42"
      }
    }
  }]
}
```

<br />

`sts:TagSession` is required: every token carries the agent as a session tag, and AWS refuses the role without it.<br />

The subject ends in the agent's install generation (`42` here; every exchange's audit entry, refused ones too, names the full subject: `oz0 audit --action token.`), so
`StringEquals` on it trusts one install of one agent. After a reinstall the agent has a new subject, and you
update the policy on purpose.<br />

Do not match the generation with a wildcard (`StringLike` on
`oz0:agent:acme:triage:*`): any later agent named `triage` in the tenant, from any plugin and any publisher, would
then assume the role, which is what the generation in the subject is there to prevent. For the same reason
`aws:PrincipalTag/oz0-agent` (the agent's name alone) may narrow a policy but never replaces the subject.
3. A model provider for Bedrock:```bash [Terminal]
oz0 provider set bedrock --preset bedrock-federated --base-url https://bedrock-runtime.eu-north-1.amazonaws.com \
  --aws-role arn:aws:iam::123456789012:role/oz0-agents --aws-region eu-north-1 --region eu
```

<br />

or a connection for tools ([in this guide](https://docs.oz0.ai/fleet/cloud-federation#in-a-plugin)).

## Google Cloud

1. A workload identity pool and an OIDC provider for the issuer: the issuer URI is the tenant's issuer; map
`google.subject` to `assertion.sub` and, if you like, `attribute.agent` to `assertion.sub`; keep the default allowed
audience (`https://iam.googleapis.com/projects/<number>/locations/global/workloadIdentityPools/<pool>/providers/<provider>`).
Google can take the key set as an upload instead of fetching it: `oz0 tenant oidc jwks <tenant> > jwks.json`, then
give the file to the provider (`--jwk-json-path` in `gcloud`), and upload it again after every rotation.
2. Access for the agent, one of the following, always by the full subject (with its install generation): a
`principalSet` over an attribute without the generation, such as `attribute.agent` mapped from the agent's name,
trusts every later agent of that name in the tenant.
  - grant roles to the principal `principal://iam.googleapis.com/projects/<number>/locations/global/workloadIdentityPools/<pool>/subject/oz0:agent:acme:triage:<generation>` directly, or
  - grant it `roles/iam.workloadIdentityUser` on a service account, and name that service account: the edge then
  exchanges the federated token for the service account's own (`generateAccessToken`, an hour).
3. A model provider for Vertex AI:```bash [Terminal]
oz0 provider set vertex --preset vertex-federated --base-url https://europe-west1-aiplatform.googleapis.com --region eu \
  --gcp-workload-provider //iam.googleapis.com/projects/123456/locations/global/workloadIdentityPools/oz0/providers/acme \
  --gcp-service-account agents@acme-prod.iam.gserviceaccount.com
```

## Microsoft Entra

1. An app registration (or a user-assigned managed identity) with a federated credential: issuer = the
tenant's issuer, subject identifier = the agent's `sub`, audience = `api://AzureADTokenExchange`. Entra matches all
three exactly and takes no wildcards, so each agent needs a federated credential of its own, and an app
registration or managed identity holds at most 20: one app serves at most 20 agents (use more apps for more).
Entra fetches the discovery document over the internet.<warning>

Microsoft supports workload identity federation only with issuers that sign their tokens with RS256; tokens signed
with other algorithms "may work, but haven't been tested". The tenant's issuer signs with ES256, which AWS and
Google document as supported. Entra's path is tested here only against a stand-in of Entra's token endpoint and has
not yet run against Microsoft's: treat it as unproven until it has.

</warning>
2. Access for the app: a role on the Azure OpenAI or AI Foundry resource (Cognitive Services OpenAI User, for
example).
3. A model provider:```bash [Terminal]
oz0 provider set foundry --preset azure-foundry-federated --base-url https://acme.services.ai.azure.com/anthropic \
  --entra-directory acme.onmicrosoft.com --client-id 11112222-bbbb-3333-cccc-4444dddd5555 --region eu
```

<br />

(`azure-openai-federated` for Azure OpenAI's v1 API.)

### Where a provider's token may go

A Google or Entra token is a bearer token: whoever receives it can call the cloud as the agent until it ends. So a
federated provider's `--base-url` must be a host of its cloud's own model API, on https's port:

<table>
<thead>
  <tr>
    <th>
      Cloud
    </th>
    
    <th>
      Hosts
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Google Cloud
    </td>
    
    <td>
      <code>
        aiplatform.googleapis.com
      </code>
      
      , <code>
        &lt;region>-aiplatform.googleapis.com
      </code>
      
      , <code>
        generativelanguage.googleapis.com
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Entra
    </td>
    
    <td>
      None of its own: only the resources the operator names, below
    </td>
  </tr>
</tbody>
</table>

`provider set` refuses any other address, for the record and for each pool member, and the edge checks the
address again at every call, before it asks the cloud for a token; the call follows no redirect. A host
outside the list (a sovereign cloud's, a private endpoint's, a test's fake) is one the operator names for the cloud
on the servers: `--cloud-endpoint entra-models=https://aoai.private.acme.example` (or `gcp-models=...`), repeatable.
A record matches such a host only with the same scheme, host and port, so `http://aoai.private.acme.example:443` is
not `https://aoai.private.acme.example` and is refused: the token never goes out in clear text.

Azure's model hosts (`<resource>.openai.azure.com`, `<resource>.services.ai.azure.com`,
`<resource>.cognitiveservices.azure.com`) are names any Azure customer can register, so such a name says Microsoft runs
the host, not that the resource is yours: someone who may write a provider record could otherwise point a federated
provider at a resource of their own.

So an Entra record needs its host named on the servers, on management and on the
edges alike: `--cloud-endpoint entra-resources=https://acme-ai.openai.azure.com`, repeatable. An Entra token goes to
those and to `entra-models` hosts, and to no other host. With neither named, `provider set` refuses the record, the edge
refuses every call of one already saved, and the error names the flag to set.

An Entra record's `auth.scope` must be a scope of Azure's model API (`https://cognitiveservices.azure.com/.default` or
`https://ai.azure.com/.default`, or a sovereign cloud's), so a record cannot ask for a token of another service; a host
named with `entra-models` answers for itself, and takes the scope its record names.

An AWS session signs each request for the host it reaches, which no other host can use, so a Bedrock record's address
is not limited this way.

## In a plugin

A tool reaches a cloud through a connection of kind `federated`, and runs under bindings of it: its credential reaches
the node only sealed, so a connection without bindings, or a remote tool, gets nothing.

```yaml [oz0-plugin.yaml]
egress: ["*.s3.eu-north-1.amazonaws.com", "storage.googleapis.com"]
requires:
  inject:
    aws:
      - {host: "*.s3.eu-north-1.amazonaws.com", sigv4: {service: s3, region: eu-north-1}}
    gcs:
      - {host: storage.googleapis.com, header: Authorization, format: "Bearer {secret}"}
connections:
  - name: aws
    description: The tenant's AWS account, by the agent's own identity
    tools: [files]
    params:
      - {name: role_arn, description: "The IAM role the agents assume"}
      - {name: region, description: "The role's region", default: eu-north-1}
    token:
      kind: federated
      federated: {cloud: aws, role_arn: "{params.role_arn}", region: "{params.region}"}
      pass: {region: "{params.region}"}
    scopes:
      - functions: [files.put, files.get]
  - name: gcs
    description: The tenant's Google Cloud project
    tools: [files]
    params:
      - {name: provider, description: "The workload identity pool provider"}
    token:
      kind: federated
      federated: {cloud: gcp, workload_provider: "{params.provider}"}
    scopes:
      - functions: [files.copy_to_gcs]
```

<table>
<thead>
  <tr>
    <th>
      <code>
        federated
      </code>
    </th>
    
    <th>
      
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        cloud
      </code>
    </td>
    
    <td>
      <code>
        aws
      </code>
      
      , <code>
        gcp
      </code>
      
       or <code>
        entra
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        role_arn
      </code>
      
      , <code>
        region
      </code>
    </td>
    
    <td>
      AWS: the role and the region of its STS endpoint
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        workload_provider
      </code>
      
      , <code>
        service_account
      </code>
    </td>
    
    <td>
      Google Cloud: the pool provider's resource name, and a service account to impersonate
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        directory
      </code>
      
      , <code>
        client_id
      </code>
    </td>
    
    <td>
      Entra: the directory and the application with the federated credential
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        scope
      </code>
    </td>
    
    <td>
      What a Google or Entra token is for (Google's default: <code>
        cloud-platform
      </code>
      
      )
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        audience
      </code>
    </td>
    
    <td>
      The workload token's audience, when the cloud's trust names another than the default
    </td>
  </tr>
</tbody>
</table>

Each field is a value or a template of the connection's params; a tenant admin sets the params, never a secret.
`plugin lint` refuses a federated AWS connection bound without `sigv4`, a Google or Entra one bound with it, and a
connection with no bindings at all.

In the tool, the call's `_meta["oz0/connections"]["aws"]["token"]` is a placeholder access key ID. Give it to your
AWS SDK as the access key ID, with any secret access key and no session token; the node's egress proxy takes the SDK's
signature off and signs the request again with the session ([SigV4 signing](https://docs.oz0.ai/admin/credential-injection#sign-aws-requests)).
For Google and Entra, the token is a placeholder bearer token, put on as any [bound connection's token](https://docs.oz0.ai/admin/credential-injection#bind-a-connections-token).

## Private clusters

AWS and Entra fetch the issuer's discovery document and key set over public HTTPS; Google can take an uploaded key set.
A cluster whose edges the internet cannot reach:

1. Starts its edges with `--public-edge-url https://oidc.acme.example`: the issuer becomes
`https://oidc.acme.example/oidc/<tenant>`, a place you control.
2. Publishes the documents there:```bash [Terminal]
oz0 tenant oidc publish acme --to /srv/www/oidc.acme.example        # a folder a web server serves
oz0 tenant oidc publish acme --to 's3://acme-oidc?region=eu-north-1' # a bucket behind your https name
```

<br />

It reads the documents from an edge (`--from https://<edge>:<port>`, by default the CLI context's), checks them,
and writes `oidc/<tenant>/.well-known/openid-configuration` and `oidc/<tenant>/jwks.json` with an hour's cache.
For an `s3://` target the credentials come from the `AWS_*` variables or `--credentials-file`.
3. Publishes again after every rotation: the key set holds the next key ahead of a rotation, so tokens keep
verifying until the following one; after an emergency rotation, publish at once, since its new key is in no
published set.

## When it fails

<table>
<thead>
  <tr>
    <th>
      What you see
    </th>
    
    <th>
      Why
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        412 oz0_key_not_bound
      </code>
      
      : "its cloud's token goes only to the cloud's own model API"
    </td>
    
    <td>
      The record's address is not a host of its cloud's model API (see above), so no token was asked for and nothing was sent
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        403 oz0_federation
      </code>
      
      : "called only by an agent's model call"
    </td>
    
    <td>
      A call with no agent behind it (an operator's, or an older node's that names no job): there is no agent to name in the token. Use a provider with a key for such calls
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        AccessDenied
      </code>
      
       from AWS STS
    </td>
    
    <td>
      The trust policy does not match <code>
        aud
      </code>
      
       or <code>
        sub
      </code>
      
      , or does not allow <code>
        sts:TagSession
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        InvalidIdentityToken
      </code>
    </td>
    
    <td>
      AWS could not fetch or match the issuer's documents: the provider URL, or an unpublished private cluster
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        invalid_target
      </code>
      
       from Google
    </td>
    
    <td>
      The pool provider's name, or its allowed audiences
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        AADSTS70021
      </code>
      
       from Entra
    </td>
    
    <td>
      No federated credential matches the issuer, the subject and the audience exactly
    </td>
  </tr>
  
  <tr>
    <td>
      A tool's upload is refused for "chunk by chunk"
    </td>
    
    <td>
      The SDK signs each chunk of the body with its own key; configure it for unsigned payloads with a trailing checksum (its default over HTTPS) or a hash of the whole body
    </td>
  </tr>
</tbody>
</table>

## Limits

- "Bound to the call" holds inside Orchestrator Zero. The credential a cloud issues is a bearer credential at that
cloud for its hour; it reaches only the edge and, sealed, the node's egress proxy.
- Workload tokens only: a cloud never receives a person's identifier from this path.
- Entra: Microsoft supports only RS256-signed issuer tokens, and the issuer signs ES256, so the Entra exchange
runs outside what Microsoft has tested; an app registration or managed identity holds at most 20 federated
credentials, one per agent.
- The fakes these features are tested against are shaped from the clouds' documentation; see
[integration maturity](https://docs.oz0.ai/reference/compatibility#integrations-awaiting-live-validation).


## Sitemap

See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.
