Choose a Temporal backend

For: cluster operators.

Every job is a Temporal workflow. Orchestrator Zero can keep those workflows in three places, and your nodes cannot tell which: they always talk to the edge, which reaches Temporal for them.

With an external backend or Temporal Cloud the server runs no Temporal of its own: no membership ring, no Temporal schemas in your database. Run server init with the same --temporal-backend (the Helm chart's init Job passes temporal.backend): only embedded, the default, sets up Temporal's databases and schemas in your PostgreSQL. Everything else, nodes, plugins, jobs, flows, routines, approvals and the console, works the same.

Configure embedded Temporal, an external cluster or Temporal Cloud. Nodes keep the same edge connection.

Choose a task

What the edge does for the nodes

• Credentials. The edge adds the backend's API key, or presents its client certificate, on every call it forwards. Nodes never get Temporal credentials, so blocking a node still takes effect on its next call.
• Namespaces. Nodes, the CLI and the console use the tenant's namespace, which is named like the tenant. The edge rewrites it to the backend's name on the way in and back on the way out, in every field and error that carries one. On Temporal Cloud the backend's name is <namespace>.<account> unless you map it.
• Isolation. A node's call to another tenant's namespace is refused, also when the namespace is inside a command, such as a child workflow or a signal to another workflow. Two tenants never share a backend namespace: a mapping that may be another tenant's name is refused, and if two tenants still come to one name (say a mapping to acme.<account> made before tenant acme existed), the edge refuses both tenants' calls and jobs and logs the clash until you map one elsewhere.

An external Temporal cluster

oz0 server start --temporal-backend external \
  --temporal-address temporal.internal:7233 \
  --temporal-tls --temporal-tls-ca-file /etc/oz0/temporal-ca.pem \
  --temporal-tls-cert-file /etc/oz0/temporal-client.pem --temporal-tls-key-file /etc/oz0/temporal-client.key

Payloads are encrypted with each tenant's key before they reach your cluster (see Encrypted payloads).

• Leave out the --temporal-tls* flags for a plaintext frontend; add --temporal-api-key-file if your frontend takes API keys, which the server sends only over TLS (--temporal-tls).
• Namespaces and search attributes. The server checks that each tenant's namespace exists and has the search attributes the runtime sets (Oz0PendingApprovals, Oz0Routine, Oz0RoutineVersion, Oz0Trigger). If one is missing, that tenant's jobs are refused, and the log and the refusal say exactly what to create:

  namespace default lacks the search attributes Oz0PendingApprovals, Oz0Routine, ...:
  temporal operator search-attribute create --namespace default --name Oz0PendingApprovals --type Int; ...
  

  
  Create them yourself, or start the server with --temporal-create-namespaces to let it create them as it does on its embedded Temporal.
• Your Temporal must have Worker Versioning (Worker Deployments) enabled; the server logs whether the backend refuses it, or Schedules, at start.

Temporal Cloud

Create one namespace per tenant first, with Temporal's Ops API, tcld or Terraform; the server never creates anything on Temporal Cloud. Each namespace needs the four search attributes above:

tcld namespace create --namespace default --region aws-eu-central-1 --retention-days 30
tcld namespace search-attributes add --namespace default.a1b2c \
  --search-attribute Oz0PendingApprovals=Int --search-attribute Oz0Routine=Keyword \
  --search-attribute Oz0RoutineVersion=Int --search-attribute Oz0Trigger=Keyword

Then start the servers (or only the edges) with the account ID and a key:

oz0 server start --temporal-backend cloud --temporal-cloud-account a1b2c \
  --temporal-api-key-dir /etc/oz0/temporal-keys

• Keys. --temporal-api-key-dir holds one file per namespace, named like it (default.a1b2c), so each namespace can have its own service account. --temporal-api-key-file or --temporal-api-key-env give one key for every namespace. For mTLS, use --temporal-tls-cert-file and --temporal-tls-key-file: one certificate for every namespace, whose CA you add to each namespace. Never put a key on the command line.
• Rotation. Write the new key into its file: the edge uses it within 2 seconds. A new client certificate is used within 10 seconds, for new connections. A new CA file needs a restart.
• Addresses. The edge reaches each namespace at its namespace endpoint, {namespace}.tmprl.cloud:7233 by default; set --temporal-address for another, with {namespace} where the namespace's name goes.
• Your own names. To use an existing namespace, map the tenant to it: oz0 tenant temporal-namespace acme prod-agents.a1b2c. --default maps it back. Change it between jobs: jobs that run stay in the namespace they started in.
• If a namespace is missing, or its key is refused, the tenant's jobs are refused and the log names what to create or fix.

Limits to know

Temporal Cloud's defaults (limits) are lower than the embedded backend's:

• 100 task queues per Worker Deployment version. A tenant's version holds a queue per agent and flow, plus their sandbox queues, so a tenant on Temporal Cloud has room for far fewer agents than on the embedded backend. Split large teams into tenants.
• 500 actions per second per namespace to start with, rising with use; 10 schedule requests per second.
• 2 MB per payload: larger payloads go to External Storage on the edge, which works the same on every backend.

Flags

Previous section links

How long histories are kept: open guide.

Encrypted payloads: open guide.

Encryption status and key rotation: open guide.

Temporal's UIs: the codec endpoint: open guide.

More than one backend, and moving a tenant: open guide.