Fleet

Join nodes

Turn any machine into a node, with a join token or by approval.

For: cluster operators.

Choose a token for unattended installation or approval for a node an operator will accept. After joining, check the node in the fleet before sending it work.

A node needs only outbound access to the edge on ports 7233 and 7443. It creates and stores its private key locally; joining sends no private key to the edge. At sandbox tier 0, plugins run with the node user's rights and can read that key. Use tier 1 to keep plugins out of the node's identity files.

With one command

On the console's Nodes page, Add nodes creates a join token and shows one command for Linux and macOS, and one for Windows (see Windows):

Terminal
d=$(mktemp -d) && curl -fsSk https://edge.example.com:7443/v1/ca.crt -o "$d/ca.crt" \
  && echo "<sha256 of the CA>  $d/ca.crt" | (sha256sum -c --quiet || shasum -a 256 -c --quiet) \
  && curl -fsS --cacert "$d/ca.crt" https://edge.example.com:7443/install.sh \
  | OZ0_JOIN_TOKEN='K10...' OZ0_CA="$d/ca.crt" OZ0_CA_SHA256='<sha256 of the CA>' sh -s --

It downloads the cluster's CA certificate into a folder only you can write, checks it against the hash in the command, and fetches the install script over TLS that trusts only that CA, so nothing it runs can have been changed on the way. A fixed path in /tmp is never used: another user of the machine could put a file of their own there first.

The script copies the certificate into a folder of its own, checks the copy against the same hash before it downloads anything, and trusts only the copy (--ca FILE and --ca-sha256 HEX say the same as OZ0_CA and OZ0_CA_SHA256; one of the two for the CA is required).

The script downloads orchestrator-zero-node for the machine's platform and checks its digest, installs it in /usr/local/bin (or ~/.local/bin without root), joins, and runs the node as a service: with sudo, a systemd unit or launchd daemon of the machine that runs as its own account; without, a service of your user.

On its first start the node downloads the runtime from the edge too.

install.sh takes --name (the node's name, the machine's host name when left out; at most 253 bytes, as long as a host name may be, with no control or formatting characters, since lists show it in terminals and pages; the edge refuses any other), --data-dir, --dir for the binary, --foreground to run the node in the terminal instead of as a service, --no-run to install and join only, --service-name for a second node on the machine, --no-linger, and --no-sandbox-setup.

As root on Linux it first runs oz0-node sandbox setup, which installs bubblewrap and, on Ubuntu 24.04, loads Ubuntu's own AppArmor profile for it, so that the node starts at sandbox tier 1; without root it prints the node's tier and the command that fixes tier 0.

Where there is no systemd, such as in a container, the node runs in the foreground.

The edge serves what make dist built; see Install. With its toolchain/ folder the node installs uv, Python, the runtime's packages and Node.js from the edge as well, and reaches nothing else (Outbound connections).

With a join token

A token admits nodes without an operator approving each one. Create it on the operator side:

Operator
oz0 token create --tenant default --max-uses 10 --ttl 24h --label gpu=a100

The token is shown once. It carries the cluster's CA hash, so the node can check the server before it sends anything.

A token is bounded and never unlimited: left alone it admits one node and lasts 24 hours. --max-uses takes 1 to 1,000 and --ttl at most 720h (30 days); 0, which once meant "unlimited" or "forever", is refused with the reason by the CLI and the console, and anything past the bounds is refused by all three, the admin API too.

In the admin API, a max_uses of 0 or left out takes the default of one use (the wire cannot tell the two apart), and a ttl of 0 is refused; the token it answers names its bounds.

The edge checks the bounds again each time a token is used, so a token stored before they existed is refused at use with the command that replaces it.

A token joins nodes only to its own tenant and cluster, and a node's ID comes from its own key, so a token never yields another node's identity. On the machine:

Machine
OZ0_JOIN_TOKEN='K10...' oz0-node join --server https://edge.example.com:7443

Then start it, in the foreground (see Run a node as a service to keep it running):

Machine
oz0-node run

Labels on the token are given to every node that joins with it:

Operator
oz0 node list

By approval

Without a token, the node waits until an operator accepts it, as with salt-key -a. Get the CA hash first:

Operator
oz0 cluster info                 # shows the CA hash

On the machine:

Machine
oz0-node join --server https://edge.example.com:7443 --ca-hash sha256:...

The node prints its ID and its key's fingerprint, and waits:

Waiting for approval as node-7f3a01b2c4d5 (key fingerprint sha256:7f3a01b2c4d5...).
An operator accepts this node with:
  oz0 node accept node-7f3a01b2c4d5

On the operator side, compare the fingerprint and accept it:

Operator
oz0 node list --pending
oz0 node accept node-7f3a01b2c4d5 --tenant default --label zone=eu

node reject refuses it instead. A join without a token needs no credential, so the nodes waiting for approval are bounded: at most 1,000 wait in a region (the region of the edge they join through) at once, at most 20 of them from one address (an IPv6 one by its /64; behind trusted proxies, the client they name), and a waiting node keeps its place for 24 hours from when it first asked, however often it asks again.

On IPv6 one subscriber's allocation can hold many /64s, so each edge also remembers which keys it has seen wait and holds a whole /48 to the same 20: past that it refuses a new key from that network, and a key that waits, or a node already let in, goes on asking. That count is each edge's own and starts again when it restarts, so an edge reachable over IPv6 from networks you do not trust should still join its nodes with a token.

After that it is forgotten the next time a new node needs a place; a node still running asks again within seconds and waits as a new one, so an approval that takes days still finds it. A waiting node's joined-from label says which address it counts against; accepting it gives it the labels you accept it with instead.

While the list or the address's share is full, a further join without a token is refused with that reason (ResourceExhausted, which the node waits on for a few minutes): accept or reject the waiting nodes, or join with a token, which never waits.

Many nodes behind one address, such as a NAT, need a token to join more than 20 at a time. Each address may also send only a few join requests a second.

Once accepted, the node finishes joining and says so (Joined cluster ...), and node list shows it with its labels:

Operator
oz0 node list

The node's ID is the start of its key's fingerprint, so comparing the two is how you know the node you accept is the machine you joined.

What run does

oz0-node run runs in the foreground. On first start it installs a pinned uv and a managed Python, and the runtime: the bundle given with --runtime-bundle, or else the one the edge serves, checked against its digest. Then it:

  • opens the control stream to the edge and keeps it open,
  • runs a local Temporal proxy and a local LLM proxy, both holding the node's certificate,
  • starts the agent runtime, which polls oz0.default, the node's own node.<id> queue, the queues of the agents it has, and any queues listed in its queues label,
  • downloads, builds and starts the plugins its tenant has installed for it.

Certificates renew themselves. A blocked node stops its runtime and asks again every ten seconds.

Labels and queues

Labels are key-value pairs: gpu=a100, zone=eu-north, team=data. They decide which plugins a node gets (plugin install --select) and which agents it may run (requires.labels in an agent). Set them on the token or at node accept.

The special label queues lists extra task queues the node polls, for example --label queues=reports,batch. Only operators set it.

Run a node as a service

Machine
oz0-node service install

installs the node you joined as a systemd unit, a launchd job or a Windows service that starts at boot, comes back after a crash, updates itself and stays down once purged. As root it is a service of the machine, which runs as the account oz0-node with its data in /var/lib/orchestrator-zero-node: join with that --data-dir. See Nodes as services.

Small machines

A Raspberry Pi with a 64-bit OS is a linux/arm64 node like any other, good for agents that drive local hardware or run light tools. Give it a label such as device=pi and require it in the agents that need it.

Copyright © 2026