Reference

Admin API

Retry admin API calls safely with idempotency keys.

For: Developers calling the admin API.

The admin API (oz0.admin.v1.AdminService, over Connect on the management port) is what the console and the CLI call. A call that changes something may have been applied even when its answer was lost on the way back. To retry such a call safely, send the same Idempotency-Key header with the same request:

The header: 1 to 128 visible ASCII characters (no spaces), such as a UUID. Anything else is refused with invalid_argument. A new call takes a new key; a retry reuses the first call's.
The same key and request within 24 hours gets the first call's answer again, without the call running again; the answer carries the header Idempotency-Replayed: true.
The same key with another request (or another method) is refused with already_exists and an oz0.runtime.v1.IdConflict detail whose reason is oz0_id_conflict.
While the first call still runs, a second one is refused with aborted (reason oz0_id_pending): try again in a moment with the same key.
What is kept: a success, and an error the same request would meet again (invalid_argument, not_found, already_exists, failed_precondition, out_of_range, unimplemented). A failure a retry may get past (unavailable, internal, deadline_exceeded, a permission not yet given) is not kept, so the retry runs the call.
Keys are per caller: another person's call under the same key is that person's own, and a call the server refuses for its caller keeps nothing.

Access is checked again

Before a kept answer is given again, the server checks your current permission for the method. If you lost it, you get the permission_denied a fresh call gets, without the answer; given the access back within the 24 hours, the same retry gets the first answer. A retry of your own password change is yours whatever your cluster role. A method that declares no access to ask again (RenewOperatorCertificate) is not replayed: the retry gets failed_precondition, and a new key makes a new call.

Stored credentials

An answer that carries a new key, token, password or code is kept sealed with a key derived from your Idempotency-Key, which the server does not store: a copy of the database does not reveal it.

oz0, the SDKs and the console send a key on every call that takes one and retry unavailable with it. A call that takes no header ignores it. Job starts on the runtime API work the same way through their job ID; see the runtime API.

Calls that take the header

Call
`AddIncidentNote`	Takes `Idempotency-Key`
`AllowlistCommand`	Takes `Idempotency-Key`
`ContestDecision`	Takes `Idempotency-Key`
`OpenIncident`	Takes `Idempotency-Key`
`RecordProvisioningEvent`	Takes `Idempotency-Key`
`RenewOperatorCertificate`	Takes `Idempotency-Key`
`RequestBudget`	Takes `Idempotency-Key`
`RollbackSecret`	Takes `Idempotency-Key`
`RotateEvidenceKey`	Takes `Idempotency-Key`
`RotateTenantOIDC`	Takes `Idempotency-Key`
`RotateTenantPayloadKey`	Takes `Idempotency-Key`
`StartComparison`	Takes `Idempotency-Key`
`StartContinuation`	Takes `Idempotency-Key`
`StartReplay`	Takes `Idempotency-Key`
`StartRollout`	Takes `Idempotency-Key`
`TestNotificationRule`	Takes `Idempotency-Key`
`CreateAPIKey`	Takes `Idempotency-Key`; the kept answer is sealed
`CreateApp`	Takes `Idempotency-Key`; the kept answer is sealed
`CreateChannelLinkCode`	Takes `Idempotency-Key`; the kept answer is sealed
`CreateJoinToken`	Takes `Idempotency-Key`; the kept answer is sealed
`CreateUser`	Takes `Idempotency-Key`; the kept answer is sealed
`SetUserPassword`	Takes `Idempotency-Key`; the kept answer is sealed

Calls that need no header

Calls that only read take none and are not listed. These change something and are safe to send again as they are:

Call	Why
`AbortRollout`	Sets a value: twice is the same as once
`AcceptNode`	Sets a value: twice is the same as once
`AddPluginIndex`	Has a natural key: a second create under it is refused
`AllowProvisionerInTenant`	Sets a value: twice is the same as once
`ApplyPluginConfig`	Decided once, by its ID and hash
`ApproveChangeRequest`	Decided once, by its ID and hash
`ApprovePluginTools`	Sets a value: twice is the same as once
`BindCredential`	Sets a value: twice is the same as once
`BlockNode`	Sets a value: twice is the same as once
`CancelContinuation`	Sets a value: twice is the same as once
`CancelReplay`	Sets a value: twice is the same as once
`CancelTenantMove`	Sets a value: twice is the same as once
`CloseIncident`	Sets a value: twice is the same as once
`CollectArtifacts`	Sets a value: twice is the same as once
`ConfirmRegistry`	Sets a value: twice is the same as once
`CreateCandidate`	Has a natural key: a second create under it is refused
`CreateTeam`	Has a natural key: a second create under it is refused
`CreateTenant`	Has a natural key: a second create under it is refused
`DecideBudgetRequest`	Decided once, by its ID and hash
`DeleteApp`	Sets a value: twice is the same as once
`DeleteChannelBinding`	Sets a value: twice is the same as once
`DeleteInventoryItem`	Sets a value: twice is the same as once
`DeleteModelAlias`	Sets a value: twice is the same as once
`DeleteModelMapRule`	Sets a value: twice is the same as once
`DeleteNode`	Sets a value: twice is the same as once
`DeleteNotificationRule`	Sets a value: twice is the same as once
`DeleteOIDCProvider`	Sets a value: twice is the same as once
`DeleteProvider`	Sets a value: twice is the same as once
`DeleteRoutine`	Sets a value: twice is the same as once
`DeleteSecret`	Sets a value: twice is the same as once
`DeleteSetting`	Sets a value: twice is the same as once
`DeleteSubjectRule`	Sets a value: twice is the same as once
`DeleteTeam`	Sets a value: twice is the same as once
`DeleteUser`	Sets a value: twice is the same as once
`DrainNode`	Sets a value: twice is the same as once
`EndSessions`	Sets a value: twice is the same as once
`EraseSubject`	Sets a value: twice is the same as once
`GrantReadsTarget`	Sets a value: twice is the same as once
`GrantUses`	Sets a value: twice is the same as once
`ImportAdvisories`	Sets a value: twice is the same as once
`InstallPlugin`	Has a natural key: a second create under it is refused
`LinkIncident`	Sets a value: twice is the same as once
`LinkTemporalCloudUser`	Sets a value: twice is the same as once
`LinkUser`	Sets a value: twice is the same as once
`MarkIncidentNotified`	Sets a value: twice is the same as once
`MoveTenant`	Has a natural key: a second create under it is refused
`NameRegion`	Sets a value: twice is the same as once
`PurgeNode`	Sets a value: twice is the same as once
`PutNotificationRule`	Sets a value: twice is the same as once
`PutOIDCProvider`	Sets a value: twice is the same as once
`RefreshPluginIndex`	Sets a value: twice is the same as once
`RejectChangeRequest`	Decided once, by its ID and hash
`RejectNode`	Sets a value: twice is the same as once
`ReleaseKillSwitch`	Sets a value: twice is the same as once
`RemoveAccount`	Sets a value: twice is the same as once
`RemoveAdvisories`	Sets a value: twice is the same as once
`RemoveCandidate`	Sets a value: twice is the same as once
`RemoveChannelMember`	Sets a value: twice is the same as once
`RemoveMembership`	Sets a value: twice is the same as once
`RemoveMyChannelLink`	Sets a value: twice is the same as once
`RemovePlugin`	Sets a value: twice is the same as once
`RemovePluginIndex`	Sets a value: twice is the same as once
`RemoveProvisionerFromTenant`	Sets a value: twice is the same as once
`RemoveTeamMember`	Sets a value: twice is the same as once
`RenewCandidate`	Sets a value: twice is the same as once
`RenewMemoryEntry`	Sets a value: twice is the same as once
`ResetUserMFA`	Sets a value: twice is the same as once
`ResolveEvalFinding`	Sets a value: twice is the same as once
`RestrictSubject`	Sets a value: twice is the same as once
`RetireInventoryItem`	Sets a value: twice is the same as once
`RetirePlugin`	Sets a value: twice is the same as once
`RevokeAPIKey`	Sets a value: twice is the same as once
`RevokeJoinToken`	Sets a value: twice is the same as once
`RevokeMCPClient`	Sets a value: twice is the same as once
`RevokeProvides`	Sets a value: twice is the same as once
`RevokeReads`	Sets a value: twice is the same as once
`RevokeReadsTarget`	Sets a value: twice is the same as once
`RevokeUses`	Sets a value: twice is the same as once
`RollBackPluginConfig`	Decided once, by its ID and hash
`SaveChannelBinding`	Sets a value: twice is the same as once
`SaveRoutine`	Sets a value: twice is the same as once
`SeedModelAliases`	Sets a value: twice is the same as once
`SetAILabel`	Sets a value: twice is the same as once
`SetAccountActive`	Sets a value: twice is the same as once
`SetAccountGroups`	Sets a value: twice is the same as once
`SetAccountMemberships`	Sets a value: twice is the same as once
`SetAgentLegalBasis`	Sets a value: twice is the same as once
`SetAgentPersonMemory`	Sets a value: twice is the same as once
`SetAssessmentFields`	Sets a value: twice is the same as once
`SetChannelMember`	Sets a value: twice is the same as once
`SetDataRules`	Sets a value: twice is the same as once
`SetDutyRules`	Sets a value: twice is the same as once
`SetErasurePolicy`	Sets a value: twice is the same as once
`SetFourEyes`	Sets a value: twice is the same as once
`SetIncidentSettings`	Sets a value: twice is the same as once
`SetInventory`	Sets a value: twice is the same as once
`SetKeepFor`	Sets a value: twice is the same as once
`SetKillSwitch`	Sets a value: twice is the same as once
`SetLabelPolicies`	Sets a value: twice is the same as once
`SetLegalHold`	Sets a value: twice is the same as once
`SetMCPClientPolicy`	Sets a value: twice is the same as once
`SetMFAPolicy`	Sets a value: twice is the same as once
`SetMailDomains`	Sets a value: twice is the same as once
`SetMailOutside`	Sets a value: twice is the same as once
`SetMailSender`	Sets a value: twice is the same as once
`SetMembership`	Sets a value: twice is the same as once
`SetMemorySettings`	Sets a value: twice is the same as once
`SetModelAlias`	Sets a value: twice is the same as once
`SetModelCanary`	Sets a value: twice is the same as once
`SetModelMapRule`	Sets a value: twice is the same as once
`SetModelVetting`	Sets a value: twice is the same as once
`SetMyNotificationAddress`	Sets a value: twice is the same as once
`SetNodeSandboxOptOut`	Sets a value: twice is the same as once
`SetNotifySettings`	Sets a value: twice is the same as once
`SetOrphanPause`	Sets a value: twice is the same as once
`SetProvider`	Sets a value: twice is the same as once
`SetPurposes`	Sets a value: twice is the same as once
`SetQualitySettings`	Sets a value: twice is the same as once
`SetQuota`	Sets a value: twice is the same as once
`SetRetention`	Sets a value: twice is the same as once
`SetRoutinePaused`	Sets a value: twice is the same as once
`SetSecret`	Sets a value: twice is the same as once
`SetSecretExpiry`	Sets a value: twice is the same as once
`SetSecretReminders`	Sets a value: twice is the same as once
`SetSetting`	Sets a value: twice is the same as once
`SetSpendAlerts`	Sets a value: twice is the same as once
`SetSubjectRule`	Sets a value: twice is the same as once
`SetTeamMember`	Sets a value: twice is the same as once
`SetTenantAsk`	Sets a value: twice is the same as once
`SetTenantBudget`	Sets a value: twice is the same as once
`SetTenantCORS`	Sets a value: twice is the same as once
`SetTenantCache`	Sets a value: twice is the same as once
`SetTenantChatSettings`	Sets a value: twice is the same as once
`SetTenantEgress`	Sets a value: twice is the same as once
`SetTenantLabels`	Sets a value: twice is the same as once
`SetTenantMaxEffort`	Sets a value: twice is the same as once
`SetTenantMoveDeadline`	Sets a value: twice is the same as once
`SetTenantOIDC`	Sets a value: twice is the same as once
`SetTenantRegion`	Sets a value: twice is the same as once
`SetTenantRemoteSandbox`	Sets a value: twice is the same as once
`SetTenantRemovedAccountKeys`	Sets a value: twice is the same as once
`SetTenantReviewWidening`	Sets a value: twice is the same as once
`SetTenantRoleRules`	Sets a value: twice is the same as once
`SetTenantSandboxIsolation`	Sets a value: twice is the same as once
`SetTenantSandboxLimits`	Sets a value: twice is the same as once
`SetTenantTemporalNamespace`	Sets a value: twice is the same as once
`SetTenantVulnerableInstalls`	Sets a value: twice is the same as once
`SetTestApprovals`	Sets a value: twice is the same as once
`SetToolPolicy`	Sets a value: twice is the same as once
`SetToolSwitches`	Sets a value: twice is the same as once
`SetWorksCouncil`	Sets a value: twice is the same as once
`TrustPluginIndex`	Sets a value: twice is the same as once
`UnbindCredential`	Sets a value: twice is the same as once
`UnblockNode`	Sets a value: twice is the same as once
`UndrainNode`	Sets a value: twice is the same as once
`UnlinkIncident`	Sets a value: twice is the same as once
`UnlinkUser`	Sets a value: twice is the same as once
`UnretireInventoryItem`	Sets a value: twice is the same as once
`UnretirePlugin`	Sets a value: twice is the same as once
`UpdateCandidate`	Sets a value: twice is the same as once
`UpdateIncident`	Sets a value: twice is the same as once
`UpgradeNodes`	Sets a value: twice is the same as once
`UpsertAccount`	Sets a value: twice is the same as once

Spend refusals

The object every budget, quota and tree-limit refusal carries, so an agent or an app can wait, ask or stop without reading the message.

Compatibility

Deployment options, platform requirements and integration maturity.

On this page