Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

caliban-operator

caliban-operator is the Kubernetes operator for caliban agent workloads. It reconciles two custom resources — a Workspace and a CalibanTask — into a sandboxed pod running caliband and its agents, composing the Kubernetes SIG agent-sandbox project rather than managing pods itself.

It is written in Rust on kube-rs.

Status: both CRDs and both reconcile loops are implemented and running in a
homelab cluster. The API is v1alpha1 and may still change without conversion
webhooks.

Where it sits

The operator owns infrastructure; prospero owns agent lifecycle. That split is deliberate and recorded in ADR 0005.

prospero ──CRUD────────▶ Workspace CR        sources, named providers,
                                             credentialsRef, egress, agentPolicy
                                             (this operator validates + reports status)

prospero ──CRUD/watch──▶ CalibanTask CR  ──reconcile──▶  agent-sandbox Sandbox pod
                         workspaceRef,                   caliband + caliban agents
                         providerRef, posture

Prospero creates and watches the custom resources; it never creates a pod. The operator never decides which agent runs or when — it decides whether a task is admissible, pins the configuration it will run under, and reconciles the sandbox that results.

The two resources

  • Workspace is the configuration plane: git sources, named model providers and their credentials, egress rules, storage settings, and the agent policy that governs what an agent in this workspace is allowed to do.

  • CalibanTask is one unit of work. It references a Workspace and (optionally) one of its named providers, and carries the task itself.

When a task is admitted, the operator resolves and pins the workspace configuration into status.resolvedWorkspace, once. Everything the sandbox is built from comes from that pinned snapshot, so editing the Workspace afterwards cannot change a running task underneath itself. That rule is ADR 0004.

Permission posture

A task declares a permissionPosture. unattended — an agent whose tool calls are never held for a human — is admitted only under a Workspace whose agentPolicy.allowUnattended is true. The workspace, not the task, is where that authority is granted, so a task cannot promote itself. The same rule governs the workspace's own unsupervised agentPolicy settings. See ADR 0006.

The effective posture is reported in status and shown as a Posture printer column, so kubectl get ctask says what a task actually got rather than what it asked for.

Where to go next

CRD reference — Workspace and CalibanTask

Both custom resources live in group caliban.caliban-ai.dev, version v1alpha1, and are namespaced with a status subresource. v1alpha1 means the shape may change without conversion webhooks (ADR 0001).

The Rust types in src/crd.rs and src/workspace.rs are the source of truth; deploy/crd/{calibantask,workspace}.yaml is generated from them and a test fails if the committed YAML drifts. See Regenerating the CRDs.

KindPluralShort namePrinter columns
WorkspaceworkspacescwsPhase, Age
CalibanTaskcalibantasksctaskPhase, Posture, Age

Runnable samples: deploy/samples/.


Workspace

Durable, shared config a CalibanTask references by name: git sources, named model providers (with their credential references), workspace-wide env, egress and agent policy. The operator is the sole reader of provider credential Secrets — a client such as prospero only ever sees the by-name reference (ADR 0004).

The Workspace controller provisions nothing. It validates the spec and writes status.

spec

FieldTypeRequiredDefaultNotes
displayNamestring (min 1)yes—Dashboard label.
sources[]listno[]Git checkouts. A workspace may be registered bare and gain sources later.
providers[]list (min 1)yes—Named providers a task can bind to.
defaultProviderstringno—Implicit when exactly one provider is defined.
env[]{name, value}no[]Non-secret env injected into every agent pod. Never put credentials here — use a provider credentialsRef.
isolationobjectno—Default isolation for agents in this workspace; a task's spec.isolation overrides it.
egressobjectno—Unset keeps allow-all egress. A task cannot override it.
agentPolicyobjectno—What agents here may do. A task cannot override it.

sources[] entries:

FieldTypeRequiredDefaultNotes
namestring (min 1)yes—Matches caliband's workspace source name.
repostring (min 1)yes—Git remote to clone.
refstringnomainBranch or tag only — git clone --depth 1 --branch rejects a raw commit SHA.
pathstring (min 1)yes—Absolute checkout path. Must be a directory strictly under the operator's workspace root (default /work) and distinct from every other source's path, or the Workspace goes Failed.

providers[] entries:

FieldTypeRequiredNotes
namestring (min 1)yesUnique within the workspace.
kindstring (min 1)yesanthropic, openai, google/gemini take their exact env names from caliban-contract. Other kinds (azure, bedrock, vertex, …) get <KIND>_BASE_URL / <KIND>_API_KEY, with azure/azure-openai mapped to AZURE_OPENAI_*.
baseUrlstringnoProjected under the kind's native base-URL env var.
modelstringnoRecorded in the pin. Not projected as env — the model reaches the worker through caliband's SpawnSpec.
credentialsRef{secretName, key}noOmit for keyless providers (e.g. a local openai-compatible endpoint). Reaches the pod as a secretKeyRef; the operator never inlines the value.

Base URL and API key are projected under provider-native env names, because caliban implements no CALIBAN_* namespace for them — it reads OPENAI_BASE_URL, ANTHROPIC_API_KEY and so on. The kind is additionally projected as CALIBAN_PROVIDER, not as a provider selector (selection travels in caliband's SpawnSpec) but because it is the documented input to a user-configured apiKeyHelper.

egress restricts agent-pod egress to DNS plus the listed destinations, replacing the default allow-all rule. An empty allow list is DNS-only.

FieldTypeRequiredDefaultNotes
egress.allow[].cidrstring (min 1)yes—IPv4 or IPv6 CIDR, e.g. 192.0.2.0/24, 2001:db8::/32. A malformed CIDR or a prefix longer than the address fails the Workspace.
egress.allow[].ports[]list of intno[]TCP ports. Empty allows every port. Anything outside 1-65535 fails the Workspace.

When you set egress, remember to include the workspace's git remotes, its model provider endpoints, and gonzalod if a task uses spec.state.

agentPolicy — what agents in this workspace may do

Typed settings for caliban's own permission and extension surface, so a cluster author configures them as CRD fields rather than as raw env entries. They win over an env entry of the same name. Every field is optional, and an unset field leaves caliban's own default alone — which is not the same as setting it to false.

The whole block is pinned into status.resolvedWorkspace.agentPolicy with the rest of the Workspace, so it cannot change under a running task.

The authorization switch:

FieldTypeDefaultNotes
allowUnattendedboolfalseAuthorizes both a task asking for permissionPosture: unattended and the unsupervised settings in the next table. Whoever can edit this Workspace controls it — treat Workspace write access as privileged (ADR 0006).

Permission settings. The three marked ⚠ leave no human answering prompts, so each is admitted only when allowUnattended: true — otherwise the task is denied before pinning with reason PostureNotPermitted, and the message names the setting. One switch is the whole answer to "may agents here run unsupervised?", so the same power cannot be reached through a back door (ADR 0006, decision 7).

FieldTypeNotes
permissionModeenumdefault, acceptEdits, plan, auto, ⚠ dontAsk, ⚠ bypassPermissions. Projected as CALIBAN_DEFAULT_PERMISSION_MODE; caliban's own --permission-mode flag still wins over it. dontAsk turns every Ask into Allow by caliban's definition, so it grants what bypassPermissions does; auto still classifies each call, so it does not count as unsupervised.
autoAllowbool⚠ Allow tools that would otherwise prompt. CALIBAN_AUTO_ALLOW.
noPermissionsbool⚠ Disable permission gating altogether — the bluntest lever. CALIBAN_NO_PERMISSIONS.

Extension settings. These reduce what an agent can reach rather than granting it power, so none of them needs allowUnattended.

FieldTypeNotes
noMcpboolNo MCP server discovery. CALIBAN_NO_MCP.
noHooksboolBypass every external hook handler; caliban's in-process hooks still run. CALIBAN_NO_HOOKS.
noSkillsboolNo skill discovery at startup. CALIBAN_NO_SKILLS.
noSubAgentboolDisable the built-in agent tool, so an agent cannot spawn sub-agents. CALIBAN_NO_SUB_AGENT.
strictKnownMarketplacesboolBlock plugin installs from marketplaces caliban does not recognise. CALIBAN_STRICT_KNOWN_MARKETPLACES.
enabledPlugins[]list of stringAn allow-list: every other discovered plugin is disabled. CALIBAN_ENABLED_PLUGINS, comma-joined. An empty list enables none, which is deliberately different from omitting the field — omitted enables everything caliban discovers, so the two cannot be collapsed.
blockedMarketplaces[]list of stringMarketplace names to block; unset blocks none. CALIBAN_BLOCKED_MARKETPLACES, comma-joined.
parallelToolLimitinteger ≥ 1Maximum concurrent tool invocations per turn. CALIBAN_PARALLEL_TOOL_LIMIT. The schema enforces minimum: 1 because caliban refuses 0 at startup; unset leaves caliban's default of one less than the CPU count.

Unlike a task's own posture, this policy is checked at admission only. It travels inside the pin, so it cannot change under a running task — and revoking a setting therefore does not disturb a task already running.

status

FieldNotes
phasePending (not yet reconciled) → Ready (all providers and credential Secrets resolve) or Failed.
messageWhy it failed, e.g. provider 'planner': secret 'anthropic-key' key 'api-key' not found.
observedGenerationThe .metadata.generation this status reflects.
conditions[]Standard Kubernetes conditions.

Validation runs in this order, first problem wins: source paths (under root, distinct) → egress CIDRs and ports → duplicate provider names → a resolvable defaultProvider → an existing Secret key for every credentialsRef.

The controller deliberately does not watch Secrets — that would need cluster-wide list/watch on every Secret. Recovery rides the requeue instead: a Ready workspace is re-checked every 300s, a Pending or Failed one every 15s, so creating a missing Secret unblocks it within seconds.


CalibanTask

One agent session: a reference to a Workspace, the prompt, and per-run overrides. The operator reconciles it into an agent-sandbox Sandbox, a token-less ServiceAccount and a NetworkPolicy (ADR 0002).

spec

FieldTypeRequiredDefaultConsumed by
workspaceRef.namestring (min 1)yes—operator
providerRefstringnothe workspace's defaultProvider, else its sole provideroperator
task.promptstring (min 1)yes—prospero
task.agentTypestringno—carried on the CR; the operator does not read it
task.interactiveboolnofalseprospero (SpawnSpec.interactive); the operator only carries it
task.permissionPostureenumnosupervisedoperator (admission gate) + prospero
model.namestring (min 1)nothe resolved provider's modeloperator (written into the pin)
model.routerConfigRefstringno—operator
state.gonzaloEndpointstringno—operator
state.modeenum: remote | localnoremote when gonzaloEndpoint is setoperator
state.tokenRef{secretName, key}no—operator
isolation.runtimeClassstringnothe workspace defaultoperator (pod runtimeClassName)
isolation.worktreesstringno—declared, not yet consumed
resources.classstringno—declared; SandboxTemplate selection is deferred
lifecycle.idleTimeoutstringno—declared; idle→pause mapping is deferred
lifecycle.onDeletestring: checkpoint | deleteno—declared; the drain finalizer is deferred
tools[]list of stringno—declared, not yet consumed

Which closed sets the API server enforces. task.permissionPosture, state.mode and the Workspace's agentPolicy.permissionMode carry an enum in the generated schema, so a typo in any of them is rejected at admission.

state.mode is a deliberate hybrid: the enum lives in the schema while the Rust type stays a String. A task stored before the enum existed may hold any value, and a typed enum would make such an object fail to deserialize — the operator could not read it at all. So storage::state_problem still rejects an unknown mode at reconcile time (InvalidState), which is what those pre-existing objects get.

The strings in isolation.worktrees, resources.class and lifecycle.* carry no schema enum and are unread. For lifecycle.onDelete that is deliberate rather than an oversight: #42 implements it (drain finalizer → Draining phase → status.checkpointRef, designed in caliban ADR 0057) and waits only on prospero reporting AgentsDrained, so pinning the accepted values now would freeze a surface that work may still refine.

model.name — per-task model override

A task may name its own model without the Workspace binding a whole new provider for it. The override is applied to the pinned resolution, so:

  • the bound provider's endpoint and credential still apply — only the model moves;
  • it cannot change under a running task, like everything else in the pin (ADR 0004);
  • prospero needs no change, since it already builds SpawnSpec.model from the pinned provider.

It is applied after admission, because which model to run is not an admissibility question. A blank name is treated as a misconfiguration rather than a selection: it leaves the provider's model alone instead of clearing it, and the schema's minLength: 1 rejects it outright. model.name is independent of model.routerConfigRef and may be set alongside it.

spec.isolation is a per-run override that wins over the workspace default. That is exactly why egress and agentPolicy are Workspace-level fields and not part of the shared isolation block — a task must not be able to widen a restriction the workspace imposed.

permissionPosture

ValueMeaning
supervisedA permission prompt goes to a human, who answers it. The default, and what an unset field means — the system fails closed.
unattendedThe session runs under an authorized bypass profile with no human in the loop.

unattended is admitted only when the referenced Workspace sets agentPolicy.allowUnattended: true. A missing policy — including a pin made before the field existed — does not permit it. The check runs before pinning, so a denied task is never pinned to the policy that denied it; allowing it on the Workspace, or editing the task back to supervised, recovers it. The operator never silently downgrades a posture.

The same switch also authorizes the Workspace's own unsupervised agentPolicy settings, so a task is denied with the same PostureNotPermitted reason whether the unsupervised power was asked for by the task or by the workspace — see agentPolicy.

A pinned task is re-checked against its pin on every reconcile. Revoking allowUnattended therefore does not disturb a task already running, but editing a running task's spec to unattended fails it if the pin does not allow it.

The operator does not hand the posture to caliband. prospero maps it onto SpawnSpec.permission_posture, reading status.permissionPosture — the posture the operator admitted — not the spec field.

state (gonzalo persistence)

spec.state becomes caliban's storage settings by environment (CALIBAN_STORAGE_SUBSTRATE, CALIBAN_STORAGE_REMOTE_URL, CALIBAN_STORAGE_REMOTE_TOKEN_ENV), so the operator never authors caliban's settings file. The CRD's local is caliban's fs. Requires a caliban build with environment overrides for those settings.

A tokenRef projects the gonzalod bearer token into the pod as GONZALO_TOKEN from the named Secret; only the variable's name appears in the pod spec. Use a principal with read/write on the gonzalod caliban namespace only — no write on fleet or fleet-audit.

These combinations fail the task with reason InvalidState: an unknown mode; mode: remote without gonzaloEndpoint; mode: local with a gonzaloEndpoint it would never use; a tokenRef without remote storage; a tokenRef whose Secret or key is missing.

status

FieldOwnerNotes
phaseoperatorSee below.
calibandEndpointoperator<serviceFQDN>:<calibandPort>, set only once the Sandbox is Ready.
sandboxRef.nameoperator<task>-sbx.
resolvedWorkspaceoperatorThe pinned config (sources, the single resolved provider, env, isolation, egress, agentPolicy). Set once at admission; later Workspace edits do not re-pin a running task.
permissionPostureoperatorThe posture the task was admitted with. The only posture prospero acts on.
conditions[]splitA map-list keyed by type. The operator owns Ready; prospero owns AgentsSettled. Each applies only its own entries under its own field manager, so neither clobbers the other (ADR 0005).
checkpointRef—Declared; checkpointing is deferred.

The operator writes status with server-side apply under field manager caliban-operator. A field it owns but omits is removed, so a cleared value is omitted rather than serialized as null.

Phases

PhaseWhen
PendingObserved before its first successful apply — no Sandbox yet.
ProvisioningThe Sandbox is applied but has not reported Ready.
Runningagent-sandbox reports the Sandbox Ready=True — the pod passed its readiness probe on caliband's control port and the Service is ready.
DrainingDeclared; not yet produced (the drain/checkpoint lifecycle is deferred).
Completedprospero's AgentsSettled=True with reason Succeeded.
Failedprospero's AgentsSettled=True with reason Failed, or a terminal operator-side failure (below).

A serviceFQDN alone is not readiness: it appears as soon as the Service object exists, before caliband has bound its port. Running gates on agent-sandbox's own Ready condition instead.

Terminal phases come from prospero, because infrastructure cannot tell "agent running" from "agent finished, pod still up" — only caliband's agent list knows, and the operator is never a caliband client. Without prospero deployed, a finished task stays Running.

Ready condition reasons (operator-owned)

reasonstatusMeaning and recovery
RunningTruecaliband is up and attachable.
SandboxNotReadyFalseagent-sandbox reported not-ready; message carries its detail (e.g. "Pod is Running but not Ready").
CompletedFalseAgents finished successfully; the task is done, so not Ready.
AgentsFailedFalseOne or more agents failed or crashed.
InvalidNameFalseThe task name is too long for its Sandbox's Service name (<task>-sbx, max 63 chars). Names are immutable — recreate under a shorter name. The operator stops retrying.
WorkspaceUnresolvedFalseNo such Workspace, a Failed one, or an unresolvable/ambiguous providerRef. message names which. Re-checked every 30s; fixing the Workspace recovers the task.
InvalidStateFalsespec.state is malformed or its token Secret is missing. Re-checked every 30s.
PostureNotPermittedFalseUnsupervised power without agentPolicy.allowUnattended — either the task's permissionPosture: unattended or the Workspace's own permissionMode: dontAsk/bypassPermissions, autoAllow, noPermissions. message names which setting. Re-checked every 30s.

AgentsSettled reasons are prospero's: Succeeded, Failed, or AgentsActive (not settled — an idle interactive task stays unsettled).

A task whose Workspace exists but is still Pending is not failed: it is not pinned either, and the reconcile simply requeues in 10s to re-check once the Workspace controller has validated it.

Events

kubectl describe calibantask <name> shows what the logs would otherwise hold. Notes are truncated at 1 kB; publishing is best-effort and never fails or delays a reconcile.

ReasonTypeWhen
PhaseChangedNormalThe phase moves, e.g. Provisioning → Running.
UnattendedAdmittedNormalA task was admitted unattended; the note names the Workspace whose policy allowed it.
ReconcileErrorWarningA reconcile returned an error.
InvalidName / WorkspaceUnresolved / InvalidState / PostureNotPermittedWarningThe task was marked Failed; the Event reason is the failure's own reason code.

What one CalibanTask creates

ObjectNameNotes
Sandbox (agents.x-k8s.io/v1beta1)<task>-sbxThe caliband pod, its workspace PVC, and a headless Service giving the stable serviceFQDN.
ServiceAccount<task>-saautomountServiceAccountToken: false and no Role/RoleBinding — the pod holds zero API permissions. The token-less SA is the RBAC posture.
NetworkPolicy<task>-netpolpolicyTypes: [Ingress, Egress], selecting the pod by the labels the operator propagates.

All three carry a controller OwnerReference to the task, so deleting the task cascades, and are applied with server-side apply so re-reconciles converge on one object. Deleting a task tears its Sandbox down immediately, without checkpointing in-flight agents (the drain finalizer is deferred).

The pod runs every container — the git-clone init container included — as one uid/gid (default 10001, caliban's app user) with runAsNonRoot: true and fsGroup set to the same gid, so the clone step and the agent cannot drift apart and a non-root clone succeeds whatever mode the volume provisioner gave the volume root. See Deploying for the one case this does not repair.

The reconcile re-runs on every change to the task and to the Sandboxes it owns, so a Sandbox becoming Ready reconciles the task at once. The 300s periodic resync is only a safety net against missed events.


Regenerating the CRDs

The committed YAML is generated, never hand-edited. A *_is_in_sync test fails if it drifts from the Rust types:

cargo run --bin crdgen calibantask > deploy/crd/calibantask.yaml
cargo run --bin crdgen workspace   > deploy/crd/workspace.yaml
cargo test --workspace

The Sandbox type in src/sandbox.rs is a read/write view of a CRD agent-sandbox owns, not a schema of record. crdgen never emits it and no test drift-guards it; fields the operator omits are pruned by the API server's structural schema.

After a CRD change, the copies in the caliban-crds Helm chart need the same update — that chart names deploy/crd/*.yaml here as its source of truth.

Deploying and configuring caliban-operator

The operator is a controller: it watches Workspace and CalibanTask custom resources cluster-wide and creates Sandbox, ServiceAccount and NetworkPolicy objects. It serves no inbound traffic, so it has no Service, Ingress or port.

This page is the reference for the operator's own configuration surface and the cluster prerequisites it assumes. The Helm charts themselves live in caliban-ai/helm-charts; for chart values, see that repo.

Prerequisites

  1. agent-sandbox — the agents.x-k8s.io/v1beta1 Sandbox CRD and its controller must be present. This is a cluster-level prerequisite, like a CNI or a RuntimeClass, normally installed by the cluster admin. The operator's Sandbox view is written against agent-sandbox v0.5.0; helm-charts vendors that version as charts/agent-sandbox and the umbrella bundles it by default. Without it, every reconcile's Sandbox apply fails and tasks stay Provisioning — a retried error, not a crash.
  2. The caliban CRDs — calibantasks and workspaces, installed as their own step from the caliban-crds chart (Helm does not upgrade a crds/ directory, so that chart keeps them in templates/). They are copies of deploy/crd/ here, which is the source of truth. Applying them directly works too:
    kubectl apply -f deploy/crd/
    
    Upgrade the CRDs before the operator. Against an older CRD, a field the operator relies on — spec.task.permissionPosture, spec.agentPolicy, spec.model.name — is either rejected outright (kubectl apply decodes strictly: unknown field "spec.task.permissionPosture") or silently pruned by the structural schema, so an unattended task would read back as supervised.
  3. Lease RBAC, before the operator image — leader election is on by default, so the operator needs coordination.k8s.io/leases in its own namespace. The chart's lease Role is harmless against an older operator, so upgrading the chart first is always safe; the reverse order is not. See Leader election.
  4. A session-plane TLS Secret and token Secret — see Session plane below.
  5. A default StorageClass — each task's workspace PVC is created with no storageClassName, so it binds the cluster default (deliberately cluster-agnostic).

Kubernetes versions

The operator compiles against the v1_32 k8s-openapi API surface (the lowest the kube 4.0 / k8s-openapi 0.28 pairing offers). It uses only long-stable PodSpec, NetworkPolicy and apiextensions.k8s.io/v1 fields, so it is forward-compatible across the usual skew. The original target was k3s v1.31 (ADR 0001); it currently runs on k3s v1.36. No chart declares a kubeVersion floor, so treat this as a record of what has been run, not an enforced constraint.

NetworkPolicy is only enforced if the cluster's CNI implements it. On a CNI that ignores it, the per-task policy is applied and inert — the isolation posture is silently absent.

Installing

CRDs first, then the operator. Via the charts (caliban-ai/helm-charts, published to the OCI registry oci://ghcr.io/caliban-ai/charts, not an HTTP chart repo — there is no helm repo add):

helm install caliban-crds     oci://ghcr.io/caliban-ai/charts/caliban-crds
helm install caliban-operator oci://ghcr.io/caliban-ai/charts/caliban-operator

(If the registry refuses an anonymous pull, the GHCR package is still private — helm registry login ghcr.io first.)

Or install the caliban-system umbrella chart, which can bring agent-sandbox, gonzalo and prospero with it. Either way, keep environment-specific values in a separate private overlay (-f private-values.yaml); the charts ship neutral, cluster-agnostic defaults only.

Then confirm it is up and try a task:

kubectl get deploy -l app.kubernetes.io/name=caliban-operator   # 1/1 available
kubectl apply -f deploy/samples/workspace.yaml
kubectl get workspaces          # wait for Phase: Ready
kubectl apply -f deploy/samples/calibantask.yaml
kubectl get calibantasks -A     # Phase, Posture

The samples target namespace team-a and reference Secrets that do not exist by default, so the Workspace will report Failed with the missing Secret named in status.message until you create them. That is the intended first signal.

Configuration

Every setting is an environment variable on the controller container; there are no flags and no config file. Unset means the default below.

Images and the agent pod

VariableDefaultNotes
CALIBAND_IMAGEghcr.io/caliban-ai/caliban:latestThe caliband image run in each Sandbox. Pin a real tag in production — latest makes an agent pod's version unreproducible.
CALIBAN_GIT_IMAGEalpine/git:latestThe git-clone init container that populates the workspace.
CALIBAN_WORKSPACE_ROOT/workWorkspace mount path in the pod. Every Workspace source path must sit strictly under it.
CALIBAN_WORKSPACE_STORAGE10GiRequested size of each task's workspace PVC.
CALIBAN_AGENT_UID10001uid every container in the sandbox pod runs as.
CALIBAN_AGENT_GID10001gid for the same, and the pod's fsGroup.

Note what is not here: caliban's own permission and extension behaviour is not operator configuration. It is set per workspace, as typed Workspace.spec.agentPolicy fields, which the operator projects into each agent pod as CALIBAN_* environment (CALIBAN_DEFAULT_PERMISSION_MODE, CALIBAN_AUTO_ALLOW, CALIBAN_NO_PERMISSIONS, CALIBAN_NO_MCP, CALIBAN_NO_HOOKS, CALIBAN_NO_SKILLS, CALIBAN_NO_SUB_AGENT, CALIBAN_STRICT_KNOWN_MARKETPLACES, CALIBAN_ENABLED_PLUGINS, CALIBAN_BLOCKED_MARKETPLACES, CALIBAN_PARALLEL_TOOL_LIMIT). Those values win over a Workspace.spec.env entry of the same name, and the three unsupervised ones are gated by agentPolicy.allowUnattended. There is deliberately no cluster-wide default for them: the authority is write access to a Workspace.

Ports

VariableDefaultNotes
CALIBAND_PORT8443caliband's TCP+TLS control port; also the readiness/startup probe target and the ingress port the NetworkPolicy opens.
CALIBAN_AGENT_PORT_BASE7100Bottom of caliband's per-agent stream port window. Passed to caliband and opened in the NetworkPolicy from one setting, so the two cannot drift.
CALIBAN_AGENT_PORT_END7999Top of that window, inclusive.

The operator validates these at startup and refuses to start on a port outside 1-65535, an empty window (BASE > END), or a CALIBAND_PORT that falls inside the agent window. The error names the offending variable, rather than letting a NetworkPolicy the API server rejects fail every reconcile.

Session plane

caliband serves TLS and requires a bearer token. Both come from pre-existing Secrets in the task's namespace, mounted or projected by the operator.

VariableDefaultNotes
CALIBAN_SESSION_TLS_SECRETcaliban-session-plane-tlsServing-cert Secret with keys tls.crt, tls.key, ca.crt. Mounted read-only at /etc/caliband/tls.
CALIBAN_SESSION_TOKEN_SECRETcaliban-session-plane-tokenBearer-token Secret.
CALIBAN_SESSION_TOKEN_KEYtokenKey within the token Secret.
CALIBAN_SESSION_SERVER_NAMEcalibandThe name the serving cert is verified against. Must equal that cert's SAN and the client's TLS SNI, or the workers caliband spawns fail the handshake and their status reports are silently dropped.

The token is never an argv value — it reaches the pod from the Secret as an environment variable, so it stays out of kubectl get pod -o yaml and the process table. The ca.crt is handed to caliband so it can pass a trust anchor down to the workers it spawns.

Agent pod resources

Defaults give every agent pod Burstable QoS: requests, no limits — so a pod is not first in line for eviction and is admitted under a ResourceQuota, without a guessed ceiling OOM-killing an agent. An explicitly empty value clears a default rather than only overriding it.

VariableDefaultNotes
CALIBAND_CPU_REQUEST250m
CALIBAND_MEMORY_REQUEST512Mi
CALIBAND_CPU_LIMITunset
CALIBAND_MEMORY_LIMITunset

The git-clone init container gets the same values. A pod's effective request is the max of any init container and the sum of its app containers, so this costs nothing at scheduling while still admitting the init step under quota.

Cluster topology and network reach

VariableDefaultNotes
CALIBAN_CLUSTER_DOMAINcluster.localThe cluster's DNS domain, used to build caliband's advertise host <sandbox>.<namespace>.svc.<domain>. A blank value keeps the default; a trailing root dot is trimmed.
CALIBAN_INGRESS_POD_SELECTORemptykey=value,key2=value2. Narrows the NetworkPolicy's ingress peer to matching pods. Empty admits every pod in the peer namespace. A malformed entry fails startup rather than silently widening or dropping the rule.
CALIBAN_INGRESS_NAMESPACEthe task's ownNamespace allowed to reach caliband, matched on its kubernetes.io/metadata.name label.

Unconfigured, ingress to caliband is open to every pod in the task's own namespace — the historical default-deny-everything-else posture. In a real deployment, narrow it to prospero: set CALIBAN_INGRESS_NAMESPACE to prospero's namespace and CALIBAN_INGRESS_POD_SELECTOR to its pod labels.

Observability

VariableDefaultNotes
RUST_LOGinfotracing-subscriber env filter. An unset value yields info, not silence.
CONTROLLER_POD_NAMEunsetRecorded as the instance on every Event the operator publishes. Set it from fieldRef: metadata.name so Events name the replica that wrote them.

RBAC the operator needs

The operator needs a cluster-scoped role — it watches both CRs across all namespaces. This is the verb set the charts grant:

apiGroupsresourcesverbs
caliban.caliban-ai.devcalibantasksget, list, watch, update, patch
caliban.caliban-ai.devcalibantasks/statusget, update, patch
caliban.caliban-ai.devworkspacesget, list, watch
caliban.caliban-ai.devworkspaces/statusget, update, patch
""secretsget
agents.x-k8s.iosandboxesget, list, watch, create, update, patch, delete
""serviceaccountsget, list, watch, create, update, patch, delete
networking.k8s.ionetworkpoliciesget, list, watch, create, update, patch, delete
events.k8s.ioeventscreate, patch
coordination.k8s.ioleasesget, list, watch, create, update, patch, delete

The leases grant is required by default, because leader election is on by default — see Leader election. It only needs to be a namespaced Role in the operator's own namespace, since the lease lives there.

Notes on the shape of that set:

  • get on secrets, never list or watch. The operator reads a Secret only by the name a Workspace references, to answer "does this key exist". It never lists or caches Secret data, which is why the Workspace controller re-checks on a requeue instead of watching Secrets.
  • No create/delete on calibantasks or workspaces. Clients such as prospero own CR lifecycle; the operator only observes them and writes status.
  • events.k8s.io is required for the Events in the CRD reference. Without it, publishing logs a warning and the reconcile proceeds — Events are best-effort.
  • prospero needs patch on calibantasks/status of its own to write the AgentsSettled condition. Without it, tasks never reach a terminal phase. That grant lives in prospero's chart, not the operator's.

Leader election

The operator elects a leader through a coordination.k8s.io Lease before running either controller, and it is on by default. A single replica simply acquires the lease immediately; the point of the default is that raising replicaCount can never silently leave two controllers force-applying the same objects.

VariableDefaultNotes
CALIBAN_LEADER_ELECTIONtrueAccepts 1/true/yes/on and 0/false/no/off; anything else fails startup rather than being guessed. Set it false only for a single-replica install that manages exclusivity itself.
CALIBAN_LEASE_NAMEcaliban-operatorThe Lease object the replicas contend for. Must not be blank.
CALIBAN_LEASE_NAMESPACEthe operator's own namespaceRead from /var/run/secrets/kubernetes.io/serviceaccount/namespace when unset. The chart sets it explicitly to the release namespace.
CALIBAN_LEASE_DURATION_SECONDS15How long the lease is held before it may be taken over.
CALIBAN_LEASE_GRACE_SECONDS5How long before expiry the holder starts renewing. Must be below the duration, or the holder could lose the lease while still believing it holds it — startup rejects that.
POD_NAMEthe pod's HOSTNAME, else caliban-operatorThe identity this replica claims the lease under. It must differ per replica, or two replicas look like the same holder and both proceed. The chart sets it from fieldRef: metadata.name; the hostname fallback only matters outside a cluster. Note this is a different variable from CONTROLLER_POD_NAME, which only labels Events.

Operationally:

  • Losing the lease stops the controllers and returns the process to standby rather than exiting, so an apiserver blip costs a pause instead of a crash loop. A replica that never wins simply waits, doing nothing.
  • A clean shutdown releases the lease rather than waiting out the duration, so failover is prompt.
  • Outside a cluster, there is no namespace file: either set CALIBAN_LEASE_NAMESPACE or CALIBAN_LEADER_ELECTION=false. The startup error says exactly this.
  • The lease is applied under field manager caliban-operator, like everything else the operator writes.

Running with CALIBAN_LEADER_ELECTION on but without the leases RBAC above is the one new way a previously-working install can break on upgrade: the operator starts, fails to acquire, and never reconciles. Upgrade the chart before the operator image — the chart's lease Role is harmless against an operator that predates leader election (an unused Role and env it ignores), whereas the reverse order fails on lease RBAC. If you keep election off, keep replicaCount: 1: with election off, nothing stops two replicas from force-applying the same objects.

In the chart this is the leaderElection block (enabled, leaseName, durationSeconds, graceSeconds), on by default. Its lease Role is gated behind both leaderElection.enabled and rbac.create, so rbac.create=false with election on is the combination to watch.

The operator's own Deployment should run as uid 10001 with runAsNonRoot: true, allowPrivilegeEscalation: false, readOnlyRootFilesystem: true and all capabilities dropped. It writes nothing to disk, so a read-only root filesystem is safe. See the container image.

Pod identity and the workspace volume

Every container in a sandbox pod — the git-clone init container included — runs as CALIBAN_AGENT_UID/CALIBAN_AGENT_GID (default 10001, matching the app user in caliban's image), with runAsNonRoot: true and fsGroup set to the same gid.

The uid sits on the pod, not on each container, so there is nothing to keep in sync between the clone step and the agent. fsGroup is what makes a non-root clone work on any volume rather than by luck: kubelet applies the group to the mounted workspace with g+rwX, so the clone does not depend on the mode the provisioner gave the volume root. runAsNonRoot turns a misconfigured uid 0 into an admission failure instead of a silent return to root-owned sources.

CALIBAN_AGENT_UID must match the caliband image's user. caliban's image bakes HOME and an XDG_RUNTIME_DIR owned by uid 10001; a different uid cannot write them.

One case this does not repair: the clone is idempotent and skips a source whose <path>/.git already exists, so a workspace volume cloned by an older, root-running init container still holds root-owned sources. The init container is now non-root and cannot chown them back, and git compares the owner uid, so it refuses the checkout even though fsGroup made the files writable. Recreate the task, or delete its workspace PVC, to re-clone as the agent.

Troubleshooting

Start with the task's own status and Events — the failure reason is always on the object:

kubectl get calibantask <name> -o jsonpath='{.status.phase} {.status.permissionPosture}{"\n"}'
kubectl describe calibantask <name>      # conditions + Events
kubectl get workspace <name> -o jsonpath='{.status.phase}: {.status.message}{"\n"}'
SymptomLikely cause
Operator pod CrashLoopBackOff immediately, log names a variableStartup validation rejected a port, uid or lease setting. Fix the named variable.
Operator runs but reconciles nothing, no errors on any taskIt is a non-leader standby, or it cannot acquire the lease. Check the log for "contending for the leader lease" / "acquired the leader lease", then kubectl get lease caliban-operator. A forbidden on leases means the lease RBAC is missing.
Two replicas both reconcilingCALIBAN_LEADER_ELECTION=false with replicaCount > 1, or both replicas sharing one POD_NAME so they look like the same lease holder.
Operator fails at startup naming CALIBAN_LEASE_NAMESPACERunning outside a cluster, where the ServiceAccount namespace file does not exist. Set that variable or CALIBAN_LEADER_ELECTION=false.
Workspace stuck FailedRead status.message — a source path outside the workspace root, a duplicate path, a bad egress CIDR, a duplicate provider name, an unresolvable defaultProvider, or a missing credential Secret key.
Task Failed, WorkspaceUnresolvedThe Workspace is missing or Failed, or providerRef names no provider. Re-checked every 30s.
Task Failed, PostureNotPermittedUnsupervised power without agentPolicy.allowUnattended. message names the setting — the task's permissionPosture: unattended, or the Workspace's own permissionMode: dontAsk/bypassPermissions, autoAllow or noPermissions.
kubectl apply rejects unknown field "spec.agentPolicy.permissionMode" (or spec.model.name)The installed CRDs predate the field. Upgrade the caliban-crds chart.
An agentPolicy setting seems ignoredIt was unset, not false — unset leaves caliban's own default. Also check the task's pin: status.resolvedWorkspace.agentPolicy is what the running task was admitted with, and editing the Workspace does not re-pin it.
A task runs the wrong modelspec.model.name is pinned at admission. Read status.resolvedWorkspace.provider.model; editing the task afterwards does not move a running one.
Task Failed, InvalidNameThe task name is too long for its <task>-sbx Service. Names are immutable — recreate with a shorter one.
Task stuck PendingNo Sandbox was created. Check the operator's RBAC on sandboxes.agents.x-k8s.io and that agent-sandbox is installed.
Task stuck ProvisioningThe Sandbox exists but is not Ready. kubectl describe sandbox <task>-sbx and check the pod: a pull failure on the caliband or git image, an unschedulable PVC, or caliband not binding its control port.
Task reaches Running and never leavesExpected without prospero — the operator cannot observe agent completion. Check prospero is deployed and holds patch on calibantasks/status.
Agents report nothing; interactive agents never go idleA session-plane TLS mismatch. CALIBAN_SESSION_SERVER_NAME must equal the serving cert's SAN.
Clone fails with "detected dubious ownership"A pre-existing workspace volume with root-owned sources. Delete the PVC and recreate the task.
Agent cannot reach a git remote or a providerThe Workspace declares spec.egress and the destination is not in the allow-list. Remember DNS is always allowed but the destinations are not.

caliban-operator container image

The operator ships as a multi-arch container image at ghcr.io/caliban-ai/caliban-operator (linux/amd64 + linux/arm64).

What's in it

A two-stage build (Dockerfile):

  • builder — rust:1-bookworm, cargo build --release --bin caliban-operator. The exact toolchain comes from rust-toolchain.toml, which rustup reads from the build context, so the version is written in one place only (#90). (rustls/ring + kube; no openssl/protoc/git2 native deps).
  • runtime — debian:bookworm-slim + ca-certificates, running the caliban-operator controller binary as a non-root user (uid 10001, matching the Helm chart's securityContext). The operator writes nothing to disk, so it runs fine under a read-only root filesystem.

The image contains only the controller (caliban-operator), not the crdgen dev tool. ENTRYPOINT is the binary and it takes no arguments.

Configuration is entirely by environment — there are no flags and no config file. See Deploying and configuring for the full set with defaults; the operator refuses to start on an invalid port window, selector or uid, naming the offending variable. The Helm chart in caliban-ai/helm-charts wires a subset of them and leaves the rest on their compiled defaults.

Build locally

docker build -t caliban-operator:dev .

Publishing (CI)

.github/workflows/release-image.yml mirrors the sibling repos' native per-arch pipeline (no QEMU):

  • Pull requests touching the build inputs build both arches on native runners (amd64 on ubuntu-latest, arm64 on ubuntu-24.04-arm) with no push — pure validation.
  • v* tags (and workflow_dispatch) build each arch, push it by digest, then a merge job assembles the multi-arch manifest list and pushes the {{version}} and sha-<sha> tags via docker buildx imagetools create.

Pin a version tag in a deployment rather than tracking a floating one, so an operator rollout is reproducible.

Cut a release by tagging:

git tag v0.6.1 && git push origin v0.6.1

Afterwards, bump appVersion in the caliban-operator Helm chart to match, and copy any changed deploy/crd/*.yaml into the caliban-crds chart.

Changelog

All notable changes to caliban-operator are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning. While the project is pre-1.0, the minor version is bumped for new features and the patch version for fixes.

The operator is not published to crates.io (publish = false). A release is a container image: tagging a merge commit v* fires release-image.yml, which publishes ghcr.io/caliban-ai/caliban-operator:<version> for linux/amd64 and linux/arm64.

Entries before v0.6.0 were back-filled from the release: commit messages that were, until this file existed, the only record of what each tag contained. They are therefore summaries at the granularity those commits recorded, not the per-change detail used from v0.6.0 onwards.

Unreleased

0.7.0 - 2026-10-06

The release that makes a second replica safe, and moves the agent's posture out of untyped env and into the API.

Leader election is on by default, so replicas: 2 is a standby rather than a race. A Workspace now declares its agent policy as typed fields the operator validates and projects — and the settings that leave an agent unsupervised are admitted only where the Workspace authorizes them, extending ADR 0006's rule to the policy itself. A task can override just the model. The sandbox pod runs as the agent's own uid, so clones it writes are usable.

Needs caliban-crds >= 0.2.9 for the agentPolicy fields, model.name and the
state.mode enum, and the Lease RBAC from caliban-operator chart >= 0.2.11.
Upgrading begins enforcing the unsupervised-policy gate: a Workspace that sets
permissionMode dontAsk/bypassPermissions or noPermissions without
agentPolicy.allowUnattended is refused, fail-closed, where v0.6.0 ignored it.

Added

  • Leader election, on by default. The controller acquires a coordination.k8s.io Lease before running either reconciler and returns to standby if it loses it, so running more than one replica is safe rather than a race between two writers. Configured by CALIBAN_LEADER_ELECTION (default on), CALIBAN_LEASE_NAME, CALIBAN_LEASE_NAMESPACE, CALIBAN_LEASE_DURATION_SECONDS and CALIBAN_LEASE_GRACE_SECONDS; the holder identity comes from POD_NAME. Needs the Lease RBAC added in caliban-charts (#48, PR #96)

  • Typed agent permission posture on Workspace. spec.agentPolicy gained permissionMode, autoAllow and noPermissions, projected into the agent's environment instead of travelling as untyped env entries. The settings that leave an agent unsupervised — permissionMode: dontAsk or bypassPermissions, noPermissions — are admitted only on a Workspace whose agentPolicy.allowUnattended is true, which extends ADR 0006's existing authorization rule to the Workspace's own policy (recorded as decision 7 in that ADR) (#51, PR #88)

  • Typed agent extension posture on Workspace. spec.agentPolicy also gained noMcp, noHooks, noSkills, noSubAgent, strictKnownMarketplaces, enabledPlugins, blockedMarketplaces and parallelToolLimit. Typed fields take precedence over any same-named env entry, and list fields project even when empty — an empty enabledPlugins means enable none, which is a different instruction from leaving it unset (#87, PR #89)

  • Per-task model override. CalibanTask.spec.model.name overrides the resolved workspace provider's model for one task, applied at the resolve-and-pin step so the override is recorded in status.resolvedWorkspace like every other pinned value (#52, PR #91)

Fixed

  • The sandbox pod runs as the agent's uid. The pod spec now carries a securityContext with runAsUser/runAsGroup/fsGroup set to the agent uid (default 10001, configurable via CALIBAN_AGENT_UID / CALIBAN_AGENT_GID) and runAsNonRoot. Before this, clones written by the agent were owned by a different uid than the one that had to use them (#79, PR #85)

  • An invalid state.mode is refused at admission. The CRD schema now carries enum: [remote, local], so the apiserver rejects a bad mode on write — the same moment permissionPosture already failed. The enum is in the schema and deliberately not in the Rust type: a task stored before the enum existed may hold any value, and a typed enum would make such an object fail to deserialize entirely, which is worse than the clear Failed status it gets now. The reconcile-time cross-field checks (remote needs an endpoint, local must not have one) are unchanged (#94, PR #97)

Internal

  • The Rust toolchain now lives in exactly one place. rust-toolchain.toml moved to 1.99.0 and is the only pin: ci.yml installs with dtolnay/rust-toolchain@stable and the Dockerfile uses FROM rust:1-bookworm, both deferring to the file, which rustup honours when cargo runs. Previously the version was written in three places that could drift (#90, PR #98)

  • caliban-contract pinned to =0.15.0, in lockstep with prospero (caliban-ai/prospero#266) so the contract's two downstream consumers never disagree about its version. Nothing the operator compiles against changed — launch.rs is the only file differing between the versions. Worth knowing downstream: CalibandLaunch.router_config kept its type but changed meaning, from inline config JSON to a filesystem path, so a caller passing JSON still compiles and silently has it read as a path. The operator passes the mounted ConfigMap path (#95, PR #99)

  • Documentation audit against the shipped code: CRD and deploying references brought current, README moved to v0.6.0, and the missing ADR template restored (PR #86)

  • This file, and an mdBook guide published from docs/guide/. The operator was the only repo in the suite with neither. The guide ingests the ADR log, this changelog and the three reference docs at build time, so each has exactly one editable copy (#92, PR #100)

  • Six append-only notes recording where the code has overtaken ADRs 0001, 0002 and 0003 — the volumeClaimTemplates type, two Sandbox fields that were never declared, the control-plane TLS environment variables removed in #59, the now-configurable cluster DNS domain, serde_yaml → serde_norway, and the target cluster moving to k3s v1.36. No decision prose was altered (#93, PR #101)

  • The guide's published links are now checked. Crawling the newly published site found three 404s on the ADR 0000 page — template.md, README.md and a bootstrap-spec reference — each correct as a file on GitHub and dead once rendered, because only files listed in SUMMARY.md become pages. The ADR ingest now rewrites links that leave the book to absolute URLs, protecting the ingested chapters so an in-book ../<chapter>.md still resolves, and covering both the inline ](url) and reference [label]: url forms. The bootstrap reference points at caliban, where that document actually lives — it had never resolved in this repo. check-links.sh runs in docs.yml and resolves every internal href, plus any link back into this repository, against what the build produced; it was proven to fail by re-introducing a dead link. mdBook validates no markdown links, so all three had published with zero build warnings (PR #103, PR #104)

  • Changelog catch-up: #100 and #101 were missing from the file #100 created (PR #102)

0.6.0 - 2026-09-19

Per-session permission posture, and caliband's launch surface sourced from the contract crate rather than hand-written strings.

Added

  • CalibanTask.spec.task.permissionPosture (supervised | unattended), admitted only under a Workspace whose agentPolicy.allowUnattended is true. The effective posture is reported in status and shown as a Posture printer column. Recorded as ADR 0006 (#81)

Changed

  • caliband's flags and environment variable names now come from caliban-contract 0.14.0, and provider credential environment names from ProviderKind, so the operator no longer carries its own copy of caliband's launch surface (#83)

  • CRD documentation fixes: tokenRef principal guidance, and an egress example using an RFC 5737 documentation address (#77)

Requires caliban-crds >= 0.2.5 for the permissionPosture / agentPolicy fields.

0.5.0 - 2026-09-16

Terminal phases, workspace egress control, storage settings and Kubernetes Events — 17 PRs.

Added

  • AgentsSettled resolves to a terminal phase (#75)
  • Workspace egress allow-list (#74)
  • spec.state storage settings plus a gonzalod token (#76)
  • Kubernetes Events for reconcile outcomes (#73)
  • Sandbox resource requests and limits (#62)
  • Configurable cluster DNS domain (#71)
  • Server-side-apply status writes (#67)
  • A watch on owned Sandbox objects (#68)

Fixed

  • Reconcile, config, RBAC and CRD fixes (#60, #61, #63, #65, #69, #70, #72)
  • Documentation corrections (#66, #77)

0.4.0 - 2026-09-13

Interactive tasks, sandbox reachability fixes, and the removal of Ollama.

Added

  • CalibanTask.spec.task.interactive (#28, PR #29)

Fixed

  • Project the provider base URL and API key under provider-native environment variable names (#31)
  • Pass the session-plane CA to caliband (#33)
  • Make Workspace.spec.sources optional (#34)
  • Pass --tls-server-name so workers verify against the certificate SAN (#36)

Removed

  • Ollama support. Samples, CRDs and tests migrated to kind: openai with an explicit baseUrl, which covers the same deployments without a dedicated provider kind (#39)

0.3.1 - 2026-07-19

Fixed

  • Per-agent stream reachability in Kubernetes. build_sandbox passes --advertise-host <sandbox>.<ns>.svc.cluster.local and pins --agent-port-base, and the sandbox NetworkPolicy opens the per-agent stream window (7100–7999) alongside 8443. Without this, prosperod could not attach to a k8s agent's output because caliband advertised tcp://0.0.0.0:7100 (#24, #25, PR #26)

0.3.0 - 2026-07-18

Fixed

  • build_sandbox launches caliband with the shared session-plane bearer token and TLS, so the pod binds instead of crash-looping on "a non-empty bearer token is required" (#22, PR #23)

0.2.0 - 2026-07-12

The configuration plane: a Workspace custom resource, and resolve-and-pin.

Added

  • Workspace CRD (v1alpha1), with CalibanTask.workspaceRef / providerRef resolve-and-pin (#11, PR #12)

Changed

  • Breaking (pre-v1). Inline CalibanTask.spec.workspace removed in favour of workspaceRef (#11, PR #12)
  • Gate the CalibanTask pin on Workspace readiness (#13, PR #18)

Fixed

  • CRD schema validation, graceful shutdown, and serde_yaml → serde_norway (#2, PR #10)

Removed

  • Dead WorkspacePhase::Reconciling variant (#14, PR #17)
  • Orphaned CalibanTaskStatus.workspace field (#15, PR #16)

0.1.1 - 2026-07-05

First release that reconciles end to end.

Fixed

  • volumeClaimTemplates TypeMeta, so a CalibanTask reconciles into a Sandbox that agent-sandbox v0.5.0 accepts (#7, PR #8)

0.1.0 - 2026-07-05

Added

  • The container image and its multi-arch release workflow. A two-stage Dockerfile (rust:1.95-bookworm → debian:bookworm-slim + ca-certificates) builds the controller and runs it non-root as uid 10001, matching the Helm chart's securityContext; the controller writes nothing to disk, so it runs under a read-only root filesystem. release-image.yml builds both arches and publishes ghcr.io/caliban-ai/caliban-operator on v* tags (caliban-ai/caliban#358, PR #5)

Architecture Decision Records

ADR 0000 · Record architecture decisions

  • Status: accepted
  • Date: 2026-06-14

Context

caliban has kept Architecture Decision Records since the Layer-0 bootstrap (ADRs 0001–0047). The original 2026-05-22 Layer-0 bootstrap design placed them at the repository root in adrs/, reasoning that ADRs are first-class Layer-0 deliverables and top-level placement makes them impossible to miss.

Since then the sibling repositories adopted the conventional adr-tools / MADR layout instead: prospero and gonzalo both keep their records under docs/adr/, seed the log with a meta "record architecture decisions" entry, and (prospero) ship a template.md. caliban was the outlier — root adrs/ (plural), no meta record, no template — which created cross-repo confusion and path mismatches for anyone moving between the three repos.

There was no ADR stating why caliban records decisions or where they live; that rationale lived only in a feature design doc, which is exactly the kind of external dependency ADRs are supposed to avoid.

Decision

We will keep Architecture Decision Records under docs/adr/, in MADR-lite format (a lightweight extension of Michael Nygard's original ADR style), matching sibling repos prospero and gonzalo. Specifically:

  • Location: docs/adr/ (singular adr), not the former root adrs/. This supersedes the root-placement decision in the Layer-0 bootstrap design; existing records were relocated with git mv to preserve history.
  • This meta record is numbered 0000 so the existing 0001–0047 numbering is preserved — no renumbering churn, and the log still opens with a record of the practice itself.
  • Format: each ADR is one append-only file NNNN-kebab-title.md with Context, Decision, and Consequences. A decision is changed by writing a new ADR that supersedes the old one, never by rewriting history.
  • Template: new ADRs start from template.md.
  • Status legend: accepted / superseded / proposed / rejected, indexed in README.md.

Consequences

  • Positive: one consistent ADR convention across caliban / gonzalo / prospero; the conventional, tooling-friendly docs/adr/ location; and the rationale for the practice now lives in an ADR rather than a feature design doc, so it is self-sustaining.
  • Negative: a one-time churn to relocate the directory and update every inbound reference (crate rustdoc, README, the mdBook guide, the parity matrix, and the historical design docs). ADRs no longer sit at the repo root, so they are slightly less discoverable from a bare ls — mitigated by a pointer from the top-level README.md.
  • Revisit if: the agreed cross-sibling ADR standard changes, or the docs/adr/ layout proves harder to maintain than the root placement it replaced.

ADR 0001 · kube-rs stack + CalibanTask CRD API

  • Status: accepted
  • Date: 2026-07-04
  • Source: k8s system-design spec (§"CalibanTask CR + workspace model", §"caliban-operator") in the caliban-ai docs hub · caliban #282 · epic #274

Context

The k8s epic (#274) needs a Kubernetes operator that reconciles a CalibanTask custom resource into a sandboxed caliband pod (via agent-sandbox). The repository was stood up in #277 as bare scaffolding (an empty caliban-operator crate, AGPL, Rust 1.95, a fmt/clippy/build/test CI). This ADR records the foundational, hard-to-change decisions that #282 commits to: the controller stack and the CalibanTask API surface (group/version/kind), which becomes a compatibility contract the moment a CR is applied to a cluster.

Constraints and forces:

  • The operator is a thin caliban-semantics layer over agent-sandbox — it must not reimplement pods/isolation/warm-pools. So the stack is a controller runtime, not a pod scheduler.
  • The target homelab runs k3s v1.31; the CRD and the client must work there, and the k8s-openapi version pin selects the Kubernetes API surface the operator compiles against.
  • kube-rs pairs kube with a specific k8s-openapi major; the two must be chosen together or the build won't resolve. Picking versions by hand is brittle.
  • The spec fixes the CR shape (workspace.sources, task, model, state, isolation, resources, lifecycle; status phase/calibandEndpoint/ sandboxRef/checkpointRef/workspace.materialized/conditions).

Decision

  1. Controller stack: kube-rs. Depend on kube (features runtime, derive, client) + k8s-openapi (feature v1_32) + schemars (CRD schema generation) + tokio + serde/serde_json + serde_yaml (CRD YAML emission) + thiserror + tracing + futures. Exact compatible versions are resolved with cargo add (which honors kube's k8s-openapi pairing) and then pinned in Cargo.lock; we do not hand-pick the kube/k8s-openapi pair. The resolved pair is kube 4.0 + k8s-openapi 0.28, whose lowest offered API feature is v1_32 (there is no v1_31 for this pairing). v1_32 is the compiled API surface; it is forward-compatible with the target k3s v1.31 apiserver for the objects we use (CRD apiextensions.k8s.io/v1, core objects, and the agent-sandbox CRDs consumed in #283) — the Kubernetes API is tolerant across a one-minor skew for these.

  2. API: caliban.caliban-ai.dev/v1alpha1, kind CalibanTask, namespaced, with a status subresource. Defined via #[derive(CustomResource, …)] on a CalibanTaskSpec struct mirroring the spec's fields, with a separate CalibanTaskStatus. v1alpha1 signals the API is unstable and may change without conversion webhooks — appropriate for an MVP; graduation to v1beta1/v1 is a later ADR. The status subresource keeps spec (intent) and status (observed) on separate update paths, as Kubernetes convention requires for controllers.

  3. CRD YAML is generated, not hand-written. A crdgen binary emits CalibanTask::crd() as YAML; the generated CRD is committed (so the helm chart in #284 and kubectl apply consumers have a stable artifact) and a test asserts the committed YAML is in sync with the Rust types (regenerate-and-diff), so the struct is the single source of truth.

  4. #282 delivers a no-op reconcile that sets .status.phase = Pending. The controller skeleton watches CalibanTask, and its reconcile only initializes status to Pending and requeues — it creates no Kubernetes objects. The real reconcile (CalibanTask → agent-sandbox Sandbox + RBAC/NetworkPolicy) is #283. The phase enum is the spec's lifecycle: Pending → Provisioning → Running → Draining → Completed | Failed.

Consequences

  • Positive: the operator has a real, cluster-installable CRD and a running (if inert) controller — the #282 acceptance — on a conventional, well-supported Rust k8s stack. The generated-and-checked CRD makes the Rust types authoritative and gives #284's chart a stable file to package. v1alpha1 buys freedom to iterate the API through P2/P3 without conversion machinery.
  • Negative: v1alpha1 means the CR shape can break between releases with no automated migration — acceptable pre-1.0, but every CR author is exposed to it. Pinning k8s-openapi to v1_32 ties the compiled API surface to that release; bumping it later is a deliberate, tested change. Committing generated YAML adds a keep-in-sync test that must be run when the types change (the test is the guard).
  • Revisit if: the API stabilizes enough to warrant v1beta1/v1 + a conversion webhook; the target cluster's Kubernetes version moves far enough that the v1_32 k8s-openapi surface needs raising (or drops below the tolerated skew with k3s v1.31); or agent-sandbox's own API (consumed from #283) forces a different k8s-openapi pairing.
  • serde_yaml was replaced by serde_norway — superseded. Decision 1 lists serde_yaml in the stack. It was unmaintained and carried a RUSTSEC advisory, so the dependency is now serde_norway, its maintained fork, with the same role (CRD YAML generation) and a compatible API. The decision — generate the CRD YAML from the Rust types and commit it — is unchanged; only the crate that does it moved.
  • The target cluster has moved to k3s v1.36 — context changed. The Context and decision 2 are written against k3s v1.31, and the "Revisit if" clause above names the cluster version moving as a trigger. It has: the homelab now runs k3s v1.36 (see Deploying). k8s-openapi is still pinned to v1_32, which remains within the tolerated skew and compiles against a surface older than the server — the forward-compatible direction, and the one the decision intended. No change is required today; this is recorded so the version in the Context is not read as current, and so the trigger is visibly accounted for rather than silently passed.

ADR 0002 · Reconcile CalibanTask → agent-sandbox Sandbox (+ per-task SA & NetworkPolicy)

  • Status: accepted
  • Date: 2026-07-04
  • Amended by: ADR 0005 — decision 6 (status gains prospero's AgentsSettled condition as an input, via server-side apply field ownership)
  • Source: k8s system-design spec (§"caliban-operator", §"agent-sandbox integration") in the caliban-ai docs hub · caliban #283 · epic #274 · builds on ADR 0001

Context

ADR 0001 fixed the controller stack (kube 4.0 / k8s-openapi 0.28 / v1_32) and the CalibanTask API. #282 shipped a placeholder reconcile that only initialized .status.phase = Pending. #283 makes the reconcile real: turn a CalibanTask into a running, sandboxed caliband pod reachable at a stable DNS, with per-task RBAC and network isolation, and drive .status from the backing Sandbox.

The spec is explicit that the operator is a thin caliban-semantics layer over agent-sandbox — it must not reimplement pods/isolation. So the reconcile composes existing objects rather than managing pods directly.

Forces:

  • agent-sandbox owns the Sandbox CRD (agents.x-k8s.io/v1beta1). We consume it — we must not generate, install, or drift-guard its schema. Verified v0.5.0 SandboxSpec = { podTemplate{metadata,spec: core.PodSpec} (required), service bool, operatingMode enum(Running|Suspended)=Running, shutdownPolicy enum(Delete|Retain)=Retain, shutdownTime, volumeClaimTemplates []core.PVC }; SandboxStatus.serviceFQDN is the pod's stable DNS.
  • Reconcile must be idempotent and self-healing — re-running on the same CalibanTask must converge, not duplicate, and children must be garbage-collected when the CalibanTask is deleted.
  • Cluster-agnostic (the eventual charts are public): no home-cluster identifiers baked into generated objects.
  • k3s v1.31 target vs v1_32 compile surface (ADR 0001): stick to long-stable PodSpec/NetworkPolicy fields.
  • The caliband pod needs zero Kubernetes API access; its identity exists only to be a NetworkPolicy/PodSecurity subject.

Decision

  1. Consume Sandbox as a foreign typed resource. Hand-author a minimal Rust Sandbox via #[derive(CustomResource)] pointed at group = "agents.x-k8s.io", version = "v1beta1", kind = "Sandbox", namespaced, status = "SandboxStatus", declaring only the fields we set or read (podTemplate, service, operatingMode, shutdownPolicy, shutdownTime, volumeClaimTemplates; status serviceFQDN, conditions). podTemplate reuses k8s_openapi PodTemplateSpec; volumeClaimTemplates reuses PersistentVolumeClaim. We never call Sandbox::crd(), emit its YAML, or add it to the drift-sync test — agent-sandbox installs its own CRD. Extra SandboxSpec fields we omit are pruned by the API server's structural schema; this is a read/write view, not the schema of record.

  2. Reconcile = pure builders + a thin apply loop. The mapping CalibanTask → {Sandbox, ServiceAccount, NetworkPolicy} and the status derivation are pure functions (build_sandbox, build_service_account, build_network_policy, derive_status), unit-tested with no cluster. The async reconcile only: server-side-applies each child, reads back the Sandbox, and patches CalibanTask.status.

  3. Idempotency via server-side apply + owner references. Children are applied with Patch::Apply under field manager caliban-operator (force), so re-reconciles converge on one object. Every child carries a controller OwnerReference to its CalibanTask, so deleting the task cascades. Child names are deterministic: <task>-sbx (Sandbox), <task>-sa (ServiceAccount), <task>-netpol (NetworkPolicy).

  4. Least-privilege RBAC = a dedicated, token-less ServiceAccount with no bound Role. The caliband pod is given its own per-task ServiceAccount with automountServiceAccountToken: false, referenced from the podTemplate. Because no Role/RoleBinding is created, the pod holds zero API permissions — strictly the least privilege. An empty Role would be noise; the token-less SA is the RBAC posture. (The operator's own cluster RBAC — the right to create Sandboxes/SAs/NetworkPolicies — is deploy-time chart concern #284, not this reconcile.)

  5. NetworkPolicy: default-deny with the minimal working allowances. The per-task NetworkPolicy selects the Sandbox's pod labels and sets policyTypes: [Ingress, Egress] with: deny-all by default, allow DNS egress (UDP/TCP 53), allow general egress (git clone + provider APIs; the empty-to rule allows egress to all destinations at MVP, in-cluster included — restricting it to non-cluster CIDRs is deferred because it would require an environment-specific cluster CIDR, a home-cluster identifier we don't bake in), and allow ingress on the caliband port (8443) from the task's namespace. The spec's finer rules (prospero→caliband, pod→gonzalo) require cross-namespace identity the CR does not carry generically at MVP; they are parameterized in a later iteration. The MVP ships a real, cluster-agnostic default-deny posture (selectors are the pod's own labels — no home-cluster identifiers).

  6. Status is derived from the Sandbox, not initialized-and-frozen. This evolves #282's init-once no-op: derive_status(task, sandbox) returns Provisioning once the Sandbox is applied but has no serviceFQDN yet, Running once serviceFQDN is populated, and sets calibandEndpoint = "<serviceFQDN>:8443" and sandboxRef accordingly. Pending remains the pre-provision default (a CalibanTask observed before its first successful apply). The status patch is skipped when the derived status equals the observed one (no-op churn avoidance, same discipline as #282).

  7. Explicit deferrals (out of #283 scope, tracked upstream): the caliban-aware drain finalizer (needs caliband's checkpoint gRPC — deferred with the transport #314), idle→pause/resume mapping, SandboxTemplate/SandboxWarmPool (Phase 4), and full Secret/ConfigMap credential projection. #283 projects only what the acceptance path needs: the workspace PVC, the caliband image + runtimeClass, and the gonzalo endpoint / model-router refs as env, so the pod comes up reachable.

Consequences

  • A CalibanTask now materializes real cluster objects. Applying one yields a Sandbox (→ caliband pod on the configured RuntimeClass, workspace PVC, stable DNS), a token-less SA, and a default-deny NetworkPolicy; .status reports Provisioning → Running with calibandEndpoint. This satisfies #283's acceptance and unblocks prospero's K8sFleet (#64) reading the endpoint.
  • agent-sandbox must be installed in the cluster for the applies to succeed (its Sandbox CRD must exist). This is the cluster prerequisite the umbrella chart bundles by default (helm-charts#6); without it the operator's Sandbox apply fails and the task stays Provisioning — an honest, retried error, not a crash.
  • The operator needs cluster RBAC to create these children — it can create sandboxes.agents.x-k8s.io, serviceaccounts, and networkpolicies.networking.k8s.io, and patch calibantasks/status. That ClusterRole ships with the operator chart (#284); this ADR fixes the exact verb set that chart must grant.
  • Deferring the drain finalizer means a deleted CalibanTask tears down its Sandbox immediately (owner-ref GC) without checkpointing in-flight agents. This is acceptable for the MVP and revisited when caliband checkpoint gRPC lands.
  • The foreign Sandbox view can rot if agent-sandbox changes its v1beta1 schema. We pin the consumed version (v1beta1) and cover the fields we use with round-trip tests against the published CRD's shape; a breaking upstream change surfaces as a deserialize/apply failure, caught in integration.
  • serviceFQDN is not readiness — resolved (#45). Decision 6's "Running once serviceFQDN is populated" reported Ready=True as soon as the Service object existed, before caliband had bound its port; a crash-looping caliband could present as Ready indefinitely and prosperod dialed a dead endpoint. The caliband container now carries TCP readiness and startup probes on the control port (no liveness probe — a restart kills every supervised agent), and derive_status gates Running on agent-sandbox's own Sandbox Ready condition, which v0.5.0 sets True only once the pod passes its readiness probe, has an IP, and the Service is ready. An FQDN without that condition stays Provisioning with no endpoint, and a reported not-ready surfaces as Ready=False / SandboxNotReady carrying agent-sandbox's message.
  • Allow-all egress could not be tightened — resolved (#58). The default egress rule above still allows every destination, and remains the default. A Workspace may now declare spec.egress.allow (CIDR + optional TCP ports), which replaces allow-all with DNS plus exactly those destinations; an empty list is DNS-only. It is a Workspace-level field rather than part of the shared IsolationSpec, because CalibanTaskSpec.isolation is a per-run override that wins over the workspace default and would otherwise let a task widen the restriction. Malformed CIDRs and out-of-range ports fail the Workspace.
  • volumeClaimTemplates could not reuse PersistentVolumeClaim — superseded (#7/#8). Decision 1 says the Sandbox view's volumeClaimTemplates reuses k8s_openapi's PersistentVolumeClaim. It cannot: that type serializes its group/version/kind, and agent-sandbox v0.5.0's structural schema rejects those on server-side apply (.spec.volumeClaimTemplates[].apiVersion: field not declared in schema), failing every reconcile. src/sandbox.rs declares a bare VolumeClaimTemplate (metadata + spec) instead, which is what the upstream schema actually describes. podTemplate does still reuse PodTemplateSpec as decision 1 says.
  • The Sandbox view omits shutdownPolicy and shutdownTime — not implemented. Decision 1 lists both among the fields the view declares; it declares podTemplate, service, operatingMode and volumeClaimTemplates only. Nothing sets a shutdown policy today, and the view is deliberately a read/write subset — fields it omits are pruned by the API server, so the gap is inert rather than a defect. It is recorded here because decision 1 reads as a complete list and is not one.

ADR 0003 · The caliband launch contract (daemon args + workspace init)

  • Status: accepted
  • Date: 2026-07-05
  • Source: caliban #366 (follow-up to #283) · epic #274 · builds on ADR 0002

Context

ADR 0002 reconciled a CalibanTask into a correctly-shaped Sandbox/SA/NetworkPolicy but explicitly deferred the exact caliband container contract ("the exact caliband env contract isn't built yet"). A home-cluster dry run exposed that the deferred contract is load-bearing: the pod as built by #283 would not run a live caliband. Verified against caliban's caliband.rs arg parser and the caliban image's Dockerfile:

  • The image is ENTRYPOINT ["caliband"], CMD ["--help"]. #283 set no container command/args, so the pod runs caliband --help → prints usage → exits → CrashLoop.
  • caliband takes --workspace-root <path> as a required flag (there is no env for it); #283 set CALIBAN_WORKSPACE_ROOT env, which caliband ignores.
  • The network switch is --listen <host:port> (or CALIBAN_DAEMON_LISTEN, format host:port); #283 set CALIBAND_LISTEN=tcp://… — wrong name and wrong format.
  • CALIBAN_WORKSPACE_SOURCES (an invented env) is read by nothing, and no one clones the workspace sources into the /work PVC, so the workspace is empty.
  • caliband's --tls-cert/--tls-key/--token are optional, so --listen without them is valid plaintext TCP.

Decision

  1. Run caliband as a daemon via container args. build_sandbox sets the caliband container's args to ["--workspace-root", <workspace_root>, "--listen", "0.0.0.0:<caliband_port>"], replacing the image's --help CMD (the caliband ENTRYPOINT is kept). This uses only flags caliband actually parses.

  2. Drop the mismatched env. Remove CALIBAND_LISTEN, CALIBAN_WORKSPACE_ROOT, and CALIBAN_WORKSPACE_SOURCES (none are read; listen + root are now flags, sources are handled by the init container). Keep GONZALO_ENDPOINT / CALIBAN_ROUTER_CONFIG_REF (consumed by the caliban agent runtime, harmless if unset).

  3. Populate the workspace with a git-clone init container. Add an init container (image from a new git_image setting, default a public alpine/git) that mounts the workspace volume and, for each spec.workspace.sources[], clones repo at ref into path — idempotently (skip if <path>/.git already exists, so pause/resume and pod restarts over the persistent PVC don't refetch or fail). Public sources only for now; private-source auth is a follow-up.

  4. Plaintext TCP for the first e2e. The operator runs caliband with --listen and no TLS/token. This is sufficient to prove the pipeline (a reachable caliband on the Sandbox DNS) and matches caliban's optional-TLS design. Provisioning per-Sandbox certs

    • a bearer token (and wiring prospero's matching client) is a tracked security follow-up — the transport already supports it (caliban #280).

Consequences

  • Applying a CalibanTask now yields a caliband that actually starts, has its workspace cloned, and listens on the Sandbox endpoint — closing the gap between "objects render" and "an agent can run". This unblocks the home-cluster e2e.
  • Plaintext TCP is a deliberate, temporary posture. The default-deny NetworkPolicy (ADR 0002) still constrains who can reach the port, but the control/stream bytes are unencrypted and unauthenticated until the TLS/token follow-up lands. Not for multi-tenant/production use.
  • The init container needs a git image and egress to clone sources — the ADR 0002 NetworkPolicy already allows general egress. Private sources will need credential projection (deferred).
  • advertise-host / per-agent ports — resolved (#24/#25). Control-plane reachability (serviceFQDN:<port>) worked, but per-agent attach dialing from outside the pod (prospero's session plane) needed --advertise-host <serviceFQDN> + per-agent port routing. build_sandbox now passes --advertise-host <sandbox>.<ns>.svc.cluster.local and pins --agent-port-base, and the NetworkPolicy opens the matching per-agent stream window alongside the control port.
  • Session-plane TLS needs the CA, not just the keypair — resolved (#32). The TLS follow-up to decision 4 landed --tls-cert/--tls-key but no --tls-ca, so caliband had no trust anchor to hand down to the workers it spawns. Their Idle/Running reports failed the handshake and were silently dropped, leaving interactive agents stuck in Running. build_sandbox now also passes --tls-ca <mount>/ca.crt (the key was already in the mounted kubernetes.io/tls Secret) and publishes CALIBAN_CONTROL_TLS_CA + CALIBAN_CONTROL_TLS_SERVER_NAME on the caliband container, which the workers inherit. The verified name comes from the new session_server_name setting and must equal the serving cert's SAN — build_status_client's localhost default never matches in-cluster. caliban #510 replaces the env inheritance with explicit forwarding in ExecWorkerLauncher.
  • The verified name must be on argv, not only in the env — resolved (#35). Once caliban #510 landed, that explicit forwarding became the problem: ExecWorkerLauncher sets each worker's CALIBAN_CONTROL_TLS_SERVER_NAME from caliband's own resolved name, overriding the value the worker used to inherit from the pod env. Absent --tls-server-name, caliband resolves that name from --advertise-host (the per-pod DNS name), which the serving cert cannot prove — its only SAN is the session-plane name. So workers verified against the wrong name, the handshake failed, and interactive agents were still stuck in Running even with the CA correctly in place. build_sandbox now passes --tls-server-name <session_server_name> alongside --tls-ca. The general lesson: when a downstream component forwards a value explicitly, the env var stops being the contract — argv is.
  • git clone --depth 1 --branch <ref> accepts branch/tag names only, not commit SHAs — a ref set to a raw commit SHA will fail the clone (the CRD's ref defaults to main; SHA-pinning is a follow-up).
  • GONZALO_ENDPOINT was never consumed — resolved (#41, #53). Decision 2 kept it as "consumed by the caliban agent runtime", but nothing in caliban reads it: caliban reaches a remote gonzalod only through its storage.substrate / storage.remote.url / storage.remote.token_env settings. (CALIBAN_ROUTER_CONFIG_REF was the same kind of mistake, resolved in #44.) caliban #659 added environment overrides for those settings, and spec.state now projects CALIBAN_STORAGE_SUBSTRATE (local → fs, remote → remote), CALIBAN_STORAGE_REMOTE_URL, and, with state.tokenRef, CALIBAN_STORAGE_REMOTE_TOKEN_ENV=GONZALO_TOKEN with GONZALO_TOKEN from the referenced Secret. The operator sets caliban's settings by environment rather than authoring its settings file (ADR 0005). Requires a caliban build that includes #659.
  • The control-plane TLS env vars are no longer published — resolved (#59). The note above still says the operator "publishes CALIBAN_CONTROL_TLS_CA + CALIBAN_CONTROL_TLS_SERVER_NAME on the caliband container, which the workers inherit". It no longer does. caliband ≥ 0.8.0 (caliban#510, #512) sets both on each worker it spawns, explicitly, from its own --tls-ca / --tls-server-name — and the operator passes both flags. Publishing the same values in the pod env was therefore a second mechanism that the argv overrides and that could only drift from it, so it was removed. Inheritance is not how the workers get these values; caliband sets them. caliband_env_carries_no_control_plane_tls_vars keeps them out.
  • The cluster DNS domain is no longer hardcoded — resolved (#54). The advertise-host note above writes <sandbox>.<ns>.svc.cluster.local as a fixed suffix. The domain is now CALIBAN_CLUSTER_DOMAIN, defaulting to cluster.local, so the behaviour described is the default rather than the only possibility. A cluster served under another domain no longer gets an unreachable advertise host.

ADR 0004 · Workspace CRD + CalibanTask resolve-and-pin

  • Status: accepted
  • Date: 2026-07-11
  • Source: Prospero K8s Config Plane design (umbrella design doc: docs/superpowers/plans/2026-07-11-workspace-crd-and-resolve-and-pin.md) · caliban-operator #11 · epic #274 · cross-repo: prospero mirror (prospero#141), chart RBAC (helm-charts#30) · builds on ADR 0001

Context

ADR 0001 put the workspace (sources, model/provider config) inline on CalibanTask.spec.workspace. Prospero's K8s Config Plane epic needs a UI where a user edits workspace-level config (git sources, which model providers are available, which Secret backs each) independently of, and prior to, launching any task. With the config living only inline on a CalibanTask, prospero has nowhere to persist it: there is no object to CRUD before a task exists, and no durable home for it once a task completes and its CalibanTask is deleted. Prospero's existing k8s-config-plane endpoint has no backing resource to write to and returns 405 Method Not Allowed for exactly this reason.

The epic's shape for this is explicit: prospero becomes a pure editor of a CRD — it CRUDs Kubernetes objects and lets the operator, not prospero, own reconciliation and credential access. That requires the config to be its own first-class, independently-lifecycled resource, and it requires prospero to never need Secret read access (the operator remains the only component that reads Secret values — a pre-existing, load-bearing security boundary, not a new one introduced here).

Forces:

  • Config must outlive any single task and be editable without one. Sources and provider bindings are workspace-scoped, shared across many CalibanTasks, and meaningfully exist before the first task is created.
  • Multiple named providers, not one. A workspace may bind a cheap/local model (e.g. a local openai-compatible server such as llama.cpp) for routine work and a stronger hosted model (e.g. Anthropic) for harder tasks; a task picks one by name at submission time, not at workspace-authoring time.
  • Prospero must never read Secrets. Its CRUD surface is the Workspace/ CalibanTask specs only; credentialsRef carries a Secret name + key, never a value, so the config plane can be edited without granting secrets RBAC to prospero.
  • Immutable-run guarantee must survive the split. #283 and #366 (ADRs 0002–0003) established that a running task's config is stable for its lifetime. Moving provider config out of the task's own spec and into a separately-editable Workspace must not let a config edit reach into a task that's already running.
  • Pre-v1alpha1 API, no migration machinery. As in ADR 0001, v1alpha1 buys the freedom to reshape the CR without a conversion webhook; this ADR spends that budget on the workspace split rather than deferring it.

Decision

  1. A first-class Workspace CRD (caliban.caliban-ai.dev/v1alpha1, namespaced, status subresource, shortname cws) holds the durable, shared config: sources (the git checkouts), a named providers list (each {name, kind, baseUrl, model, credentialsRef} — a credentialsRef is a by-name Secret+key reference, never a value), an optional defaultProvider, non-secret env, and default isolation. WorkspaceStatus carries phase (Pending → Ready | Failed — validation is synchronous, so there is no observable intermediate reconciling state), conditions, observedGeneration, and a human-readable message. The CRD YAML is generated from the Rust types and golden-tested in sync, per ADR 0001's discipline.

  2. A pure validation function backs a lightweight status-only reconciler. validate_workspace(spec, secret_present: Fn(&str, &str) -> bool) checks unique provider names, a resolvable defaultProvider, and that every credentialsRef names an existing Secret key — the existence check only. The controller supplies secret_present by calling the Kubernetes API; the operator is the sole reader of Secret contents in the system, unchanged from before this ADR but now made structural: nothing else needs secrets RBAC because nothing else touches Workspace reconciliation. The controller's only side effect is patching Workspace.status — it creates no pods, no children; provisioning stays entirely in the CalibanTask reconcile (ADR 0002).

  3. CalibanTask references a Workspace by name and resolves-and-pins at admission. CalibanTaskSpec gains workspaceRef: { name } (required) and providerRef: Option<String> (which named provider to bind; falls back to the workspace's defaultProvider, then to the sole provider if there's exactly one). resolve_workspace(spec, provider_ref) is a pure function that flattens the chosen provider and the workspace's sources/env/isolation into a single ResolvedWorkspace, erroring on a dangling workspaceRef/providerRef or an ambiguous default. The CalibanTask reconcile calls it once and pins the result into CalibanTaskStatus.resolvedWorkspace; if that field is already set, later Workspace edits are not re-resolved into it — the pod is always built from the pinned snapshot, preserving the immutable-run guarantee across the split.

  4. Inline CalibanTask.spec.workspace is removed, not deprecated. Per ADR 0001's v1alpha1 bet, this is a breaking, pre-v1 change with no conversion shim: existing CalibanTask CRs must be re-created against the new schema (workspaceRef/ providerRef) with a companion Workspace applied first. There is no dual-write or fallback path to the old inline shape.

  5. Sample CRs are the cross-repo contract fixture. deploy/samples/workspace.yaml and deploy/samples/calibantask.yaml are committed, mutually-consistent examples (the sample task's providerRef names a provider the sample workspace defines) and are golden-tested here (workspace::tests::sample_manifests_deserialize — both deserialize against the Rust types and resolve_workspace succeeds against them). Prospero's mirror types are validated against these same fixtures (prospero#141), so this is the shared fixture two repos check against independently.

Consequences

  • Prospero gets a durable object to edit and no path to Secrets. Workspace authoring (sources, provider bindings) no longer requires an existing task, and prospero's CRUD surface never needs secrets RBAC — credentialsRef is name-only. This is the change that turns the 405 from the epic's Context into a real endpoint.
  • The immutable-run guarantee is preserved, now across two objects instead of one. A running task's pod is built from status.resolvedWorkspace, pinned once; editing the backing Workspace (rotating a provider, changing a source) affects only tasks admitted after the edit, never one already running. This mirrors ADR 0002's status-derivation discipline (compute, compare, patch only on change) applied to a resolve instead of a derive.
  • This is a breaking API change with no migration. Every existing CalibanTask in any cluster must be replaced (workspaceRef + a companion Workspace), and every caller (prospero, any manual manifests) must move off inline spec.workspace in lockstep with this operator version. Acceptable pre-1.0 per ADR 0001's v1alpha1 bet; would not be acceptable post-graduation.
  • The operator's own cluster RBAC needs a Workspace/workspaces/status grant. This is a small addition to the ClusterRole ADR 0002 already required for CalibanTask; the actual chart change is cross-repo and deferred to helm-charts#30, not addressed here.
  • Multi-namespace tenancy remains deferred. workspaceRef is a same-namespace, by-name reference (mirroring WorkspaceRef's single name field); a CalibanTask cannot bind to a Workspace in another namespace. Cross-namespace workspace sharing, if ever needed, is a later ADR.
  • Prospero's mirror types can rot independently of this repo. The sample CRs are the shared contract, but there is no compile-time link between this repo's Rust types and prospero's — a schema change here that isn't reflected in prospero#141's mirror surfaces only at prospero's own test run, not here.

ADR 0005 · Operator owns infrastructure, prospero owns agent lifecycle, caliban owns its contract

  • Status: accepted
  • Date: 2026-09-13
  • Source: caliban-operator #37 design discussion · follow-ups: caliban #656 (contract crate), prospero #228 (AgentsSettled), helm-charts #52 (RBAC), caliban-operator #64 (server-side apply status) · amends decision 6 of ADR 0002

Context

Three components cooperate around a CalibanTask:

  • caliban-operator reconciles the CR into a Sandbox, ServiceAccount, and NetworkPolicy, and derives status from Kubernetes objects (ADR 0002). It launches caliband by writing its pod spec (ADR 0003). It has no network client for caliband.
  • prospero dials caliband over TLS + bearer token to spawn, attach, stream, and poll agents (list()).
  • caliband supervises the agents inside the sandbox.

caliban's contract with the other two is implicit and copied by hand. The operator types caliband's flag and env names as strings in src/resources.rs; prospero mirrors caliban's control wire types in its own wire.rs, deliberately not depending on caliban-supervisor because that crate is the whole daemon. Each drift has surfaced only at runtime, silently: provider env names caliban never read (#30), a missing CA and then a wrong verified server name that kept agents from ever reporting Idle (#32, #35), and a router-config env name nothing read (#44).

#37 exposed the next step of this pattern. A CalibanTask never leaves Running because only caliband's agent list knows an agent finished — the pod stays up. The obvious fix, having the operator poll caliband, would make the operator a third hand-coupled consumer of caliban's wire protocol (a TLS client, token handling, protocol version lockstep), alongside prospero, which already has all of that.

Decision Drivers

  • Minimize the number of components that speak caliban's control protocol.
  • Turn contract drift from silent runtime failure into a compile-time error.
  • Keep one owner per status field; no component may clobber another's writes.
  • Preserve sandbox isolation (sandbox pods stay token-less).

Considered Options

  1. Operator polls caliband via caliban-supervisor or a mirrored client. Single status writer, but a third protocol consumer and a heavy daemon dependency.
  2. caliband writes its own lifecycle into the CR. Protocol-free for everyone else, but the sandbox pod would need a Kubernetes API token and RBAC, weakening isolation.
  3. caliband exits when its agents finish (pod Succeeded/Failed). Fully Kubernetes-native, but only fits one-shot tasks, not interactive ones.
  4. Split responsibilities, meet in the CR via server-side apply field ownership, and publish the contract (chosen).

Decision

We will draw the component boundaries as follows:

  1. The operator owns infrastructure. It manages the Sandbox, pod, ServiceAccount, and NetworkPolicy, and derives infrastructure state (Pending/Provisioning/Running, calibandEndpoint, sandboxRef, its Ready condition) from Kubernetes objects only. It does not become a caliband client.
  2. prospero owns agent lifecycle and is the only component that speaks caliband's control protocol. It reports agent state into the CR as an AgentsSettled condition: settled when no agent is Spawning/Running/Idle; reason Failed if any ended Failed/Crashed, else Succeeded.
  3. They meet in CalibanTask.status through server-side apply field ownership. Each component applies only the fields it owns under its own field manager (caliban-operator, prospero); conditions is a map-list keyed by type. The operator maps AgentsSettled to phase: Completed/Failed without knowing the wire protocol. This amends decision 6 of ADR 0002, which derived status from the Sandbox alone.
  4. caliban owns and publishes its contract as a thin caliban-contract crate with no daemon internals: a typed caliband launch builder (flags and env) and the control wire types, with an optional feature-gated client. The operator builds caliband's launch arguments from it; prospero replaces its mirrored wire.rs with it. ADR 0003's launch contract is unchanged in substance — only where its names are defined moves.

Out of scope: Draining/checkpoint lifecycle (#42, #43), which follows the same split when implemented.

Consequences

Positive

  • The operator never takes a caliband client, TLS stack, or protocol version dependency; only prospero speaks the wire.
  • Renamed or removed caliband flags, env names, and wire fields become compile errors in the consumers instead of silent runtime failures.
  • Status has explicit per-field ownership; merge-patch null/empty-array workarounds in the operator go away.
  • kubectl get calibantasks can distinguish live tasks from finished ones (#37) without weakening sandbox isolation.

Negative

  • Terminal phase depends on prospero running; without it, finished tasks stay Running (today's behaviour, no worse).
  • Cross-repo sequencing: caliban (contract crate, caliban#656), prospero (AgentsSettled, prospero#228), helm-charts (prospero patch on calibantasks/status, helm-charts#52), and the operator (server-side apply status, #64; phase mapping, #37).
  • Consumers upgrade the contract crate in step with caliban releases — explicit coupling that previously existed implicitly.

Revisit if

  • A second agent-lifecycle observer appears alongside prospero, or prospero stops being part of every deployment.
  • caliban gains a Kubernetes-native completion signal that works for interactive tasks.
  • Server-side apply field ownership on status proves unreliable across the Kubernetes versions we support.

ADR 0006 · Unattended permission posture is authorized by Workspace policy

  • Status: accepted
  • Date: 2026-09-18
  • Source: caliban-operator #80, amended by #51 (decision 7) · caliban ADR 0059 (per-session permission posture), caliban #676 (SpawnSpec.permission_posture) · prospero ADR 0010 (inbound API scopes) · builds on ADR 0004 (resolve-and-pin) and ADR 0005 (component split)

Context

caliban now runs each agent session under a permission posture chosen per session:

  • supervised: a permission prompt goes to a human, who answers it.
  • unattended: the session runs under a bypass profile with no human in the loop.

The default is supervised, so the system fails closed. In Kubernetes the posture a human chose travels on the CalibanTask CR. prospero reads it and puts it on the agent's SpawnSpec. The operator owns the CRD, so it owns the field and its admission rules.

Running unattended is a privileged act. prospero enforces this at its own API: only an admin-scoped token may ask prospero for an unattended session. That check does not cover the cluster. Anyone with create on calibantasks in a namespace can write the CR directly and skip prospero entirely. The operator therefore needs its own gate, one that doesn't depend on who created the CR.

Considered Options

  1. Trust the CR as written. Whoever can create a task can make it unattended. This is the same as having no gate: create on calibantasks is granted far more widely than the right to run agents unsupervised.
  2. Workspace policy. Add Workspace.spec.agentPolicy.allowUnattended, default false. An unattended task is admitted only under a Workspace that sets it. The authority is write access to Workspaces, which is already the privileged object that holds providers, credentials references and egress.
  3. Admission-time identity check. A ValidatingAdmissionPolicy (CEL on request.userInfo) restricts which principals may create unattended tasks. This is precise, but it ships in helm-charts, needs a group or ServiceAccount convention the charts don't define yet, and does nothing for tasks created before the policy existed.

Decision

We will adopt option 2, and leave option 3 as a possible addition later.

  1. CalibanTask.spec.task.permissionPosture is an optional enum, supervised | unattended. Unset means supervised.
  2. unattended is admitted only when the referenced Workspace sets agentPolicy.allowUnattended: true. A missing policy, or a pin made before the field existed, does not permit it.
  3. The check runs before pinning. A denied task is not pinned, is marked Failed with reason PostureNotPermitted and a Warning Event, and is re-checked on a short interval. Allowing it on the Workspace, or editing the task back to supervised, recovers it; the operator never silently downgrades it.
  4. The policy is pinned into status.resolvedWorkspace with the rest of the Workspace config, as with every other part of the Workspace (ADR 0004). A pinned task is checked against its pin on every reconcile. Revoking allowUnattended does not disturb a task that is already running. Editing a running task's spec to unattended fails it if the pin doesn't allow that.
  5. The posture is visible. status.permissionPosture records the posture a task was admitted with, and appears as a printer column. A Normal UnattendedAdmitted Event names the Workspace whose policy allowed it.
  6. The operator does not hand the posture to caliband. prospero maps it onto SpawnSpec.permission_posture (ADR 0005).
  7. allowUnattended authorizes the Workspace's own unsupervised settings too (amended for #51). #51 makes caliban's permission surface typed on the same agentPolicy block: permissionMode, autoAllow, noPermissions. Three of those settings — permissionMode: dontAsk, permissionMode: bypassPermissions, and either switch set to true — leave no human answering prompts, which is the power decision 2 gates. A Workspace asking for one of them is admitted only when it also sets allowUnattended: true; otherwise the task is denied before pinning, exactly as in decision 3, with the reason naming the setting. Without this, the same power would have two doors: one gated, one not. Unlike a task's posture, the policy cannot change under a running task — it travels inside the pin — so it is checked at admission only.

Consequences

Positive

  • A task can't run unattended unless a Workspace author allowed it, however the task was created. prospero's admin scope and this gate cover the two ways in.
  • Unattended tasks show up in kubectl get calibantasks (the Posture column) and in Events.
  • Uses the existing resolve-and-pin model; no new admission infrastructure.
  • One switch is the whole answer to "may agents here run unsupervised?", covering both a task's posture and the Workspace's own permission settings, so an auditor reads one field rather than four.

Negative

  • The gate is only as strong as Workspace RBAC. Anyone who can edit a namespace's Workspace can allow unattended tasks there. Deployments must treat Workspace write access as privileged.
  • Revoking allowUnattended only affects new admissions; a running unattended task keeps its pin until it finishes or is deleted.
  • Within a Workspace that allows it, the gate doesn't limit which principals may create unattended tasks.

Revisit if

  • Tenants need unattended access limited to specific users within a Workspace. At that point, add the admission-time identity check (option 3) in helm-charts.
  • Revocation must stop running tasks, not just block new ones.