caliban-operator
caliban-operator is the Kubernetes operator for caliban
agent workloads. It reconciles two custom resources — a Workspace and a
CalibanTask — into a sandboxed pod running caliband and its agents, composing
the Kubernetes SIG agent-sandbox project
rather than managing pods itself.
It is written in Rust on kube-rs.
Status: both CRDs and both reconcile loops are implemented and running in a
homelab cluster. The API is v1alpha1 and may still change without conversion
webhooks.
Where it sits
The operator owns infrastructure; prospero owns agent lifecycle. That split is deliberate and recorded in ADR 0005.
prospero ──CRUD────────▶ Workspace CR sources, named providers,
credentialsRef, egress, agentPolicy
(this operator validates + reports status)
prospero ──CRUD/watch──▶ CalibanTask CR ──reconcile──▶ agent-sandbox Sandbox pod
workspaceRef, caliband + caliban agents
providerRef, posture
Prospero creates and watches the custom resources; it never creates a pod. The operator never decides which agent runs or when — it decides whether a task is admissible, pins the configuration it will run under, and reconciles the sandbox that results.
The two resources
-
Workspaceis the configuration plane: git sources, named model providers and their credentials, egress rules, storage settings, and the agent policy that governs what an agent in this workspace is allowed to do. -
CalibanTaskis one unit of work. It references aWorkspaceand (optionally) one of its named providers, and carries the task itself.
When a task is admitted, the operator resolves and pins the workspace
configuration into status.resolvedWorkspace, once. Everything the sandbox is
built from comes from that pinned snapshot, so editing the Workspace afterwards
cannot change a running task underneath itself. That rule is
ADR 0004.
Permission posture
A task declares a permissionPosture. unattended — an agent whose tool calls
are never held for a human — is admitted only under a Workspace whose
agentPolicy.allowUnattended is true. The workspace, not the task, is where that
authority is granted, so a task cannot promote itself. The same rule governs the
workspace's own unsupervised agentPolicy settings. See
ADR 0006.
The effective posture is reported in status and shown as a Posture printer
column, so kubectl get ctask says what a task actually got rather than what it
asked for.
Where to go next
- CRD reference — every
WorkspaceandCalibanTaskfield, its default, plus phases, conditions and the Events the operator emits. - Deploying and configuring — prerequisites, every environment variable, the RBAC the operator needs, and troubleshooting.
- Container image — what ships and how it is built.
- Changelog — what changed in each release.
- Architecture Decision Records — the accepted decisions and why they were made.
CRD reference — Workspace and CalibanTask
Both custom resources live in group caliban.caliban-ai.dev, version
v1alpha1, and are namespaced with a status subresource.
v1alpha1 means the shape may change without conversion webhooks
(ADR 0001).
The Rust types in src/crd.rs and src/workspace.rs are the source of truth;
deploy/crd/{calibantask,workspace}.yaml is generated from them and a test
fails if the committed YAML drifts. See Regenerating the CRDs.
| Kind | Plural | Short name | Printer columns |
|---|---|---|---|
Workspace | workspaces | cws | Phase, Age |
CalibanTask | calibantasks | ctask | Phase, Posture, Age |
Runnable samples: deploy/samples/.
Workspace
Durable, shared config a CalibanTask references by name: git sources, named
model providers (with their credential references), workspace-wide env, egress
and agent policy. The operator is the sole reader of provider credential
Secrets — a client such as prospero only ever sees the by-name reference
(ADR 0004).
The Workspace controller provisions nothing. It validates the spec and writes
status.
spec
| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
displayName | string (min 1) | yes | — | Dashboard label. |
sources[] | list | no | [] | Git checkouts. A workspace may be registered bare and gain sources later. |
providers[] | list (min 1) | yes | — | Named providers a task can bind to. |
defaultProvider | string | no | — | Implicit when exactly one provider is defined. |
env[] | {name, value} | no | [] | Non-secret env injected into every agent pod. Never put credentials here — use a provider credentialsRef. |
isolation | object | no | — | Default isolation for agents in this workspace; a task's spec.isolation overrides it. |
egress | object | no | — | Unset keeps allow-all egress. A task cannot override it. |
agentPolicy | object | no | — | What agents here may do. A task cannot override it. |
sources[] entries:
| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
name | string (min 1) | yes | — | Matches caliband's workspace source name. |
repo | string (min 1) | yes | — | Git remote to clone. |
ref | string | no | main | Branch or tag only — git clone --depth 1 --branch rejects a raw commit SHA. |
path | string (min 1) | yes | — | Absolute checkout path. Must be a directory strictly under the operator's workspace root (default /work) and distinct from every other source's path, or the Workspace goes Failed. |
providers[] entries:
| Field | Type | Required | Notes |
|---|---|---|---|
name | string (min 1) | yes | Unique within the workspace. |
kind | string (min 1) | yes | anthropic, openai, google/gemini take their exact env names from caliban-contract. Other kinds (azure, bedrock, vertex, …) get <KIND>_BASE_URL / <KIND>_API_KEY, with azure/azure-openai mapped to AZURE_OPENAI_*. |
baseUrl | string | no | Projected under the kind's native base-URL env var. |
model | string | no | Recorded in the pin. Not projected as env — the model reaches the worker through caliband's SpawnSpec. |
credentialsRef | {secretName, key} | no | Omit for keyless providers (e.g. a local openai-compatible endpoint). Reaches the pod as a secretKeyRef; the operator never inlines the value. |
Base URL and API key are projected under provider-native env names, because
caliban implements no CALIBAN_* namespace for them — it reads
OPENAI_BASE_URL, ANTHROPIC_API_KEY and so on. The kind is additionally
projected as CALIBAN_PROVIDER, not as a provider selector (selection travels in
caliband's SpawnSpec) but because it is the documented input to a
user-configured apiKeyHelper.
egress restricts agent-pod egress to DNS plus the listed destinations,
replacing the default allow-all rule. An empty allow list is DNS-only.
| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
egress.allow[].cidr | string (min 1) | yes | — | IPv4 or IPv6 CIDR, e.g. 192.0.2.0/24, 2001:db8::/32. A malformed CIDR or a prefix longer than the address fails the Workspace. |
egress.allow[].ports[] | list of int | no | [] | TCP ports. Empty allows every port. Anything outside 1-65535 fails the Workspace. |
When you set egress, remember to include the workspace's git remotes, its
model provider endpoints, and gonzalod if a task uses spec.state.
agentPolicy — what agents in this workspace may do
Typed settings for caliban's own permission and extension surface, so a cluster
author configures them as CRD fields rather than as raw env entries. They win
over an env entry of the same name. Every field is optional, and an unset
field leaves caliban's own default alone — which is not the same as setting it
to false.
The whole block is pinned into status.resolvedWorkspace.agentPolicy with the
rest of the Workspace, so it cannot change under a running task.
The authorization switch:
| Field | Type | Default | Notes |
|---|---|---|---|
allowUnattended | bool | false | Authorizes both a task asking for permissionPosture: unattended and the unsupervised settings in the next table. Whoever can edit this Workspace controls it — treat Workspace write access as privileged (ADR 0006). |
Permission settings. The three marked ⚠ leave no human answering prompts, so
each is admitted only when allowUnattended: true — otherwise the task is
denied before pinning with reason PostureNotPermitted, and the message names
the setting. One switch is the whole answer to "may agents here run
unsupervised?", so the same power cannot be reached through a back door
(ADR 0006, decision 7).
| Field | Type | Notes |
|---|---|---|
permissionMode | enum | default, acceptEdits, plan, auto, ⚠ dontAsk, ⚠ bypassPermissions. Projected as CALIBAN_DEFAULT_PERMISSION_MODE; caliban's own --permission-mode flag still wins over it. dontAsk turns every Ask into Allow by caliban's definition, so it grants what bypassPermissions does; auto still classifies each call, so it does not count as unsupervised. |
autoAllow | bool | ⚠ Allow tools that would otherwise prompt. CALIBAN_AUTO_ALLOW. |
noPermissions | bool | ⚠ Disable permission gating altogether — the bluntest lever. CALIBAN_NO_PERMISSIONS. |
Extension settings. These reduce what an agent can reach rather than
granting it power, so none of them needs allowUnattended.
| Field | Type | Notes |
|---|---|---|
noMcp | bool | No MCP server discovery. CALIBAN_NO_MCP. |
noHooks | bool | Bypass every external hook handler; caliban's in-process hooks still run. CALIBAN_NO_HOOKS. |
noSkills | bool | No skill discovery at startup. CALIBAN_NO_SKILLS. |
noSubAgent | bool | Disable the built-in agent tool, so an agent cannot spawn sub-agents. CALIBAN_NO_SUB_AGENT. |
strictKnownMarketplaces | bool | Block plugin installs from marketplaces caliban does not recognise. CALIBAN_STRICT_KNOWN_MARKETPLACES. |
enabledPlugins[] | list of string | An allow-list: every other discovered plugin is disabled. CALIBAN_ENABLED_PLUGINS, comma-joined. An empty list enables none, which is deliberately different from omitting the field — omitted enables everything caliban discovers, so the two cannot be collapsed. |
blockedMarketplaces[] | list of string | Marketplace names to block; unset blocks none. CALIBAN_BLOCKED_MARKETPLACES, comma-joined. |
parallelToolLimit | integer ≥ 1 | Maximum concurrent tool invocations per turn. CALIBAN_PARALLEL_TOOL_LIMIT. The schema enforces minimum: 1 because caliban refuses 0 at startup; unset leaves caliban's default of one less than the CPU count. |
Unlike a task's own posture, this policy is checked at admission only. It travels inside the pin, so it cannot change under a running task — and revoking a setting therefore does not disturb a task already running.
status
| Field | Notes |
|---|---|
phase | Pending (not yet reconciled) → Ready (all providers and credential Secrets resolve) or Failed. |
message | Why it failed, e.g. provider 'planner': secret 'anthropic-key' key 'api-key' not found. |
observedGeneration | The .metadata.generation this status reflects. |
conditions[] | Standard Kubernetes conditions. |
Validation runs in this order, first problem wins: source paths (under root,
distinct) → egress CIDRs and ports → duplicate provider names → a resolvable
defaultProvider → an existing Secret key for every credentialsRef.
The controller deliberately does not watch Secrets — that would need
cluster-wide list/watch on every Secret. Recovery rides the requeue instead:
a Ready workspace is re-checked every 300s, a Pending or Failed one
every 15s, so creating a missing Secret unblocks it within seconds.
CalibanTask
One agent session: a reference to a Workspace, the prompt, and per-run
overrides. The operator reconciles it into an agent-sandbox Sandbox, a
token-less ServiceAccount and a NetworkPolicy
(ADR 0002).
spec
| Field | Type | Required | Default | Consumed by |
|---|---|---|---|---|
workspaceRef.name | string (min 1) | yes | — | operator |
providerRef | string | no | the workspace's defaultProvider, else its sole provider | operator |
task.prompt | string (min 1) | yes | — | prospero |
task.agentType | string | no | — | carried on the CR; the operator does not read it |
task.interactive | bool | no | false | prospero (SpawnSpec.interactive); the operator only carries it |
task.permissionPosture | enum | no | supervised | operator (admission gate) + prospero |
model.name | string (min 1) | no | the resolved provider's model | operator (written into the pin) |
model.routerConfigRef | string | no | — | operator |
state.gonzaloEndpoint | string | no | — | operator |
state.mode | enum: remote | local | no | remote when gonzaloEndpoint is set | operator |
state.tokenRef | {secretName, key} | no | — | operator |
isolation.runtimeClass | string | no | the workspace default | operator (pod runtimeClassName) |
isolation.worktrees | string | no | — | declared, not yet consumed |
resources.class | string | no | — | declared; SandboxTemplate selection is deferred |
lifecycle.idleTimeout | string | no | — | declared; idle→pause mapping is deferred |
lifecycle.onDelete | string: checkpoint | delete | no | — | declared; the drain finalizer is deferred |
tools[] | list of string | no | — | declared, not yet consumed |
Which closed sets the API server enforces. task.permissionPosture,
state.mode and the Workspace's agentPolicy.permissionMode carry an enum in
the generated schema, so a typo in any of them is rejected at admission.
state.mode is a deliberate hybrid: the enum lives in the schema while the Rust
type stays a String. A task stored before the enum existed may hold any value,
and a typed enum would make such an object fail to deserialize — the operator
could not read it at all. So storage::state_problem still rejects an unknown
mode at reconcile time (InvalidState), which is what those pre-existing objects
get.
The strings in isolation.worktrees, resources.class and lifecycle.* carry
no schema enum and are unread. For lifecycle.onDelete that is deliberate
rather than an oversight: #42 implements it (drain finalizer → Draining phase →
status.checkpointRef, designed in caliban ADR 0057) and waits only on prospero
reporting AgentsDrained, so pinning the accepted values now would freeze a
surface that work may still refine.
model.name — per-task model override
A task may name its own model without the Workspace binding a whole new
provider for it. The override is applied to the pinned resolution, so:
- the bound provider's endpoint and credential still apply — only the model moves;
- it cannot change under a running task, like everything else in the pin (ADR 0004);
- prospero needs no change, since it already builds
SpawnSpec.modelfrom the pinned provider.
It is applied after admission, because which model to run is not an
admissibility question. A blank name is treated as a misconfiguration rather
than a selection: it leaves the provider's model alone instead of clearing it,
and the schema's minLength: 1 rejects it outright. model.name is independent
of model.routerConfigRef and may be set alongside it.
spec.isolation is a per-run override that wins over the workspace default.
That is exactly why egress and agentPolicy are Workspace-level fields and
not part of the shared isolation block — a task must not be able to widen a
restriction the workspace imposed.
permissionPosture
| Value | Meaning |
|---|---|
supervised | A permission prompt goes to a human, who answers it. The default, and what an unset field means — the system fails closed. |
unattended | The session runs under an authorized bypass profile with no human in the loop. |
unattended is admitted only when the referenced Workspace sets
agentPolicy.allowUnattended: true. A missing policy — including a pin made
before the field existed — does not permit it. The check runs before pinning,
so a denied task is never pinned to the policy that denied it; allowing it on the
Workspace, or editing the task back to supervised, recovers it. The operator
never silently downgrades a posture.
The same switch also authorizes the Workspace's own unsupervised agentPolicy
settings, so a task is denied with the same PostureNotPermitted reason whether
the unsupervised power was asked for by the task or by the workspace — see
agentPolicy.
A pinned task is re-checked against its pin on every reconcile. Revoking
allowUnattended therefore does not disturb a task already running, but editing
a running task's spec to unattended fails it if the pin does not allow it.
The operator does not hand the posture to caliband. prospero maps it onto
SpawnSpec.permission_posture, reading status.permissionPosture — the posture
the operator admitted — not the spec field.
state (gonzalo persistence)
spec.state becomes caliban's storage settings by environment
(CALIBAN_STORAGE_SUBSTRATE, CALIBAN_STORAGE_REMOTE_URL,
CALIBAN_STORAGE_REMOTE_TOKEN_ENV), so the operator never authors caliban's
settings file. The CRD's local is caliban's fs. Requires a caliban build
with environment overrides for those settings.
A tokenRef projects the gonzalod bearer token into the pod as GONZALO_TOKEN
from the named Secret; only the variable's name appears in the pod spec. Use a
principal with read/write on the gonzalod caliban namespace only — no write
on fleet or fleet-audit.
These combinations fail the task with reason InvalidState: an unknown mode;
mode: remote without gonzaloEndpoint; mode: local with a gonzaloEndpoint
it would never use; a tokenRef without remote storage; a tokenRef whose
Secret or key is missing.
status
| Field | Owner | Notes |
|---|---|---|
phase | operator | See below. |
calibandEndpoint | operator | <serviceFQDN>:<calibandPort>, set only once the Sandbox is Ready. |
sandboxRef.name | operator | <task>-sbx. |
resolvedWorkspace | operator | The pinned config (sources, the single resolved provider, env, isolation, egress, agentPolicy). Set once at admission; later Workspace edits do not re-pin a running task. |
permissionPosture | operator | The posture the task was admitted with. The only posture prospero acts on. |
conditions[] | split | A map-list keyed by type. The operator owns Ready; prospero owns AgentsSettled. Each applies only its own entries under its own field manager, so neither clobbers the other (ADR 0005). |
checkpointRef | — | Declared; checkpointing is deferred. |
The operator writes status with server-side apply under field manager
caliban-operator. A field it owns but omits is removed, so a cleared value is
omitted rather than serialized as null.
Phases
| Phase | When |
|---|---|
Pending | Observed before its first successful apply — no Sandbox yet. |
Provisioning | The Sandbox is applied but has not reported Ready. |
Running | agent-sandbox reports the Sandbox Ready=True — the pod passed its readiness probe on caliband's control port and the Service is ready. |
Draining | Declared; not yet produced (the drain/checkpoint lifecycle is deferred). |
Completed | prospero's AgentsSettled=True with reason Succeeded. |
Failed | prospero's AgentsSettled=True with reason Failed, or a terminal operator-side failure (below). |
A serviceFQDN alone is not readiness: it appears as soon as the Service
object exists, before caliband has bound its port. Running gates on
agent-sandbox's own Ready condition instead.
Terminal phases come from prospero, because infrastructure cannot tell "agent
running" from "agent finished, pod still up" — only caliband's agent list knows,
and the operator is never a caliband client. Without prospero deployed, a
finished task stays Running.
Ready condition reasons (operator-owned)
reason | status | Meaning and recovery |
|---|---|---|
Running | True | caliband is up and attachable. |
SandboxNotReady | False | agent-sandbox reported not-ready; message carries its detail (e.g. "Pod is Running but not Ready"). |
Completed | False | Agents finished successfully; the task is done, so not Ready. |
AgentsFailed | False | One or more agents failed or crashed. |
InvalidName | False | The task name is too long for its Sandbox's Service name (<task>-sbx, max 63 chars). Names are immutable — recreate under a shorter name. The operator stops retrying. |
WorkspaceUnresolved | False | No such Workspace, a Failed one, or an unresolvable/ambiguous providerRef. message names which. Re-checked every 30s; fixing the Workspace recovers the task. |
InvalidState | False | spec.state is malformed or its token Secret is missing. Re-checked every 30s. |
PostureNotPermitted | False | Unsupervised power without agentPolicy.allowUnattended — either the task's permissionPosture: unattended or the Workspace's own permissionMode: dontAsk/bypassPermissions, autoAllow, noPermissions. message names which setting. Re-checked every 30s. |
AgentsSettled reasons are prospero's: Succeeded, Failed, or AgentsActive
(not settled — an idle interactive task stays unsettled).
A task whose Workspace exists but is still Pending is not failed: it is
not pinned either, and the reconcile simply requeues in 10s to re-check once the
Workspace controller has validated it.
Events
kubectl describe calibantask <name> shows what the logs would otherwise hold.
Notes are truncated at 1 kB; publishing is best-effort and never fails or delays
a reconcile.
| Reason | Type | When |
|---|---|---|
PhaseChanged | Normal | The phase moves, e.g. Provisioning → Running. |
UnattendedAdmitted | Normal | A task was admitted unattended; the note names the Workspace whose policy allowed it. |
ReconcileError | Warning | A reconcile returned an error. |
InvalidName / WorkspaceUnresolved / InvalidState / PostureNotPermitted | Warning | The task was marked Failed; the Event reason is the failure's own reason code. |
What one CalibanTask creates
| Object | Name | Notes |
|---|---|---|
Sandbox (agents.x-k8s.io/v1beta1) | <task>-sbx | The caliband pod, its workspace PVC, and a headless Service giving the stable serviceFQDN. |
ServiceAccount | <task>-sa | automountServiceAccountToken: false and no Role/RoleBinding — the pod holds zero API permissions. The token-less SA is the RBAC posture. |
NetworkPolicy | <task>-netpol | policyTypes: [Ingress, Egress], selecting the pod by the labels the operator propagates. |
All three carry a controller OwnerReference to the task, so deleting the task
cascades, and are applied with server-side apply so re-reconciles converge on
one object. Deleting a task tears its Sandbox down immediately, without
checkpointing in-flight agents (the drain finalizer is deferred).
The pod runs every container — the git-clone init container included — as one
uid/gid (default 10001, caliban's app user) with runAsNonRoot: true and
fsGroup set to the same gid, so the clone step and the agent cannot drift apart
and a non-root clone succeeds whatever mode the volume provisioner gave the
volume root. See Deploying
for the one case this does not repair.
The reconcile re-runs on every change to the task and to the Sandboxes it owns,
so a Sandbox becoming Ready reconciles the task at once. The 300s periodic
resync is only a safety net against missed events.
Regenerating the CRDs
The committed YAML is generated, never hand-edited. A *_is_in_sync test fails
if it drifts from the Rust types:
cargo run --bin crdgen calibantask > deploy/crd/calibantask.yaml
cargo run --bin crdgen workspace > deploy/crd/workspace.yaml
cargo test --workspace
The Sandbox type in src/sandbox.rs is a read/write view of a CRD
agent-sandbox owns, not a schema of record. crdgen never emits it and no test
drift-guards it; fields the operator omits are pruned by the API server's
structural schema.
After a CRD change, the copies in the caliban-crds Helm chart need the same
update — that chart names deploy/crd/*.yaml here as its source of truth.
Deploying and configuring caliban-operator
The operator is a controller: it watches Workspace and CalibanTask custom
resources cluster-wide and creates Sandbox, ServiceAccount and
NetworkPolicy objects. It serves no inbound traffic, so it has no Service,
Ingress or port.
This page is the reference for the operator's own configuration surface and the cluster prerequisites it assumes. The Helm charts themselves live in caliban-ai/helm-charts; for chart values, see that repo.
Prerequisites
- agent-sandbox — the
agents.x-k8s.io/v1beta1SandboxCRD and its controller must be present. This is a cluster-level prerequisite, like a CNI or a RuntimeClass, normally installed by the cluster admin. The operator'sSandboxview is written against agent-sandbox v0.5.0; helm-charts vendors that version ascharts/agent-sandboxand the umbrella bundles it by default. Without it, every reconcile's Sandbox apply fails and tasks stayProvisioning— a retried error, not a crash. - The caliban CRDs —
calibantasksandworkspaces, installed as their own step from thecaliban-crdschart (Helm does not upgrade acrds/directory, so that chart keeps them intemplates/). They are copies ofdeploy/crd/here, which is the source of truth. Applying them directly works too:
Upgrade the CRDs before the operator. Against an older CRD, a field the operator relies on —kubectl apply -f deploy/crd/spec.task.permissionPosture,spec.agentPolicy,spec.model.name— is either rejected outright (kubectl applydecodes strictly:unknown field "spec.task.permissionPosture") or silently pruned by the structural schema, so anunattendedtask would read back assupervised. - Lease RBAC, before the operator image — leader election is on by default,
so the operator needs
coordination.k8s.io/leasesin its own namespace. The chart's leaseRoleis harmless against an older operator, so upgrading the chart first is always safe; the reverse order is not. See Leader election. - A session-plane TLS Secret and token Secret — see Session plane below.
- A default StorageClass — each task's workspace PVC is created with no
storageClassName, so it binds the cluster default (deliberately cluster-agnostic).
Kubernetes versions
The operator compiles against the v1_32 k8s-openapi API surface (the
lowest the kube 4.0 / k8s-openapi 0.28 pairing offers). It uses only
long-stable PodSpec, NetworkPolicy and apiextensions.k8s.io/v1 fields, so
it is forward-compatible across the usual skew. The original target was k3s
v1.31 (ADR 0001); it currently
runs on k3s v1.36. No chart declares a kubeVersion floor, so treat this as a
record of what has been run, not an enforced constraint.
NetworkPolicy is only enforced if the cluster's CNI implements it. On a CNI
that ignores it, the per-task policy is applied and inert — the isolation
posture is silently absent.
Installing
CRDs first, then the operator. Via the charts
(caliban-ai/helm-charts, published
to the OCI registry oci://ghcr.io/caliban-ai/charts, not an HTTP chart repo —
there is no helm repo add):
helm install caliban-crds oci://ghcr.io/caliban-ai/charts/caliban-crds
helm install caliban-operator oci://ghcr.io/caliban-ai/charts/caliban-operator
(If the registry refuses an anonymous pull, the GHCR package is still private —
helm registry login ghcr.io first.)
Or install the caliban-system umbrella chart, which can bring agent-sandbox,
gonzalo and prospero with it. Either way, keep environment-specific values in a
separate private overlay (-f private-values.yaml); the charts ship neutral,
cluster-agnostic defaults only.
Then confirm it is up and try a task:
kubectl get deploy -l app.kubernetes.io/name=caliban-operator # 1/1 available
kubectl apply -f deploy/samples/workspace.yaml
kubectl get workspaces # wait for Phase: Ready
kubectl apply -f deploy/samples/calibantask.yaml
kubectl get calibantasks -A # Phase, Posture
The samples target namespace team-a and reference Secrets that do not exist by
default, so the Workspace will report Failed with the missing Secret named in
status.message until you create them. That is the intended first signal.
Configuration
Every setting is an environment variable on the controller container; there are no flags and no config file. Unset means the default below.
Images and the agent pod
| Variable | Default | Notes |
|---|---|---|
CALIBAND_IMAGE | ghcr.io/caliban-ai/caliban:latest | The caliband image run in each Sandbox. Pin a real tag in production — latest makes an agent pod's version unreproducible. |
CALIBAN_GIT_IMAGE | alpine/git:latest | The git-clone init container that populates the workspace. |
CALIBAN_WORKSPACE_ROOT | /work | Workspace mount path in the pod. Every Workspace source path must sit strictly under it. |
CALIBAN_WORKSPACE_STORAGE | 10Gi | Requested size of each task's workspace PVC. |
CALIBAN_AGENT_UID | 10001 | uid every container in the sandbox pod runs as. |
CALIBAN_AGENT_GID | 10001 | gid for the same, and the pod's fsGroup. |
Note what is not here: caliban's own permission and extension behaviour is
not operator configuration. It is set per workspace, as typed
Workspace.spec.agentPolicy
fields, which the operator projects into each agent pod as CALIBAN_*
environment (CALIBAN_DEFAULT_PERMISSION_MODE, CALIBAN_AUTO_ALLOW,
CALIBAN_NO_PERMISSIONS, CALIBAN_NO_MCP, CALIBAN_NO_HOOKS,
CALIBAN_NO_SKILLS, CALIBAN_NO_SUB_AGENT,
CALIBAN_STRICT_KNOWN_MARKETPLACES, CALIBAN_ENABLED_PLUGINS,
CALIBAN_BLOCKED_MARKETPLACES, CALIBAN_PARALLEL_TOOL_LIMIT). Those values win
over a Workspace.spec.env entry of the same name, and the three unsupervised
ones are gated by agentPolicy.allowUnattended. There is deliberately no
cluster-wide default for them: the authority is write access to a Workspace.
Ports
| Variable | Default | Notes |
|---|---|---|
CALIBAND_PORT | 8443 | caliband's TCP+TLS control port; also the readiness/startup probe target and the ingress port the NetworkPolicy opens. |
CALIBAN_AGENT_PORT_BASE | 7100 | Bottom of caliband's per-agent stream port window. Passed to caliband and opened in the NetworkPolicy from one setting, so the two cannot drift. |
CALIBAN_AGENT_PORT_END | 7999 | Top of that window, inclusive. |
The operator validates these at startup and refuses to start on a port
outside 1-65535, an empty window (BASE > END), or a CALIBAND_PORT that
falls inside the agent window. The error names the offending variable, rather
than letting a NetworkPolicy the API server rejects fail every reconcile.
Session plane
caliband serves TLS and requires a bearer token. Both come from pre-existing Secrets in the task's namespace, mounted or projected by the operator.
| Variable | Default | Notes |
|---|---|---|
CALIBAN_SESSION_TLS_SECRET | caliban-session-plane-tls | Serving-cert Secret with keys tls.crt, tls.key, ca.crt. Mounted read-only at /etc/caliband/tls. |
CALIBAN_SESSION_TOKEN_SECRET | caliban-session-plane-token | Bearer-token Secret. |
CALIBAN_SESSION_TOKEN_KEY | token | Key within the token Secret. |
CALIBAN_SESSION_SERVER_NAME | caliband | The name the serving cert is verified against. Must equal that cert's SAN and the client's TLS SNI, or the workers caliband spawns fail the handshake and their status reports are silently dropped. |
The token is never an argv value — it reaches the pod from the Secret as an
environment variable, so it stays out of kubectl get pod -o yaml and the
process table. The ca.crt is handed to caliband so it can pass a trust anchor
down to the workers it spawns.
Agent pod resources
Defaults give every agent pod Burstable QoS: requests, no limits — so a pod
is not first in line for eviction and is admitted under a ResourceQuota,
without a guessed ceiling OOM-killing an agent. An explicitly empty value
clears a default rather than only overriding it.
| Variable | Default | Notes |
|---|---|---|
CALIBAND_CPU_REQUEST | 250m | |
CALIBAND_MEMORY_REQUEST | 512Mi | |
CALIBAND_CPU_LIMIT | unset | |
CALIBAND_MEMORY_LIMIT | unset |
The git-clone init container gets the same values. A pod's effective request is the max of any init container and the sum of its app containers, so this costs nothing at scheduling while still admitting the init step under quota.
Cluster topology and network reach
| Variable | Default | Notes |
|---|---|---|
CALIBAN_CLUSTER_DOMAIN | cluster.local | The cluster's DNS domain, used to build caliband's advertise host <sandbox>.<namespace>.svc.<domain>. A blank value keeps the default; a trailing root dot is trimmed. |
CALIBAN_INGRESS_POD_SELECTOR | empty | key=value,key2=value2. Narrows the NetworkPolicy's ingress peer to matching pods. Empty admits every pod in the peer namespace. A malformed entry fails startup rather than silently widening or dropping the rule. |
CALIBAN_INGRESS_NAMESPACE | the task's own | Namespace allowed to reach caliband, matched on its kubernetes.io/metadata.name label. |
Unconfigured, ingress to caliband is open to every pod in the task's own
namespace — the historical default-deny-everything-else posture. In a real
deployment, narrow it to prospero: set CALIBAN_INGRESS_NAMESPACE to prospero's
namespace and CALIBAN_INGRESS_POD_SELECTOR to its pod labels.
Observability
| Variable | Default | Notes |
|---|---|---|
RUST_LOG | info | tracing-subscriber env filter. An unset value yields info, not silence. |
CONTROLLER_POD_NAME | unset | Recorded as the instance on every Event the operator publishes. Set it from fieldRef: metadata.name so Events name the replica that wrote them. |
RBAC the operator needs
The operator needs a cluster-scoped role — it watches both CRs across all namespaces. This is the verb set the charts grant:
| apiGroups | resources | verbs |
|---|---|---|
caliban.caliban-ai.dev | calibantasks | get, list, watch, update, patch |
caliban.caliban-ai.dev | calibantasks/status | get, update, patch |
caliban.caliban-ai.dev | workspaces | get, list, watch |
caliban.caliban-ai.dev | workspaces/status | get, update, patch |
"" | secrets | get |
agents.x-k8s.io | sandboxes | get, list, watch, create, update, patch, delete |
"" | serviceaccounts | get, list, watch, create, update, patch, delete |
networking.k8s.io | networkpolicies | get, list, watch, create, update, patch, delete |
events.k8s.io | events | create, patch |
coordination.k8s.io | leases | get, list, watch, create, update, patch, delete |
The leases grant is required by default, because leader election is on by
default — see Leader election. It only needs to be a
namespaced Role in the operator's own namespace, since the lease lives there.
Notes on the shape of that set:
getonsecrets, neverlistorwatch. The operator reads a Secret only by the name aWorkspacereferences, to answer "does this key exist". It never lists or caches Secret data, which is why theWorkspacecontroller re-checks on a requeue instead of watching Secrets.- No
create/deleteoncalibantasksorworkspaces. Clients such as prospero own CR lifecycle; the operator only observes them and writes status. events.k8s.iois required for the Events in the CRD reference. Without it, publishing logs a warning and the reconcile proceeds — Events are best-effort.- prospero needs
patchoncalibantasks/statusof its own to write theAgentsSettledcondition. Without it, tasks never reach a terminal phase. That grant lives in prospero's chart, not the operator's.
Leader election
The operator elects a leader through a coordination.k8s.io Lease before
running either controller, and it is on by default. A single replica simply
acquires the lease immediately; the point of the default is that raising
replicaCount can never silently leave two controllers force-applying the same
objects.
| Variable | Default | Notes |
|---|---|---|
CALIBAN_LEADER_ELECTION | true | Accepts 1/true/yes/on and 0/false/no/off; anything else fails startup rather than being guessed. Set it false only for a single-replica install that manages exclusivity itself. |
CALIBAN_LEASE_NAME | caliban-operator | The Lease object the replicas contend for. Must not be blank. |
CALIBAN_LEASE_NAMESPACE | the operator's own namespace | Read from /var/run/secrets/kubernetes.io/serviceaccount/namespace when unset. The chart sets it explicitly to the release namespace. |
CALIBAN_LEASE_DURATION_SECONDS | 15 | How long the lease is held before it may be taken over. |
CALIBAN_LEASE_GRACE_SECONDS | 5 | How long before expiry the holder starts renewing. Must be below the duration, or the holder could lose the lease while still believing it holds it — startup rejects that. |
POD_NAME | the pod's HOSTNAME, else caliban-operator | The identity this replica claims the lease under. It must differ per replica, or two replicas look like the same holder and both proceed. The chart sets it from fieldRef: metadata.name; the hostname fallback only matters outside a cluster. Note this is a different variable from CONTROLLER_POD_NAME, which only labels Events. |
Operationally:
- Losing the lease stops the controllers and returns the process to standby rather than exiting, so an apiserver blip costs a pause instead of a crash loop. A replica that never wins simply waits, doing nothing.
- A clean shutdown releases the lease rather than waiting out the duration, so failover is prompt.
- Outside a cluster, there is no namespace file: either set
CALIBAN_LEASE_NAMESPACEorCALIBAN_LEADER_ELECTION=false. The startup error says exactly this. - The lease is applied under field manager
caliban-operator, like everything else the operator writes.
Running with CALIBAN_LEADER_ELECTION on but without the leases RBAC above
is the one new way a previously-working install can break on upgrade: the
operator starts, fails to acquire, and never reconciles. Upgrade the chart
before the operator image — the chart's lease Role is harmless against an
operator that predates leader election (an unused Role and env it ignores),
whereas the reverse order fails on lease RBAC. If you keep election off, keep
replicaCount: 1: with election off, nothing stops two replicas from
force-applying the same objects.
In the chart this is the leaderElection block (enabled, leaseName,
durationSeconds, graceSeconds), on by default. Its lease Role is gated
behind both leaderElection.enabled and rbac.create, so rbac.create=false
with election on is the combination to watch.
The operator's own Deployment should run as uid 10001 with
runAsNonRoot: true, allowPrivilegeEscalation: false,
readOnlyRootFilesystem: true and all capabilities dropped. It writes nothing
to disk, so a read-only root filesystem is safe. See
the container image.
Pod identity and the workspace volume
Every container in a sandbox pod — the git-clone init container included — runs
as CALIBAN_AGENT_UID/CALIBAN_AGENT_GID (default 10001, matching the app
user in caliban's image), with runAsNonRoot: true and fsGroup set to the
same gid.
The uid sits on the pod, not on each container, so there is nothing to keep
in sync between the clone step and the agent. fsGroup is what makes a non-root
clone work on any volume rather than by luck: kubelet applies the group to the
mounted workspace with g+rwX, so the clone does not depend on the mode the
provisioner gave the volume root. runAsNonRoot turns a misconfigured uid 0
into an admission failure instead of a silent return to root-owned sources.
CALIBAN_AGENT_UID must match the caliband image's user. caliban's image bakes
HOME and an XDG_RUNTIME_DIR owned by uid 10001; a different uid cannot
write them.
One case this does not repair: the clone is idempotent and skips a source
whose <path>/.git already exists, so a workspace volume cloned by an older,
root-running init container still holds root-owned sources. The init container
is now non-root and cannot chown them back, and git compares the owner uid, so
it refuses the checkout even though fsGroup made the files writable. Recreate
the task, or delete its workspace PVC, to re-clone as the agent.
Troubleshooting
Start with the task's own status and Events — the failure reason is always on the object:
kubectl get calibantask <name> -o jsonpath='{.status.phase} {.status.permissionPosture}{"\n"}'
kubectl describe calibantask <name> # conditions + Events
kubectl get workspace <name> -o jsonpath='{.status.phase}: {.status.message}{"\n"}'
| Symptom | Likely cause |
|---|---|
Operator pod CrashLoopBackOff immediately, log names a variable | Startup validation rejected a port, uid or lease setting. Fix the named variable. |
| Operator runs but reconciles nothing, no errors on any task | It is a non-leader standby, or it cannot acquire the lease. Check the log for "contending for the leader lease" / "acquired the leader lease", then kubectl get lease caliban-operator. A forbidden on leases means the lease RBAC is missing. |
| Two replicas both reconciling | CALIBAN_LEADER_ELECTION=false with replicaCount > 1, or both replicas sharing one POD_NAME so they look like the same lease holder. |
Operator fails at startup naming CALIBAN_LEASE_NAMESPACE | Running outside a cluster, where the ServiceAccount namespace file does not exist. Set that variable or CALIBAN_LEADER_ELECTION=false. |
Workspace stuck Failed | Read status.message — a source path outside the workspace root, a duplicate path, a bad egress CIDR, a duplicate provider name, an unresolvable defaultProvider, or a missing credential Secret key. |
Task Failed, WorkspaceUnresolved | The Workspace is missing or Failed, or providerRef names no provider. Re-checked every 30s. |
Task Failed, PostureNotPermitted | Unsupervised power without agentPolicy.allowUnattended. message names the setting — the task's permissionPosture: unattended, or the Workspace's own permissionMode: dontAsk/bypassPermissions, autoAllow or noPermissions. |
kubectl apply rejects unknown field "spec.agentPolicy.permissionMode" (or spec.model.name) | The installed CRDs predate the field. Upgrade the caliban-crds chart. |
An agentPolicy setting seems ignored | It was unset, not false — unset leaves caliban's own default. Also check the task's pin: status.resolvedWorkspace.agentPolicy is what the running task was admitted with, and editing the Workspace does not re-pin it. |
| A task runs the wrong model | spec.model.name is pinned at admission. Read status.resolvedWorkspace.provider.model; editing the task afterwards does not move a running one. |
Task Failed, InvalidName | The task name is too long for its <task>-sbx Service. Names are immutable — recreate with a shorter one. |
Task stuck Pending | No Sandbox was created. Check the operator's RBAC on sandboxes.agents.x-k8s.io and that agent-sandbox is installed. |
Task stuck Provisioning | The Sandbox exists but is not Ready. kubectl describe sandbox <task>-sbx and check the pod: a pull failure on the caliband or git image, an unschedulable PVC, or caliband not binding its control port. |
Task reaches Running and never leaves | Expected without prospero — the operator cannot observe agent completion. Check prospero is deployed and holds patch on calibantasks/status. |
| Agents report nothing; interactive agents never go idle | A session-plane TLS mismatch. CALIBAN_SESSION_SERVER_NAME must equal the serving cert's SAN. |
| Clone fails with "detected dubious ownership" | A pre-existing workspace volume with root-owned sources. Delete the PVC and recreate the task. |
| Agent cannot reach a git remote or a provider | The Workspace declares spec.egress and the destination is not in the allow-list. Remember DNS is always allowed but the destinations are not. |
caliban-operator container image
The operator ships as a multi-arch container image at
ghcr.io/caliban-ai/caliban-operator (linux/amd64 + linux/arm64).
What's in it
A two-stage build (Dockerfile):
- builder —
rust:1-bookworm,cargo build --release --bin caliban-operator. The exact toolchain comes fromrust-toolchain.toml, which rustup reads from the build context, so the version is written in one place only (#90). (rustls/ring + kube; no openssl/protoc/git2 native deps). - runtime —
debian:bookworm-slim+ca-certificates, running thecaliban-operatorcontroller binary as a non-root user (uid10001, matching the Helm chart'ssecurityContext). The operator writes nothing to disk, so it runs fine under a read-only root filesystem.
The image contains only the controller (caliban-operator), not the crdgen
dev tool. ENTRYPOINT is the binary and it takes no arguments.
Configuration is entirely by environment — there are no flags and no config file. See Deploying and configuring for the full set with defaults; the operator refuses to start on an invalid port window, selector or uid, naming the offending variable. The Helm chart in caliban-ai/helm-charts wires a subset of them and leaves the rest on their compiled defaults.
Build locally
docker build -t caliban-operator:dev .
Publishing (CI)
.github/workflows/release-image.yml mirrors the sibling repos' native
per-arch pipeline (no QEMU):
- Pull requests touching the build inputs build both arches on native
runners (amd64 on
ubuntu-latest, arm64 onubuntu-24.04-arm) with no push — pure validation. v*tags (andworkflow_dispatch) build each arch, push it by digest, then amergejob assembles the multi-arch manifest list and pushes the{{version}}andsha-<sha>tags viadocker buildx imagetools create.
Pin a version tag in a deployment rather than tracking a floating one, so an operator rollout is reproducible.
Cut a release by tagging:
git tag v0.6.1 && git push origin v0.6.1
Afterwards, bump appVersion in the caliban-operator Helm chart to match, and
copy any changed deploy/crd/*.yaml into the caliban-crds chart.
Changelog
All notable changes to caliban-operator are documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning. While the project is pre-1.0, the minor version is bumped for new features and the patch version for fixes.
The operator is not published to crates.io (publish = false). A release is a
container image: tagging a merge commit v* fires release-image.yml, which
publishes ghcr.io/caliban-ai/caliban-operator:<version> for linux/amd64 and
linux/arm64.
Entries before v0.6.0 were back-filled from the release: commit messages that
were, until this file existed, the only record of what each tag contained. They
are therefore summaries at the granularity those commits recorded, not the
per-change detail used from v0.6.0 onwards.
Unreleased
0.7.0 - 2026-10-06
The release that makes a second replica safe, and moves the agent's posture out of
untyped env and into the API.
Leader election is on by default, so replicas: 2 is a standby rather than a race.
A Workspace now declares its agent policy as typed fields the operator validates
and projects — and the settings that leave an agent unsupervised are admitted only
where the Workspace authorizes them, extending ADR 0006's rule to the policy itself.
A task can override just the model. The sandbox pod runs as the agent's own uid, so
clones it writes are usable.
Needs caliban-crds >= 0.2.9 for the agentPolicy fields, model.name and the
state.mode enum, and the Lease RBAC from caliban-operator chart >= 0.2.11.
Upgrading begins enforcing the unsupervised-policy gate: a Workspace that sets
permissionMode dontAsk/bypassPermissions or noPermissions without
agentPolicy.allowUnattended is refused, fail-closed, where v0.6.0 ignored it.
Added
-
Leader election, on by default. The controller acquires a
coordination.k8s.ioLease before running either reconciler and returns to standby if it loses it, so running more than one replica is safe rather than a race between two writers. Configured byCALIBAN_LEADER_ELECTION(default on),CALIBAN_LEASE_NAME,CALIBAN_LEASE_NAMESPACE,CALIBAN_LEASE_DURATION_SECONDSandCALIBAN_LEASE_GRACE_SECONDS; the holder identity comes fromPOD_NAME. Needs the Lease RBAC added in caliban-charts (#48, PR #96) -
Typed agent permission posture on
Workspace.spec.agentPolicygainedpermissionMode,autoAllowandnoPermissions, projected into the agent's environment instead of travelling as untypedenventries. The settings that leave an agent unsupervised —permissionMode: dontAskorbypassPermissions,noPermissions— are admitted only on a Workspace whoseagentPolicy.allowUnattendedis true, which extends ADR 0006's existing authorization rule to the Workspace's own policy (recorded as decision 7 in that ADR) (#51, PR #88) -
Typed agent extension posture on
Workspace.spec.agentPolicyalso gainednoMcp,noHooks,noSkills,noSubAgent,strictKnownMarketplaces,enabledPlugins,blockedMarketplacesandparallelToolLimit. Typed fields take precedence over any same-namedenventry, and list fields project even when empty — an emptyenabledPluginsmeans enable none, which is a different instruction from leaving it unset (#87, PR #89) -
Per-task model override.
CalibanTask.spec.model.nameoverrides the resolved workspace provider's model for one task, applied at the resolve-and-pin step so the override is recorded instatus.resolvedWorkspacelike every other pinned value (#52, PR #91)
Fixed
-
The sandbox pod runs as the agent's uid. The pod spec now carries a
securityContextwithrunAsUser/runAsGroup/fsGroupset to the agent uid (default 10001, configurable viaCALIBAN_AGENT_UID/CALIBAN_AGENT_GID) andrunAsNonRoot. Before this, clones written by the agent were owned by a different uid than the one that had to use them (#79, PR #85) -
An invalid
state.modeis refused at admission. The CRD schema now carriesenum: [remote, local], so the apiserver rejects a bad mode on write — the same momentpermissionPosturealready failed. The enum is in the schema and deliberately not in the Rust type: a task stored before the enum existed may hold any value, and a typed enum would make such an object fail to deserialize entirely, which is worse than the clearFailedstatus it gets now. The reconcile-time cross-field checks (remoteneeds an endpoint,localmust not have one) are unchanged (#94, PR #97)
Internal
-
The Rust toolchain now lives in exactly one place.
rust-toolchain.tomlmoved to 1.99.0 and is the only pin:ci.ymlinstalls withdtolnay/rust-toolchain@stableand theDockerfileusesFROM rust:1-bookworm, both deferring to the file, which rustup honours when cargo runs. Previously the version was written in three places that could drift (#90, PR #98) -
caliban-contractpinned to=0.15.0, in lockstep with prospero (caliban-ai/prospero#266) so the contract's two downstream consumers never disagree about its version. Nothing the operator compiles against changed —launch.rsis the only file differing between the versions. Worth knowing downstream:CalibandLaunch.router_configkept its type but changed meaning, from inline config JSON to a filesystem path, so a caller passing JSON still compiles and silently has it read as a path. The operator passes the mounted ConfigMap path (#95, PR #99) -
Documentation audit against the shipped code: CRD and deploying references brought current, README moved to v0.6.0, and the missing ADR template restored (PR #86)
-
This file, and an mdBook guide published from
docs/guide/. The operator was the only repo in the suite with neither. The guide ingests the ADR log, this changelog and the three reference docs at build time, so each has exactly one editable copy (#92, PR #100) -
Six append-only notes recording where the code has overtaken ADRs 0001, 0002 and 0003 — the
volumeClaimTemplatestype, twoSandboxfields that were never declared, the control-plane TLS environment variables removed in #59, the now-configurable cluster DNS domain,serde_yaml→serde_norway, and the target cluster moving to k3s v1.36. No decision prose was altered (#93, PR #101) -
The guide's published links are now checked. Crawling the newly published site found three 404s on the ADR 0000 page —
template.md,README.mdand a bootstrap-spec reference — each correct as a file on GitHub and dead once rendered, because only files listed inSUMMARY.mdbecome pages. The ADR ingest now rewrites links that leave the book to absolute URLs, protecting the ingested chapters so an in-book../<chapter>.mdstill resolves, and covering both the inline](url)and reference[label]: urlforms. The bootstrap reference points at caliban, where that document actually lives — it had never resolved in this repo.check-links.shruns indocs.ymland resolves every internal href, plus any link back into this repository, against what the build produced; it was proven to fail by re-introducing a dead link. mdBook validates no markdown links, so all three had published with zero build warnings (PR #103, PR #104) -
Changelog catch-up: #100 and #101 were missing from the file #100 created (PR #102)
0.6.0 - 2026-09-19
Per-session permission posture, and caliband's launch surface sourced from the contract crate rather than hand-written strings.
Added
CalibanTask.spec.task.permissionPosture(supervised|unattended), admitted only under aWorkspacewhoseagentPolicy.allowUnattendedis true. The effective posture is reported instatusand shown as aPostureprinter column. Recorded as ADR 0006 (#81)
Changed
-
caliband's flags and environment variable names now come from
caliban-contract0.14.0, and provider credential environment names fromProviderKind, so the operator no longer carries its own copy of caliband's launch surface (#83) -
CRD documentation fixes:
tokenRefprincipal guidance, and an egress example using an RFC 5737 documentation address (#77)
Requires caliban-crds >= 0.2.5 for the permissionPosture / agentPolicy fields.
0.5.0 - 2026-09-16
Terminal phases, workspace egress control, storage settings and Kubernetes Events — 17 PRs.
Added
AgentsSettledresolves to a terminal phase (#75)- Workspace egress allow-list (#74)
spec.statestorage settings plus a gonzalod token (#76)- Kubernetes Events for reconcile outcomes (#73)
- Sandbox resource requests and limits (#62)
- Configurable cluster DNS domain (#71)
- Server-side-apply status writes (#67)
- A watch on owned
Sandboxobjects (#68)
Fixed
- Reconcile, config, RBAC and CRD fixes (#60, #61, #63, #65, #69, #70, #72)
- Documentation corrections (#66, #77)
0.4.0 - 2026-09-13
Interactive tasks, sandbox reachability fixes, and the removal of Ollama.
Added
CalibanTask.spec.task.interactive(#28, PR #29)
Fixed
- Project the provider base URL and API key under provider-native environment variable names (#31)
- Pass the session-plane CA to caliband (#33)
- Make
Workspace.spec.sourcesoptional (#34) - Pass
--tls-server-nameso workers verify against the certificate SAN (#36)
Removed
- Ollama support. Samples, CRDs and tests migrated to
kind: openaiwith an explicitbaseUrl, which covers the same deployments without a dedicated provider kind (#39)
0.3.1 - 2026-07-19
Fixed
- Per-agent stream reachability in Kubernetes.
build_sandboxpasses--advertise-host <sandbox>.<ns>.svc.cluster.localand pins--agent-port-base, and the sandboxNetworkPolicyopens the per-agent stream window (7100–7999) alongside 8443. Without this, prosperod could not attach to a k8s agent's output because caliband advertisedtcp://0.0.0.0:7100(#24, #25, PR #26)
0.3.0 - 2026-07-18
Fixed
build_sandboxlaunches caliband with the shared session-plane bearer token and TLS, so the pod binds instead of crash-looping on "a non-empty bearer token is required" (#22, PR #23)
0.2.0 - 2026-07-12
The configuration plane: a Workspace custom resource, and resolve-and-pin.
Added
WorkspaceCRD (v1alpha1), withCalibanTask.workspaceRef/providerRefresolve-and-pin (#11, PR #12)
Changed
- Breaking (pre-v1). Inline
CalibanTask.spec.workspaceremoved in favour ofworkspaceRef(#11, PR #12) - Gate the
CalibanTaskpin onWorkspacereadiness (#13, PR #18)
Fixed
- CRD schema validation, graceful shutdown, and
serde_yaml→serde_norway(#2, PR #10)
Removed
- Dead
WorkspacePhase::Reconcilingvariant (#14, PR #17) - Orphaned
CalibanTaskStatus.workspacefield (#15, PR #16)
0.1.1 - 2026-07-05
First release that reconciles end to end.
Fixed
volumeClaimTemplatesTypeMeta, so aCalibanTaskreconciles into aSandboxthat agent-sandbox v0.5.0 accepts (#7, PR #8)
0.1.0 - 2026-07-05
Added
- The container image and its multi-arch release workflow. A two-stage
Dockerfile(rust:1.95-bookworm→debian:bookworm-slim+ ca-certificates) builds the controller and runs it non-root as uid 10001, matching the Helm chart'ssecurityContext; the controller writes nothing to disk, so it runs under a read-only root filesystem.release-image.ymlbuilds both arches and publishesghcr.io/caliban-ai/caliban-operatoronv*tags (caliban-ai/caliban#358, PR #5)
Architecture Decision Records
- ADR 0000 · Record architecture decisions
- ADR 0001 · kube-rs stack +
CalibanTaskCRD API - ADR 0002 · Reconcile
CalibanTask→ agent-sandboxSandbox(+ per-task SA & NetworkPolicy) - ADR 0003 · The caliband launch contract (daemon args + workspace init)
- ADR 0004 ·
WorkspaceCRD +CalibanTaskresolve-and-pin - ADR 0005 · Operator owns infrastructure, prospero owns agent lifecycle, caliban owns its contract
- ADR 0006 · Unattended permission posture is authorized by Workspace policy
ADR 0000 · Record architecture decisions
- Status: accepted
- Date: 2026-06-14
Context
caliban has kept Architecture Decision Records since the Layer-0 bootstrap
(ADRs 0001–0047). The original 2026-05-22 Layer-0 bootstrap design
placed them at the repository root in adrs/, reasoning that ADRs are first-class
Layer-0 deliverables and top-level placement makes them impossible to miss.
Since then the sibling repositories adopted the conventional
adr-tools / MADR
layout instead: prospero and gonzalo both keep their records under
docs/adr/, seed the log with a meta "record architecture decisions" entry, and
(prospero) ship a template.md. caliban was the outlier — root adrs/ (plural),
no meta record, no template — which created cross-repo confusion and path
mismatches for anyone moving between the three repos.
There was no ADR stating why caliban records decisions or where they live; that rationale lived only in a feature design doc, which is exactly the kind of external dependency ADRs are supposed to avoid.
Decision
We will keep Architecture Decision Records under docs/adr/, in MADR-lite
format (a lightweight extension of Michael Nygard's original ADR style), matching
sibling repos prospero and gonzalo. Specifically:
- Location:
docs/adr/(singularadr), not the former rootadrs/. This supersedes the root-placement decision in the Layer-0 bootstrap design; existing records were relocated withgit mvto preserve history. - This meta record is numbered
0000so the existing0001–0047numbering is preserved — no renumbering churn, and the log still opens with a record of the practice itself. - Format: each ADR is one append-only file
NNNN-kebab-title.mdwith Context, Decision, and Consequences. A decision is changed by writing a new ADR that supersedes the old one, never by rewriting history. - Template: new ADRs start from
template.md. - Status legend:
accepted/superseded/proposed/rejected, indexed inREADME.md.
Consequences
- Positive: one consistent ADR convention across caliban / gonzalo / prospero;
the conventional, tooling-friendly
docs/adr/location; and the rationale for the practice now lives in an ADR rather than a feature design doc, so it is self-sustaining. - Negative: a one-time churn to relocate the directory and update every inbound
reference (crate rustdoc, README, the mdBook guide, the parity matrix, and the
historical design docs). ADRs no longer sit at the repo root, so they are slightly
less discoverable from a bare
ls— mitigated by a pointer from the top-levelREADME.md. - Revisit if: the agreed cross-sibling ADR standard changes, or the
docs/adr/layout proves harder to maintain than the root placement it replaced.
ADR 0001 · kube-rs stack + CalibanTask CRD API
- Status: accepted
- Date: 2026-07-04
- Source: k8s system-design spec (§"
CalibanTaskCR + workspace model", §"caliban-operator") in the caliban-ai docs hub · caliban #282 · epic #274
Context
The k8s epic (#274) needs a Kubernetes operator that reconciles a CalibanTask
custom resource into a sandboxed caliband pod (via agent-sandbox). The repository
was stood up in #277 as bare scaffolding (an empty caliban-operator crate, AGPL,
Rust 1.95, a fmt/clippy/build/test CI). This ADR records the foundational,
hard-to-change decisions that #282 commits to: the controller stack and the
CalibanTask API surface (group/version/kind), which becomes a compatibility
contract the moment a CR is applied to a cluster.
Constraints and forces:
- The operator is a thin caliban-semantics layer over agent-sandbox — it must not reimplement pods/isolation/warm-pools. So the stack is a controller runtime, not a pod scheduler.
- The target homelab runs k3s v1.31; the CRD and the client must work there,
and the
k8s-openapiversion pin selects the Kubernetes API surface the operator compiles against. - kube-rs pairs
kubewith a specifick8s-openapimajor; the two must be chosen together or the build won't resolve. Picking versions by hand is brittle. - The spec fixes the CR shape (
workspace.sources,task,model,state,isolation,resources,lifecycle; statusphase/calibandEndpoint/sandboxRef/checkpointRef/workspace.materialized/conditions).
Decision
-
Controller stack: kube-rs. Depend on
kube(featuresruntime,derive,client) +k8s-openapi(featurev1_32) +schemars(CRD schema generation) +tokio+serde/serde_json+serde_yaml(CRD YAML emission) +thiserror+tracing+futures. Exact compatible versions are resolved withcargo add(which honors kube'sk8s-openapipairing) and then pinned inCargo.lock; we do not hand-pick the kube/k8s-openapi pair. The resolved pair is kube 4.0 + k8s-openapi 0.28, whose lowest offered API feature isv1_32(there is nov1_31for this pairing).v1_32is the compiled API surface; it is forward-compatible with the target k3s v1.31 apiserver for the objects we use (CRDapiextensions.k8s.io/v1, core objects, and the agent-sandbox CRDs consumed in #283) — the Kubernetes API is tolerant across a one-minor skew for these. -
API:
caliban.caliban-ai.dev/v1alpha1, kindCalibanTask, namespaced, with a status subresource. Defined via#[derive(CustomResource, …)]on aCalibanTaskSpecstruct mirroring the spec's fields, with a separateCalibanTaskStatus.v1alpha1signals the API is unstable and may change without conversion webhooks — appropriate for an MVP; graduation tov1beta1/v1is a later ADR. The status subresource keeps spec (intent) and status (observed) on separate update paths, as Kubernetes convention requires for controllers. -
CRD YAML is generated, not hand-written. A
crdgenbinary emitsCalibanTask::crd()as YAML; the generated CRD is committed (so the helm chart in #284 andkubectl applyconsumers have a stable artifact) and a test asserts the committed YAML is in sync with the Rust types (regenerate-and-diff), so the struct is the single source of truth. -
#282 delivers a no-op reconcile that sets
.status.phase = Pending. The controller skeleton watchesCalibanTask, and its reconcile only initializes status toPendingand requeues — it creates no Kubernetes objects. The real reconcile (CalibanTask→ agent-sandboxSandbox+ RBAC/NetworkPolicy) is #283. Thephaseenum is the spec's lifecycle:Pending → Provisioning → Running → Draining → Completed | Failed.
Consequences
- Positive: the operator has a real, cluster-installable CRD and a running (if
inert) controller — the #282 acceptance — on a conventional, well-supported
Rust k8s stack. The generated-and-checked CRD makes the Rust types authoritative
and gives #284's chart a stable file to package.
v1alpha1buys freedom to iterate the API through P2/P3 without conversion machinery. - Negative:
v1alpha1means the CR shape can break between releases with no automated migration — acceptable pre-1.0, but every CR author is exposed to it. Pinningk8s-openapitov1_32ties the compiled API surface to that release; bumping it later is a deliberate, tested change. Committing generated YAML adds a keep-in-sync test that must be run when the types change (the test is the guard). - Revisit if: the API stabilizes enough to warrant
v1beta1/v1+ a conversion webhook; the target cluster's Kubernetes version moves far enough that thev1_32k8s-openapisurface needs raising (or drops below the tolerated skew with k3s v1.31); or agent-sandbox's own API (consumed from #283) forces a differentk8s-openapipairing. serde_yamlwas replaced byserde_norway— superseded. Decision 1 listsserde_yamlin the stack. It was unmaintained and carried a RUSTSEC advisory, so the dependency is nowserde_norway, its maintained fork, with the same role (CRD YAML generation) and a compatible API. The decision — generate the CRD YAML from the Rust types and commit it — is unchanged; only the crate that does it moved.- The target cluster has moved to k3s v1.36 — context changed. The Context
and decision 2 are written against k3s v1.31, and the "Revisit if" clause
above names the cluster version moving as a trigger. It has: the homelab now runs
k3s v1.36 (see Deploying).
k8s-openapiis still pinned tov1_32, which remains within the tolerated skew and compiles against a surface older than the server — the forward-compatible direction, and the one the decision intended. No change is required today; this is recorded so the version in the Context is not read as current, and so the trigger is visibly accounted for rather than silently passed.
ADR 0002 · Reconcile CalibanTask → agent-sandbox Sandbox (+ per-task SA & NetworkPolicy)
- Status: accepted
- Date: 2026-07-04
- Amended by: ADR 0005 — decision 6 (status gains prospero's
AgentsSettledcondition as an input, via server-side apply field ownership) - Source: k8s system-design spec (§"
caliban-operator", §"agent-sandbox integration") in the caliban-ai docs hub · caliban #283 · epic #274 · builds on ADR 0001
Context
ADR 0001 fixed the controller stack
(kube 4.0 / k8s-openapi 0.28 / v1_32) and the CalibanTask API. #282 shipped a
placeholder reconcile that only initialized .status.phase = Pending. #283 makes
the reconcile real: turn a CalibanTask into a running, sandboxed caliband pod
reachable at a stable DNS, with per-task RBAC and network isolation, and drive
.status from the backing Sandbox.
The spec is explicit that the operator is a thin caliban-semantics layer over agent-sandbox — it must not reimplement pods/isolation. So the reconcile composes existing objects rather than managing pods directly.
Forces:
- agent-sandbox owns the
SandboxCRD (agents.x-k8s.io/v1beta1). We consume it — we must not generate, install, or drift-guard its schema. Verified v0.5.0SandboxSpec={ podTemplate{metadata,spec: core.PodSpec} (required), service bool, operatingMode enum(Running|Suspended)=Running, shutdownPolicy enum(Delete|Retain)=Retain, shutdownTime, volumeClaimTemplates []core.PVC };SandboxStatus.serviceFQDNis the pod's stable DNS. - Reconcile must be idempotent and self-healing — re-running on the same
CalibanTaskmust converge, not duplicate, and children must be garbage-collected when theCalibanTaskis deleted. - Cluster-agnostic (the eventual charts are public): no home-cluster identifiers baked into generated objects.
- k3s v1.31 target vs
v1_32compile surface (ADR 0001): stick to long-stablePodSpec/NetworkPolicyfields. - The caliband pod needs zero Kubernetes API access; its identity exists only to be a NetworkPolicy/PodSecurity subject.
Decision
-
Consume
Sandboxas a foreign typed resource. Hand-author a minimal RustSandboxvia#[derive(CustomResource)]pointed atgroup = "agents.x-k8s.io", version = "v1beta1", kind = "Sandbox", namespaced, status = "SandboxStatus", declaring only the fields we set or read (podTemplate,service,operatingMode,shutdownPolicy,shutdownTime,volumeClaimTemplates; statusserviceFQDN,conditions).podTemplatereusesk8s_openapiPodTemplateSpec;volumeClaimTemplatesreusesPersistentVolumeClaim. We never callSandbox::crd(), emit its YAML, or add it to the drift-sync test — agent-sandbox installs its own CRD. ExtraSandboxSpecfields we omit are pruned by the API server's structural schema; this is a read/write view, not the schema of record. -
Reconcile = pure builders + a thin apply loop. The mapping
CalibanTask → {Sandbox, ServiceAccount, NetworkPolicy}and the status derivation are pure functions (build_sandbox,build_service_account,build_network_policy,derive_status), unit-tested with no cluster. The async reconcile only: server-side-applies each child, reads back the Sandbox, and patchesCalibanTask.status. -
Idempotency via server-side apply + owner references. Children are applied with
Patch::Applyunder field managercaliban-operator(force), so re-reconciles converge on one object. Every child carries a controllerOwnerReferenceto itsCalibanTask, so deleting the task cascades. Child names are deterministic:<task>-sbx(Sandbox),<task>-sa(ServiceAccount),<task>-netpol(NetworkPolicy). -
Least-privilege RBAC = a dedicated, token-less ServiceAccount with no bound Role. The caliband pod is given its own per-task
ServiceAccountwithautomountServiceAccountToken: false, referenced from the podTemplate. Because noRole/RoleBindingis created, the pod holds zero API permissions — strictly the least privilege. An empty Role would be noise; the token-less SA is the RBAC posture. (The operator's own cluster RBAC — the right to create Sandboxes/SAs/NetworkPolicies — is deploy-time chart concern #284, not this reconcile.) -
NetworkPolicy: default-deny with the minimal working allowances. The per-task
NetworkPolicyselects the Sandbox's pod labels and setspolicyTypes: [Ingress, Egress]with: deny-all by default, allow DNS egress (UDP/TCP 53), allow general egress (git clone + provider APIs; the empty-torule allows egress to all destinations at MVP, in-cluster included — restricting it to non-cluster CIDRs is deferred because it would require an environment-specific cluster CIDR, a home-cluster identifier we don't bake in), and allow ingress on the caliband port (8443) from the task's namespace. The spec's finer rules (prospero→caliband, pod→gonzalo) require cross-namespace identity the CR does not carry generically at MVP; they are parameterized in a later iteration. The MVP ships a real, cluster-agnostic default-deny posture (selectors are the pod's own labels — no home-cluster identifiers). -
Status is derived from the Sandbox, not initialized-and-frozen. This evolves #282's init-once no-op:
derive_status(task, sandbox)returnsProvisioningonce the Sandbox is applied but has noserviceFQDNyet,RunningonceserviceFQDNis populated, and setscalibandEndpoint = "<serviceFQDN>:8443"andsandboxRefaccordingly.Pendingremains the pre-provision default (aCalibanTaskobserved before its first successful apply). The status patch is skipped when the derived status equals the observed one (no-op churn avoidance, same discipline as #282). -
Explicit deferrals (out of #283 scope, tracked upstream): the caliban-aware drain finalizer (needs caliband's checkpoint gRPC — deferred with the transport #314), idle→pause/resume mapping,
SandboxTemplate/SandboxWarmPool(Phase 4), and fullSecret/ConfigMapcredential projection. #283 projects only what the acceptance path needs: the workspace PVC, the caliband image + runtimeClass, and the gonzalo endpoint / model-router refs as env, so the pod comes up reachable.
Consequences
- A
CalibanTasknow materializes real cluster objects. Applying one yields a Sandbox (→ caliband pod on the configured RuntimeClass, workspace PVC, stable DNS), a token-less SA, and a default-deny NetworkPolicy;.statusreportsProvisioning → RunningwithcalibandEndpoint. This satisfies #283's acceptance and unblocks prospero'sK8sFleet(#64) reading the endpoint. - agent-sandbox must be installed in the cluster for the applies to succeed
(its
SandboxCRD must exist). This is the cluster prerequisite the umbrella chart bundles by default (helm-charts#6); without it the operator's Sandbox apply fails and the task staysProvisioning— an honest, retried error, not a crash. - The operator needs cluster RBAC to create these children — it can create
sandboxes.agents.x-k8s.io,serviceaccounts, andnetworkpolicies.networking.k8s.io, and patchcalibantasks/status. That ClusterRole ships with the operator chart (#284); this ADR fixes the exact verb set that chart must grant. - Deferring the drain finalizer means a deleted
CalibanTasktears down its Sandbox immediately (owner-ref GC) without checkpointing in-flight agents. This is acceptable for the MVP and revisited when caliband checkpoint gRPC lands. - The foreign
Sandboxview can rot if agent-sandbox changes its v1beta1 schema. We pin the consumed version (v1beta1) and cover the fields we use with round-trip tests against the published CRD's shape; a breaking upstream change surfaces as a deserialize/apply failure, caught in integration. serviceFQDNis not readiness — resolved (#45). Decision 6's "RunningonceserviceFQDNis populated" reportedReady=Trueas soon as the Service object existed, before caliband had bound its port; a crash-looping caliband could present asReadyindefinitely and prosperod dialed a dead endpoint. The caliband container now carries TCP readiness and startup probes on the control port (no liveness probe — a restart kills every supervised agent), andderive_statusgatesRunningon agent-sandbox's ownSandboxReadycondition, which v0.5.0 sets True only once the pod passes its readiness probe, has an IP, and the Service is ready. An FQDN without that condition staysProvisioningwith no endpoint, and a reported not-ready surfaces asReady=False/SandboxNotReadycarrying agent-sandbox's message.- Allow-all egress could not be tightened — resolved (#58). The default
egress rule above still allows every destination, and remains the default. A
Workspace may now declare
spec.egress.allow(CIDR + optional TCP ports), which replaces allow-all with DNS plus exactly those destinations; an empty list is DNS-only. It is a Workspace-level field rather than part of the sharedIsolationSpec, becauseCalibanTaskSpec.isolationis a per-run override that wins over the workspace default and would otherwise let a task widen the restriction. Malformed CIDRs and out-of-range ports fail the Workspace. volumeClaimTemplatescould not reusePersistentVolumeClaim— superseded (#7/#8). Decision 1 says theSandboxview'svolumeClaimTemplatesreusesk8s_openapi'sPersistentVolumeClaim. It cannot: that type serializes its group/version/kind, and agent-sandbox v0.5.0's structural schema rejects those on server-side apply (.spec.volumeClaimTemplates[].apiVersion: field not declared in schema), failing every reconcile.src/sandbox.rsdeclares a bareVolumeClaimTemplate(metadata+spec) instead, which is what the upstream schema actually describes.podTemplatedoes still reusePodTemplateSpecas decision 1 says.- The
Sandboxview omitsshutdownPolicyandshutdownTime— not implemented. Decision 1 lists both among the fields the view declares; it declarespodTemplate,service,operatingModeandvolumeClaimTemplatesonly. Nothing sets a shutdown policy today, and the view is deliberately a read/write subset — fields it omits are pruned by the API server, so the gap is inert rather than a defect. It is recorded here because decision 1 reads as a complete list and is not one.
ADR 0003 · The caliband launch contract (daemon args + workspace init)
- Status: accepted
- Date: 2026-07-05
- Source: caliban #366 (follow-up to #283) · epic #274 · builds on ADR 0002
Context
ADR 0002 reconciled a CalibanTask into a
correctly-shaped Sandbox/SA/NetworkPolicy but explicitly deferred the exact caliband
container contract ("the exact caliband env contract isn't built yet"). A home-cluster
dry run exposed that the deferred contract is load-bearing: the pod as built by #283
would not run a live caliband. Verified against caliban's caliband.rs arg parser and
the caliban image's Dockerfile:
- The image is
ENTRYPOINT ["caliband"],CMD ["--help"]. #283 set no containercommand/args, so the pod runscaliband --help→ prints usage → exits → CrashLoop. - caliband takes
--workspace-root <path>as a required flag (there is no env for it); #283 setCALIBAN_WORKSPACE_ROOTenv, which caliband ignores. - The network switch is
--listen <host:port>(orCALIBAN_DAEMON_LISTEN, formathost:port); #283 setCALIBAND_LISTEN=tcp://…— wrong name and wrong format. CALIBAN_WORKSPACE_SOURCES(an invented env) is read by nothing, and no one clones the workspace sources into the/workPVC, so the workspace is empty.- caliband's
--tls-cert/--tls-key/--tokenare optional, so--listenwithout them is valid plaintext TCP.
Decision
-
Run caliband as a daemon via container
args.build_sandboxsets the caliband container'sargsto["--workspace-root", <workspace_root>, "--listen", "0.0.0.0:<caliband_port>"], replacing the image's--helpCMD (thecalibandENTRYPOINT is kept). This uses only flags caliband actually parses. -
Drop the mismatched env. Remove
CALIBAND_LISTEN,CALIBAN_WORKSPACE_ROOT, andCALIBAN_WORKSPACE_SOURCES(none are read; listen + root are now flags, sources are handled by the init container). KeepGONZALO_ENDPOINT/CALIBAN_ROUTER_CONFIG_REF(consumed by the caliban agent runtime, harmless if unset). -
Populate the workspace with a git-clone init container. Add an init container (image from a new
git_imagesetting, default a publicalpine/git) that mounts the workspace volume and, for eachspec.workspace.sources[], clonesrepoatrefintopath— idempotently (skip if<path>/.gitalready exists, so pause/resume and pod restarts over the persistent PVC don't refetch or fail). Public sources only for now; private-source auth is a follow-up. -
Plaintext TCP for the first e2e. The operator runs caliband with
--listenand no TLS/token. This is sufficient to prove the pipeline (a reachable caliband on the Sandbox DNS) and matches caliban's optional-TLS design. Provisioning per-Sandbox certs- a bearer token (and wiring prospero's matching client) is a tracked security follow-up — the transport already supports it (caliban #280).
Consequences
- Applying a
CalibanTasknow yields a caliband that actually starts, has its workspace cloned, and listens on the Sandbox endpoint — closing the gap between "objects render" and "an agent can run". This unblocks the home-cluster e2e. - Plaintext TCP is a deliberate, temporary posture. The default-deny NetworkPolicy (ADR 0002) still constrains who can reach the port, but the control/stream bytes are unencrypted and unauthenticated until the TLS/token follow-up lands. Not for multi-tenant/production use.
- The init container needs a git image and egress to clone sources — the ADR 0002 NetworkPolicy already allows general egress. Private sources will need credential projection (deferred).
advertise-host/ per-agent ports — resolved (#24/#25). Control-plane reachability (serviceFQDN:<port>) worked, but per-agent attach dialing from outside the pod (prospero's session plane) needed--advertise-host <serviceFQDN>+ per-agent port routing.build_sandboxnow passes--advertise-host <sandbox>.<ns>.svc.cluster.localand pins--agent-port-base, and the NetworkPolicy opens the matching per-agent stream window alongside the control port.- Session-plane TLS needs the CA, not just the keypair — resolved
(#32). The TLS follow-up to
decision 4 landed
--tls-cert/--tls-keybut no--tls-ca, so caliband had no trust anchor to hand down to the workers it spawns. Their Idle/Running reports failed the handshake and were silently dropped, leaving interactive agents stuck inRunning.build_sandboxnow also passes--tls-ca <mount>/ca.crt(the key was already in the mountedkubernetes.io/tlsSecret) and publishesCALIBAN_CONTROL_TLS_CA+CALIBAN_CONTROL_TLS_SERVER_NAMEon the caliband container, which the workers inherit. The verified name comes from the newsession_server_namesetting and must equal the serving cert's SAN —build_status_client'slocalhostdefault never matches in-cluster. caliban #510 replaces the env inheritance with explicit forwarding inExecWorkerLauncher. - The verified name must be on argv, not only in the env — resolved
(#35). Once caliban #510
landed, that explicit forwarding became the problem:
ExecWorkerLaunchersets each worker'sCALIBAN_CONTROL_TLS_SERVER_NAMEfrom caliband's own resolved name, overriding the value the worker used to inherit from the pod env. Absent--tls-server-name, caliband resolves that name from--advertise-host(the per-pod DNS name), which the serving cert cannot prove — its only SAN is the session-plane name. So workers verified against the wrong name, the handshake failed, and interactive agents were still stuck inRunningeven with the CA correctly in place.build_sandboxnow passes--tls-server-name <session_server_name>alongside--tls-ca. The general lesson: when a downstream component forwards a value explicitly, the env var stops being the contract — argv is. git clone --depth 1 --branch <ref>accepts branch/tag names only, not commit SHAs — arefset to a raw commit SHA will fail the clone (the CRD'srefdefaults tomain; SHA-pinning is a follow-up).GONZALO_ENDPOINTwas never consumed — resolved (#41, #53). Decision 2 kept it as "consumed by the caliban agent runtime", but nothing in caliban reads it: caliban reaches a remote gonzalod only through itsstorage.substrate/storage.remote.url/storage.remote.token_envsettings. (CALIBAN_ROUTER_CONFIG_REFwas the same kind of mistake, resolved in #44.) caliban #659 added environment overrides for those settings, andspec.statenow projectsCALIBAN_STORAGE_SUBSTRATE(local→fs,remote→remote),CALIBAN_STORAGE_REMOTE_URL, and, withstate.tokenRef,CALIBAN_STORAGE_REMOTE_TOKEN_ENV=GONZALO_TOKENwithGONZALO_TOKENfrom the referenced Secret. The operator sets caliban's settings by environment rather than authoring its settings file (ADR 0005). Requires a caliban build that includes #659.- The control-plane TLS env vars are no longer published — resolved (#59).
The note above still says the operator "publishes
CALIBAN_CONTROL_TLS_CA+CALIBAN_CONTROL_TLS_SERVER_NAMEon the caliband container, which the workers inherit". It no longer does. caliband ≥ 0.8.0 (caliban#510, #512) sets both on each worker it spawns, explicitly, from its own--tls-ca/--tls-server-name— and the operator passes both flags. Publishing the same values in the pod env was therefore a second mechanism that the argv overrides and that could only drift from it, so it was removed. Inheritance is not how the workers get these values; caliband sets them.caliband_env_carries_no_control_plane_tls_varskeeps them out. - The cluster DNS domain is no longer hardcoded — resolved (#54). The
advertise-hostnote above writes<sandbox>.<ns>.svc.cluster.localas a fixed suffix. The domain is nowCALIBAN_CLUSTER_DOMAIN, defaulting tocluster.local, so the behaviour described is the default rather than the only possibility. A cluster served under another domain no longer gets an unreachable advertise host.
ADR 0004 · Workspace CRD + CalibanTask resolve-and-pin
- Status: accepted
- Date: 2026-07-11
- Source: Prospero K8s Config Plane design (umbrella design doc:
docs/superpowers/plans/2026-07-11-workspace-crd-and-resolve-and-pin.md) · caliban-operator #11 · epic #274 · cross-repo: prospero mirror (prospero#141), chart RBAC (helm-charts#30) · builds on ADR 0001
Context
ADR 0001 put the workspace (sources,
model/provider config) inline on CalibanTask.spec.workspace. Prospero's K8s Config
Plane epic needs a UI where a user edits workspace-level config (git sources, which
model providers are available, which Secret backs each) independently of, and prior
to, launching any task. With the config living only inline on a CalibanTask,
prospero has nowhere to persist it: there is no object to CRUD before a task exists,
and no durable home for it once a task completes and its CalibanTask is deleted.
Prospero's existing k8s-config-plane endpoint has no backing resource to write to and
returns 405 Method Not Allowed for exactly this reason.
The epic's shape for this is explicit: prospero becomes a pure editor of a CRD — it CRUDs Kubernetes objects and lets the operator, not prospero, own reconciliation and credential access. That requires the config to be its own first-class, independently-lifecycled resource, and it requires prospero to never need Secret read access (the operator remains the only component that reads Secret values — a pre-existing, load-bearing security boundary, not a new one introduced here).
Forces:
- Config must outlive any single task and be editable without one. Sources and
provider bindings are workspace-scoped, shared across many
CalibanTasks, and meaningfully exist before the first task is created. - Multiple named providers, not one. A workspace may bind a cheap/local model
(e.g. a local
openai-compatible server such as llama.cpp) for routine work and a stronger hosted model (e.g. Anthropic) for harder tasks; a task picks one by name at submission time, not at workspace-authoring time. - Prospero must never read Secrets. Its CRUD surface is the
Workspace/CalibanTaskspecs only;credentialsRefcarries a Secret name + key, never a value, so the config plane can be edited without grantingsecretsRBAC to prospero. - Immutable-run guarantee must survive the split. #283 and #366 (ADRs 0002–0003)
established that a running task's config is stable for its lifetime. Moving
provider config out of the task's own spec and into a separately-editable
Workspacemust not let a config edit reach into a task that's already running. - Pre-
v1alpha1API, no migration machinery. As in ADR 0001,v1alpha1buys the freedom to reshape the CR without a conversion webhook; this ADR spends that budget on the workspace split rather than deferring it.
Decision
-
A first-class
WorkspaceCRD (caliban.caliban-ai.dev/v1alpha1, namespaced, status subresource, shortnamecws) holds the durable, shared config:sources(the git checkouts), a namedproviderslist (each{name, kind, baseUrl, model, credentialsRef}— acredentialsRefis a by-name Secret+key reference, never a value), an optionaldefaultProvider, non-secretenv, and defaultisolation.WorkspaceStatuscarriesphase(Pending → Ready | Failed— validation is synchronous, so there is no observable intermediate reconciling state),conditions,observedGeneration, and a human-readablemessage. The CRD YAML is generated from the Rust types and golden-tested in sync, per ADR 0001's discipline. -
A pure validation function backs a lightweight status-only reconciler.
validate_workspace(spec, secret_present: Fn(&str, &str) -> bool)checks unique provider names, a resolvabledefaultProvider, and that everycredentialsRefnames an existing Secret key — the existence check only. The controller suppliessecret_presentby calling the Kubernetes API; the operator is the sole reader of Secret contents in the system, unchanged from before this ADR but now made structural: nothing else needssecretsRBAC because nothing else touchesWorkspacereconciliation. The controller's only side effect is patchingWorkspace.status— it creates no pods, no children; provisioning stays entirely in theCalibanTaskreconcile (ADR 0002). -
CalibanTaskreferences aWorkspaceby name and resolves-and-pins at admission.CalibanTaskSpecgainsworkspaceRef: { name }(required) andproviderRef: Option<String>(which named provider to bind; falls back to the workspace'sdefaultProvider, then to the sole provider if there's exactly one).resolve_workspace(spec, provider_ref)is a pure function that flattens the chosen provider and the workspace's sources/env/isolation into a singleResolvedWorkspace, erroring on a danglingworkspaceRef/providerRefor an ambiguous default. TheCalibanTaskreconcile calls it once and pins the result intoCalibanTaskStatus.resolvedWorkspace; if that field is already set, laterWorkspaceedits are not re-resolved into it — the pod is always built from the pinned snapshot, preserving the immutable-run guarantee across the split. -
Inline
CalibanTask.spec.workspaceis removed, not deprecated. Per ADR 0001'sv1alpha1bet, this is a breaking, pre-v1 change with no conversion shim: existingCalibanTaskCRs must be re-created against the new schema (workspaceRef/providerRef) with a companionWorkspaceapplied first. There is no dual-write or fallback path to the old inline shape. -
Sample CRs are the cross-repo contract fixture.
deploy/samples/workspace.yamlanddeploy/samples/calibantask.yamlare committed, mutually-consistent examples (the sample task'sproviderRefnames a provider the sample workspace defines) and are golden-tested here (workspace::tests::sample_manifests_deserialize— both deserialize against the Rust types andresolve_workspacesucceeds against them). Prospero's mirror types are validated against these same fixtures (prospero#141), so this is the shared fixture two repos check against independently.
Consequences
- Prospero gets a durable object to edit and no path to Secrets. Workspace
authoring (sources, provider bindings) no longer requires an existing task, and
prospero's CRUD surface never needs
secretsRBAC —credentialsRefis name-only. This is the change that turns the405from the epic's Context into a real endpoint. - The immutable-run guarantee is preserved, now across two objects instead of
one. A running task's pod is built from
status.resolvedWorkspace, pinned once; editing the backingWorkspace(rotating a provider, changing a source) affects only tasks admitted after the edit, never one already running. This mirrors ADR 0002's status-derivation discipline (compute, compare, patch only on change) applied to a resolve instead of a derive. - This is a breaking API change with no migration. Every existing
CalibanTaskin any cluster must be replaced (workspaceRef+ a companionWorkspace), and every caller (prospero, any manual manifests) must move off inlinespec.workspacein lockstep with this operator version. Acceptable pre-1.0 per ADR 0001'sv1alpha1bet; would not be acceptable post-graduation. - The operator's own cluster RBAC needs a
Workspace/workspaces/statusgrant. This is a small addition to the ClusterRole ADR 0002 already required forCalibanTask; the actual chart change is cross-repo and deferred to helm-charts#30, not addressed here. - Multi-namespace tenancy remains deferred.
workspaceRefis a same-namespace, by-name reference (mirroringWorkspaceRef's singlenamefield); aCalibanTaskcannot bind to aWorkspacein another namespace. Cross-namespace workspace sharing, if ever needed, is a later ADR. - Prospero's mirror types can rot independently of this repo. The sample CRs are the shared contract, but there is no compile-time link between this repo's Rust types and prospero's — a schema change here that isn't reflected in prospero#141's mirror surfaces only at prospero's own test run, not here.
ADR 0005 · Operator owns infrastructure, prospero owns agent lifecycle, caliban owns its contract
- Status: accepted
- Date: 2026-09-13
- Source: caliban-operator #37 design
discussion · follow-ups: caliban #656 (contract
crate), prospero #228 (
AgentsSettled), helm-charts #52 (RBAC), caliban-operator #64 (server-side apply status) · amends decision 6 of ADR 0002
Context
Three components cooperate around a CalibanTask:
- caliban-operator reconciles the CR into a Sandbox, ServiceAccount, and NetworkPolicy, and derives
statusfrom Kubernetes objects (ADR 0002). It launches caliband by writing its pod spec (ADR 0003). It has no network client for caliband. - prospero dials caliband over TLS + bearer token to spawn, attach, stream, and poll agents (
list()). - caliband supervises the agents inside the sandbox.
caliban's contract with the other two is implicit and copied by hand. The operator types caliband's flag and env names as strings in src/resources.rs; prospero mirrors caliban's control wire types in its own wire.rs, deliberately not depending on caliban-supervisor because that crate is the whole daemon. Each drift has surfaced only at runtime, silently: provider env names caliban never read (#30), a missing CA and then a wrong verified server name that kept agents from ever reporting Idle (#32, #35), and a router-config env name nothing read (#44).
#37 exposed the next step of this pattern. A CalibanTask never leaves Running because only caliband's agent list knows an agent finished — the pod stays up. The obvious fix, having the operator poll caliband, would make the operator a third hand-coupled consumer of caliban's wire protocol (a TLS client, token handling, protocol version lockstep), alongside prospero, which already has all of that.
Decision Drivers
- Minimize the number of components that speak caliban's control protocol.
- Turn contract drift from silent runtime failure into a compile-time error.
- Keep one owner per status field; no component may clobber another's writes.
- Preserve sandbox isolation (sandbox pods stay token-less).
Considered Options
- Operator polls caliband via
caliban-supervisoror a mirrored client. Single status writer, but a third protocol consumer and a heavy daemon dependency. - caliband writes its own lifecycle into the CR. Protocol-free for everyone else, but the sandbox pod would need a Kubernetes API token and RBAC, weakening isolation.
- caliband exits when its agents finish (pod
Succeeded/Failed). Fully Kubernetes-native, but only fits one-shot tasks, not interactive ones. - Split responsibilities, meet in the CR via server-side apply field ownership, and publish the contract (chosen).
Decision
We will draw the component boundaries as follows:
- The operator owns infrastructure. It manages the Sandbox, pod, ServiceAccount, and NetworkPolicy, and derives infrastructure state (
Pending/Provisioning/Running,calibandEndpoint,sandboxRef, itsReadycondition) from Kubernetes objects only. It does not become a caliband client. - prospero owns agent lifecycle and is the only component that speaks caliband's control protocol. It reports agent state into the CR as an
AgentsSettledcondition: settled when no agent isSpawning/Running/Idle; reasonFailedif any endedFailed/Crashed, elseSucceeded. - They meet in
CalibanTask.statusthrough server-side apply field ownership. Each component applies only the fields it owns under its own field manager (caliban-operator,prospero);conditionsis a map-list keyed bytype. The operator mapsAgentsSettledtophase: Completed/Failedwithout knowing the wire protocol. This amends decision 6 of ADR 0002, which derived status from the Sandbox alone. - caliban owns and publishes its contract as a thin
caliban-contractcrate with no daemon internals: a typed caliband launch builder (flags and env) and the control wire types, with an optional feature-gated client. The operator builds caliband's launch arguments from it; prospero replaces its mirroredwire.rswith it. ADR 0003's launch contract is unchanged in substance — only where its names are defined moves.
Out of scope: Draining/checkpoint lifecycle (#42, #43), which follows the same split when implemented.
Consequences
Positive
- The operator never takes a caliband client, TLS stack, or protocol version dependency; only prospero speaks the wire.
- Renamed or removed caliband flags, env names, and wire fields become compile errors in the consumers instead of silent runtime failures.
- Status has explicit per-field ownership; merge-patch null/empty-array workarounds in the operator go away.
kubectl get calibantaskscan distinguish live tasks from finished ones (#37) without weakening sandbox isolation.
Negative
- Terminal phase depends on prospero running; without it, finished tasks stay
Running(today's behaviour, no worse). - Cross-repo sequencing: caliban (contract crate, caliban#656), prospero (
AgentsSettled, prospero#228), helm-charts (prosperopatchoncalibantasks/status, helm-charts#52), and the operator (server-side apply status, #64; phase mapping, #37). - Consumers upgrade the contract crate in step with caliban releases — explicit coupling that previously existed implicitly.
Revisit if
- A second agent-lifecycle observer appears alongside prospero, or prospero stops being part of every deployment.
- caliban gains a Kubernetes-native completion signal that works for interactive tasks.
- Server-side apply field ownership on
statusproves unreliable across the Kubernetes versions we support.
ADR 0006 · Unattended permission posture is authorized by Workspace policy
- Status: accepted
- Date: 2026-09-18
- Source: caliban-operator #80,
amended by #51 (decision 7) ·
caliban ADR 0059 (per-session permission posture), caliban
#676 (
SpawnSpec.permission_posture) · prospero ADR 0010 (inbound API scopes) · builds on ADR 0004 (resolve-and-pin) and ADR 0005 (component split)
Context
caliban now runs each agent session under a permission posture chosen per session:
supervised: a permission prompt goes to a human, who answers it.unattended: the session runs under a bypass profile with no human in the loop.
The default is supervised, so the system fails closed. In Kubernetes the posture a human chose travels on the CalibanTask CR. prospero reads it and puts it on the agent's SpawnSpec. The operator owns the CRD, so it owns the field and its admission rules.
Running unattended is a privileged act. prospero enforces this at its own API: only an admin-scoped token may ask prospero for an unattended session. That check does not cover the cluster. Anyone with create on calibantasks in a namespace can write the CR directly and skip prospero entirely. The operator therefore needs its own gate, one that doesn't depend on who created the CR.
Considered Options
- Trust the CR as written. Whoever can create a task can make it unattended. This is the same as having no gate:
createoncalibantasksis granted far more widely than the right to run agents unsupervised. - Workspace policy. Add
Workspace.spec.agentPolicy.allowUnattended, default false. Anunattendedtask is admitted only under a Workspace that sets it. The authority is write access to Workspaces, which is already the privileged object that holds providers, credentials references and egress. - Admission-time identity check. A ValidatingAdmissionPolicy (CEL on
request.userInfo) restricts which principals may createunattendedtasks. This is precise, but it ships in helm-charts, needs a group or ServiceAccount convention the charts don't define yet, and does nothing for tasks created before the policy existed.
Decision
We will adopt option 2, and leave option 3 as a possible addition later.
CalibanTask.spec.task.permissionPostureis an optional enum,supervised | unattended. Unset meanssupervised.unattendedis admitted only when the referenced Workspace setsagentPolicy.allowUnattended: true. A missing policy, or a pin made before the field existed, does not permit it.- The check runs before pinning. A denied task is not pinned, is marked
Failedwith reasonPostureNotPermittedand a Warning Event, and is re-checked on a short interval. Allowing it on the Workspace, or editing the task back tosupervised, recovers it; the operator never silently downgrades it. - The policy is pinned into
status.resolvedWorkspacewith the rest of the Workspace config, as with every other part of the Workspace (ADR 0004). A pinned task is checked against its pin on every reconcile. RevokingallowUnattendeddoes not disturb a task that is already running. Editing a running task's spec tounattendedfails it if the pin doesn't allow that. - The posture is visible.
status.permissionPosturerecords the posture a task was admitted with, and appears as a printer column. A NormalUnattendedAdmittedEvent names the Workspace whose policy allowed it. - The operator does not hand the posture to caliband. prospero maps it onto
SpawnSpec.permission_posture(ADR 0005). allowUnattendedauthorizes the Workspace's own unsupervised settings too (amended for #51). #51 makes caliban's permission surface typed on the sameagentPolicyblock:permissionMode,autoAllow,noPermissions. Three of those settings —permissionMode: dontAsk,permissionMode: bypassPermissions, and either switch set totrue— leave no human answering prompts, which is the power decision 2 gates. A Workspace asking for one of them is admitted only when it also setsallowUnattended: true; otherwise the task is denied before pinning, exactly as in decision 3, with the reason naming the setting. Without this, the same power would have two doors: one gated, one not. Unlike a task's posture, the policy cannot change under a running task — it travels inside the pin — so it is checked at admission only.
Consequences
Positive
- A task can't run unattended unless a Workspace author allowed it, however the task was created. prospero's
adminscope and this gate cover the two ways in. - Unattended tasks show up in
kubectl get calibantasks(thePosturecolumn) and in Events. - Uses the existing resolve-and-pin model; no new admission infrastructure.
- One switch is the whole answer to "may agents here run unsupervised?", covering both a task's posture and the Workspace's own permission settings, so an auditor reads one field rather than four.
Negative
- The gate is only as strong as Workspace RBAC. Anyone who can edit a namespace's Workspace can allow unattended tasks there. Deployments must treat Workspace write access as privileged.
- Revoking
allowUnattendedonly affects new admissions; a running unattended task keeps its pin until it finishes or is deleted. - Within a Workspace that allows it, the gate doesn't limit which principals may create unattended tasks.
Revisit if
- Tenants need unattended access limited to specific users within a Workspace. At that point, add the admission-time identity check (option 3) in helm-charts.
- Revocation must stop running tasks, not just block new ones.