prospero

Orchestration layer for caliban.

This guide is under construction. See the architecture decisions for design rationale, and the API reference for crate docs.

Changelog

All notable changes to prospero are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning. While the project is pre-1.0, the minor version is bumped for new features and the patch version for fixes.

Unreleased

0.5.0 - 2026-08-22

Replaces the dashboard. The hand-written vanilla-JS page is superseded by a Rust → WASM single-page app that shares the server's own types, and it now serves /; the old page remains at /v1, deprecated. Everything an operator previously needed v1 for — launching, killing, replying to an interactive agent, editing a workspace's config, watching a live stream — is in v2, along with things v1 never had: a per-agent timeline with a tool-call inspector, and fleet-wide spend, turns, and outcome charts over a selectable window.

The other half of this release is the k8s fleet telling the truth about what it ran. Terminal outcomes were structurally invisible under PROSPERO_FLEET=k8s: the watch loop derived agent status from the CalibanTask phase, and the operator never advances that past Running, so the new outcome charts read a permanent zero — including for runs that had demonstrably failed. Outcomes are now taken from the pod, recorded once per agent, and survive a restart.

Upgrade note. GET / now serves dashboard v2 rather than v1, and GET /app.js is gone — v1's script moved to /v1/app.js along with the page. /v2 remains a permanent alias for the new dashboard, so existing links to it keep working.

Added

• Dashboard v2 — a Rust → WASM single-page app, and now the dashboard. A Dioxus SPA in crates/dashboard, compiled to wasm32-unknown-unknown and embedded in prosperod, replacing the hand-written vanilla-JS page. The read model and the control-plane DTOs are the same Rust types the server uses, so the client cannot drift from the API the way a hand-maintained JS copy did. The crate sits outside the cargo workspace on purpose (a wasm-only crate would break the host-target build and sink the coverage floor) and has its own CI job; its built bundle is committed and include_bytes!'d, so an ordinary cargo build still needs no wasm toolchain and one binary still ships the UI (#97).
• Full operator control from the dashboard. Launch, kill, remove, respawn, interactive input, end-input, and workspace removal — every action gated on /api/capabilities, so the UI never offers an operation the active backend would answer with a 405 (#173).
• Workspace registration and configuration from the dashboard, with two form shapes chosen at runtime: the local single-provider/env form, and the k8s Workspace-CR editor with sources, named providers, and Secret references, plus the reconciliation-status pill and a provider picker on launch. This also fixed a lossy read: GET /api/workspaces was projecting the CR's source spec down and dropping the git remote and ref, so editing a k8s workspace meant retyping every remote from memory (#175).
• Live agent stream viewer — replay history from the store, then tail over SSE. A reconnect resumes at last_seq + 1 and anything at or below the high-water mark is dropped, which is the defence against the v1 reconnect storm that duplicated the timeline unboundedly (#105); a close after AgentFinished reads as "finished" rather than an error, so a healthy completed run does not sit there retrying (#178).
• Per-agent timeline with a tool-call inspector. Tool calls pair start with finish into a collapsible entry, consecutive output coalesces, and the opening context and final accounting become a header and a summary. Pairing is on the tool id and never the name — caliban's ToolCallEnd carries the id but leaves the name empty, which is what left every tool stuck "running" in v1 (#106) (#179).
• GET /api/usage — cost, turns, and terminal outcomes aggregated per workspace per UTC day over a window (since/until, or days). Computed by the store rather than by replaying the log, with identical semantics across the JSONL, SQLite, and Postgres backends, held there by a shared conformance suite (#180).
• Fleet overview charts for spend, turns, and outcomes over a selectable 24h / 7d / 30d window. Outcomes are faceted rather than stacked: measured against the real tokens, no ordering of a four-way stack is separable in light mode, so one single-hue chart per outcome is the fix rather than a mitigation. Hand-rolled SVG with native <title> tooltips — the page is served under default-src 'none', so there is no script and nothing for the CSP to refuse (#181).
• An explicit theme setting (System / Light / Dark), persisted in localStorage, replacing "whatever prefers-color-scheme says". The explicit choice wins in both directions, and the theme is applied before first paint so there is no flash of the wrong one (#183).

Changed

• The control-plane DTOs moved to prospero-types and now carry both Serialize and Deserialize. They previously lived in prospero-api, which pulls axum and tokio and so compiles for no wasm target, and each carried only the server's half of the contract — so a WASM client would have had to hand-duplicate all eight, reintroducing exactly the drift Rust/WASM was chosen to avoid. They are re-exported from their original paths and the serde output is unchanged, so this is additive for existing consumers (#172).
• Dashboard v2 is now the default surface. GET / serves the Dioxus/WASM dashboard; /v2 stays mounted as a permanent alias, since the bundle's own asset URLs are absolute /v2/... and existing links point there. The scaffold deliberately parked v2 at /v2 so / stayed untouched while epic #95 landed — that transition is complete (#191).

Deprecated

• The v1 dashboard has moved to /v1 and is deprecated. It renders a notice pointing at /, and its script now lives at /v1/app.js (GET /app.js is gone). It is kept rather than removed so an operator hitting a v2 regression has somewhere to land, but it receives no further work and carries defects v2 was built to fix — most visibly #106, where a tool call whose finish frame has a blank name stays "running" forever. Removal is a follow-up once v2 has a release of real-world use (#191).

Fixed

• Restarting prosperod no longer re-counts outcomes it already recorded. A fresh process starts with an empty view of the fleet and re-derives the terminal transition it had already written before the restart, so every outcome facet doubled on each restart — measured on a live cluster as 3 done / 3 failed becoming 6 / 6 with nothing having run. The watch loop now checks the durable log (which outlives the process) before recording an agent's outcome, so an agent that finished once is counted once (#196).

• Terminal outcomes are now derived from the pod, not the CalibanTask phase. #190 made the watch loop persist the transitions it observed, but it observed status.phase, and the operator never advances that past Running — CRs whose agents finished a day earlier still read Running, so no terminal transition ever occurred and the outcome facets stayed at zero on a real cluster. The loop now applies the same pod-caliband overlay snapshot() has always used, so the component that emits events and the one that renders them finally agree. Pod dials are bounded and concurrent, and an agent is consulted only until it is observed terminal (#194).

• An unreachable pod can no longer stall fleet observation. The status overlay dialled each pod sequentially with no deadline. A pod that black-holes its SYN (rather than refusing it) blocked until the OS connect timeout, which was survivable when only GET /api/fleet did this and is not now that the watch loop depends on it. Dials are concurrent and bounded; a miss retries on the next poll (#194).

• Terminal outcomes are now recorded under PROSPERO_FLEET=k8s. The usage aggregate counts done/failed/killed/crashed from persisted status_changed events. The local arm has always emitted them, but the k8s watch loop computed the identical transition diff and only broadcast it in-memory — so on k8s every outcome facet in the dashboard read zero, including for agents that had demonstrably failed. The loop now persists the transition it observes, elected by a single-writer observer lease so replicas don't multiply the counts (#190).

• The dashboard's usage panel refreshes on its own. It fetched once per window selection and never again, so a finished agent's spend could sit invisible until the operator toggled the window by hand. It now refetches when the fleet poll observes an agent appear, change status, or disappear, and on a one-minute heartbeat — without re-running a 30-day store aggregate on every five-second poll (#190).

• A k8s workspace's provider base URL survives an edit. The v2 editor had no base-URL input and ProviderInfo did not carry the field, so reopening the editor showed a blank box and saving wrote it back — a routine model edit silently unpicked a self-hosted provider, after which agents died instantly with a ProviderError against localhost (#188).

• k8s-spawned agents get the resolved provider and model. spawn_spec_from_task sent provider/model as None on the premise that pod env would drive selection; the caliban worker selects from SpawnSpec and nothing else, so every k8s agent fell back to caliban's default and died at preflight with ANTHROPIC_API_KEY is not set. The operator already pins the resolved provider into status.resolvedWorkspace, so its kind and model are projected onto the spawn. ensure_pod_agent also no longer attaches to an agent already in a terminal state — caliband keeps a failed agent in its registry with the endpoint it advertised, but the worker died before binding that port, turning a one-shot failure into a silent reconnect loop (#169).

• Re-submitting an identical prompt no longer claims a launch that didn't happen. Spawning is idempotent and the k8s CalibanTask name is derived from the spec, so an identical prompt resolves to the run already in flight. POST /api/workspaces/{name}/agents now reports created, and the dashboard says it attached to the existing run instead of "Launched" (#190).

0.4.0 - 2026-07-27

Makes the PROSPERO_FLEET=k8s fleet interactive. Since 0.3.0 a k8s agent could be launched from the dashboard but never talked to: the reply box renders only for an agent that is both interactive and idle, and under the k8s backend neither could ever be true. Closing that took a field on the authoritative CRD (caliban-operator#28) plus both halves of the round-trip here, and turned up a second defect — 0.3.3's agent-id decoupling had quietly broken reply delivery and the 0.3.2 status overlay. Local behavior is unchanged.

Deploy note: requires the caliban-crds chart at >= 0.2.1. Older CRDs have no spec.task.interactive, so the API server prunes the field at admission and the flag never reaches the pod.

Added

• Interactive agents under PROSPERO_FLEET=k8s. The dashboard's interactive: true now survives the CR round-trip: build_calibantask writes spec.task.interactive (the field caliban-operator#28 added to the authoritative CRD) and spawn_spec_from_task reads it back into the SpawnSpec sent to the pod's caliband. Previously the flag was silently dropped at the CR boundary and the spawn spec hardcoded interactive: false, so a k8s agent could never await input and the dashboard reply box (interactive && idle) could never appear. Requires the caliban-crds chart at >= 0.2.1, or the API server prunes the field at admission (#163).

Fixed

• Operator replies and the interactive/idle overlay now target caliband's own agent id. #159 decoupled the agent id caliband assigns from prospero's CR name, but send_input still attached by CR name (404, reply lost) and overlay_pod_status still looked records up by id (silent miss, regressing #130's reply box). send_input now resolves the pod's agent via ensure_pod_agent, and the overlay matches each pod's record by endpoint rather than by id. The pre-existing tests used one string for both ids, so they passed while the real path was broken; the new tests use distinct ids (#163).

0.3.3 - 2026-07-19

Completes the PROSPERO_FLEET=k8s control plane so a spawned agent actually runs: the previous two fixes let the caliband pod bind and decoupled spawn from reconcile, but nothing ever started the agent inside the pod. Local behavior is unchanged.

Fixed

• k8s agents now actually start (and stream). Under the k8s backend, spawning created the CalibanTask CR and a caliband pod but never started the LLM run — caliband is a passive supervisor that begins an agent only on CtlRequest::Spawn, and prospero's session plane only attached, so the attach looped forever on agent not found. prospero now spawns the agent in the pod's caliband (list-or-spawn, one agent per pod) from the CalibanTask prompt, then attaches — idempotent across poll cycles and replicas (ownership-lease-gated). Because caliband assigns the agent id itself, the attach id is decoupled from prospero's stream key (the CR name) so output still streams to the dashboard under the identity /stream expects (#159) (#160).

0.3.2 - 2026-07-18

Two fixes to the PROSPERO_FLEET=k8s control plane found in live-cluster use: spawning an agent no longer blocks the dashboard on the operator's reconcile, and an agent's interactive reply box now appears under the k8s backend. Local behavior is unchanged.

Fixed

• Spawning an agent no longer hangs the dashboard on reconcile. Under the k8s backend, K8sFleet::ensure_agent applied the CalibanTask CR and then synchronously polled (up to ~30s) for status.phase == "Running" before returning, coupling the HTTP response to the full CR → operator reconcile → pod schedule → Running chain (and blocking the entire budget when the pod never started). It now returns as soon as the CR is admitted; the background watch loop surfaces the agent and attaches its session when it reaches Running — the synchronous poll was redundant with that path (#157).
• The interactive reply box now appears for k8s agents. The dashboard shows it only when an agent is interactive and idle, and under PROSPERO_FLEET=k8s neither was sourced correctly — the status/interactive fields are now read from the pod's caliband rather than the CalibanTask CR alone (#130) (#156).

0.3.1 - 2026-07-14

Bug-fix follow-up to the 0.3.0 Kubernetes config plane, closing the four issues surfaced in k8s smoke testing on a fresh 0.3.0 deploy. The PROSPERO_FLEET=k8s fleet now stays Ready through a schema-skewed custom resource, surfaces the registered Workspace CRs it manages (instead of a synthetic phantom), and rejects an unregisterable workspace up front. Local behavior is unchanged.

Fixed

• A single un-deserializable CalibanTask no longer wedges the whole fleet. K8sFleet's watch/readiness path listed CalibanTasks strictly, so one CR that failed to deserialize (e.g. a stale task predating the now-required workspaceRef field) failed the entire poll — the fleet never populated, /readyz stuck at 503, and the pod never became Ready. The list is now decoded per-item, skipping and logging the bad CRs (#148) (#152).
• The k8s fleet snapshot reconciles with the Workspace registry. GET /api/fleet synthesized a single phantom k8s workspace and never read the registered Workspace CRs, so a registered workspace was invisible in the dashboard while the synthetic k8s entry reported workspace not registered. The snapshot now surfaces the registered Workspace CRs (agents grouped by the workspace they reference), so /api/fleet and /api/workspaces agree and a fresh deploy shows no phantom (#149, #151) (#153).
• Add-workspace rejects an invalid workspace as 400, not a raw apiserver 422. The dashboard's + add workspace posted a Workspace with empty providers/sources, which the CRD (minItems: 1 on both) rejected — so a workspace could never be registered from the dashboard. The config plane now validates at least one well-formed source and provider before apply (add and edit paths), and the form validates the same client-side (#150) (#154).

0.3.0 - 2026-07-12

The Kubernetes config plane: deploying with PROSPERO_FLEET=k8s is now a real control plane — create and configure workspaces, and launch provider-bound agents, from the dashboard — instead of a read-only viewer that returned 405 Method Not Allowed on Save. Workspaces are first-class Workspace custom resources reconciled by caliban-operator, and the dashboard is backend-aware. Local behavior is unchanged.

Added

• Kubernetes config plane (core + API). Under PROSPERO_FLEET=k8s, K8sFleet now wires a FleetAdmin over operator-owned Workspace custom resources, so POST / PUT / DELETE on /api/workspaces persist and manage real configuration — multi-source workspaces, a named-provider list, and per-provider credentials referenced by Kubernetes Secret name (prospero never reads the Secret) — instead of returning 405. A backend-neutral WorkspaceConfig DTO lets one API serve both backends (local projects its single-provider subset, unchanged); GET /api/workspaces returns the real Workspace CRs with reconciliation status; async workspace writes answer 202 Accepted; and a spawned agent binds a named provider via providerRef (#142) (#144, #145).
• Backend-aware dashboard. The dashboard fetches GET /api/capabilities and adapts. On k8s it renders a workspace editor (git sources + a named-provider list with secretName / key Secret references and a default marker), reconciliation status pills (pending / reconciling / ready / failed with the failure message on hover), and a launch-modal provider picker; on local it is byte-for-byte unchanged (#143) (#146).
• GET /api/capabilities — a backend capability seam the dashboard gates its controls on (#99) (#101).
• Frontmatter / agent-template support through spawn — a spawn can forward an agent-template markdown file to caliband's SpawnSpec.frontmatter_path (#6) (#102).
• Guiding Principles & Invariants guide page synthesizing ADRs 0002–0009 (#74) (#104).

Changed

• The CalibanTask CRD mirror moved from an inline workspace to a workspaceRef (plus an operator-pinned status.resolvedWorkspace), matching caliban-operator's frozen v1alpha1 contract. Pre-v1; existing cluster CRs are recreated under the new schema.

0.2.0 - 2026-07-11

Kubernetes high-availability, a reworked dashboard, and a full QA sweep. A second QA pass over the real prospero/caliband stack filed 23 findings; all are fixed here, alongside first-class leader election for the k8s fleet backend and a new agent-timeline dashboard.

Added

• Leader election + attach lifecycle for the K8sFleet backend. The session-plane attach — the one path that writes an agent's events to the shared store/bus — is now gated on a per-agent ownership lease, so with 2+ prosperod replicas exactly one replica owns, attaches to, and emits each agent (no more duplicate SSE events or racing per-stream seq allocation). Standalone is unchanged (SelfOwnsAll); a clustered deploy builds a LeasedOwnership lease plus heartbeat. Attach tasks are now promptly torn down on stop/remove/restart, and any agent observed Running — including operator- or peer-created ones — is streamed by the lease owner (#108, #112, #113) (#138).
• Dashboard agent timeline, tool-call inspector, and run header — a folded event timeline with expandable tool-call segments and a per-run turns/outcome header (#5) (#96).
• prospero-types crate — the normalized FleetEvent/model DTOs extracted into a small, wasm-compatible serde-only crate the WASM dashboard can share (#98) (#100).

Changed

• Under PROSPERO_FLEET=k8s, prosperod no longer builds a local FleetManager/poll loop; the k8s backend serves directly over the shared store/bus (#83) (#92).
• /readyz now reports workspaces_total/workspaces_healthy/ workspaces_unreachable (was repos_*), and user-facing error wording says "workspace" not "repo", matching the vocabulary used everywhere else (#116, #117) (#135).

Fixed

• Dashboard. Terminal-agent SSE streams no longer reconnect-storm into an unbounded, duplicated timeline with runaway memory (#105) (#128); tool calls resolve ok/fail instead of showing "running" forever (paired by tool_use_id) (#106) (#131); the fleet summary shows the workspace count, the misleading $0.0000 cost is gone, and a favicon is served (#115, #109, #119) (#134).
• API. Duplicate workspace registration returns 409 Conflict, not a misleading 503 (#111) (#139); an unknown agent's events endpoint returns 404 instead of 200 [] (#118) (#135); api_key_from_env on a keyless provider is rejected at config-set time, and rm no longer races a just-spawned agent or lags the fleet view (#120, #122, #123) (#137).
• k8s hardening. The session-plane bearer token is never sent over plaintext (#107) (#133); unrecognized CalibanTask phases map to a terminal state, calibandEndpoint is validated, lock poisoning can't wedge the fleet view, the token compare is constant-time, and --fleet-backend k8s on a non-k8s build fails before any side effects (#114, #121, #125, #126, #127) (#136).
• Tests. De-flaked the distributed_bus PG suite under parallel shared-DB load (#110) (#129) and cli_drives_the_full_stack (#85) (#94).

0.1.1 - 2026-07-05

Fixed

• The released image now builds prosperod with --features k8s, so the K8sFleet backend is compiled in and PROSPERO_FLEET=k8s works. Previously the image only ran the local backend, so an in-cluster deploy showed an empty fleet (#90). Unblocks the k8s-fleet-backend support in the prospero Helm chart.

0.1.0 - 2026-07-04

Initial containerized and licensed release of the prospero control plane — the agent orchestration layer that sits above many caliband supervisors — as part of the P0 Kubernetes deployment (epic caliban-ai/caliban#274).

Added

• ghcr.io/caliban-ai/prospero:0.1.0 — multi-arch (linux/amd64 + linux/arm64), non-root container image running prosperod (REST + SSE + dashboard on 7878); also tagged :latest and :sha-<commit>.
• Helm chart charts/prospero in caliban-ai/helm-charts, rendering standalone (SQLite + PVC) or clustered (external Postgres, N replicas) from one topology value.

Changed

• Repository relicensed to AGPL-3.0-only, matching its sibling projects.

Guiding Principles & Invariants

Prospero's design philosophy is recorded in the architecture decisions. This page synthesizes those decisions into the guiding principles, the inviolable invariants, and the scale-out roadmap — the why behind the code, in one place. It complements the ADR log; it does not replace it. Every item cites the ADR(s) it derives from; on supersession, keep this page in sync.

Scoping note. The unit a caliband manages is a workspace of 1..N repo sources, not a single repo. Per-repo is only today's implementation (caliband identity = hash(repo_root)), being generalized in lockstep by caliban #281 (supervisor) and prospero #72 (discovery). Read "per-repo" phrasings below as transitional.

Guiding principles

1. Control plane, not re-implementation. Prospero owns fleet lifecycle (spawn / kill / respawn / attach) and adds only the fleet-wide concerns caliban lacks — it never re-implements the agent runtime. The mechanism sits behind the FleetProvider trait: Local drives an existing caliband over the wire; K8s realizes the same verbs declaratively as CRUD on CalibanTask custom resources. (ADR 0002, ADR 0008)

2. Couple through the wire, nothing else. Caliban's NDJSON wire format (and, for k8s, the CalibanTask CRD's serialized form) is the only contract. Prospero mirrors the serde types and depends on no caliban crate — so the two projects evolve independently and integration breakage surfaces as a data-shape test, not a compile error. (ADR 0003)

3. Hybrid observability: live + durable, unified. Status comes from polling List; detail comes from attach-on-demand; and every normalized event is appended to durable storage, so an agent's history survives after it finishes. Live and durable views are one read path. (ADR 0004)

4. Normalize caliban's frames into a stable internal type. Raw stream-json frames become FleetEvent / FleetSnapshot; consumers never see raw frames. The normalizer is forward-compatible — an unknown frame is skipped and logged, never fatal. (ADR 0003, ADR 0004)

5. Safe-by-default isolation. Worktree isolation is the default for every spawn; sharing the working tree is an explicit opt-out (--shared-tree). Isolation stays per-source even as scoping moves to the workspace. (ADR 0005)

6. Enforce policy at the boundary, in one place. Defaults like worktree isolation are set at the API boundary, so every client (CLI, dashboard, future callers) inherits the same policy from a single control surface. (ADR 0005, ADR 0006)

7. Layered, one-directional crate boundaries. cli / daemon → api → core, acyclic; no web framework leaks into core's public API. Shared read-model DTOs live in a wasm-compatible leaf crate so the (native) server and a (WASM) dashboard cannot drift. (ADR 0006)

8. Abstraction behind traits for deferred evolution. Persistence sits behind a Store trait (realized as jsonl / sqlite / Postgres); fleet control sits behind FleetProvider (Local / K8s). Traits are the vehicle for scale-out and workspace scoping without touching call sites. (ADR 0004, ADR 0008)

Inviolable invariants

These hold across every backend and topology; a change that breaks one is a design change, not a refactor.

• No caliban crate dependency. The wire format / CRD serialized form is the sole coupling. (ADR 0003)
• Acyclic layers. Dependencies flow one way — cli/daemon → api → core — and core's public API names no web framework. (ADR 0006)
• Durability before divergence. A normalized event reaches durable history; if an append fails, that gap is itself recorded so live and durable views never silently disagree. (ADR 0004)
• Unknown frames never crash. The normalizer tolerates unrecognized frames by skip-and-log. (ADR 0003)
• Isolation is the default, opt-out is explicit. No spawn shares the working tree unless a caller says so. (ADR 0005)
• Backends are interchangeable behind the trait. Local and K8s implement the same FleetProvider verbs and emit to the same observability plane; the API request path is backend-agnostic. (ADR 0002, ADR 0008)
• The fake is a faithful double. The control plane is testable end-to-end against an in-process fake caliban, so backends are correct by construction, not by hope. (ADR 0007)

Scale-out roadmap

The trait seams above exist so prospero can grow along three independent axes without disturbing the request path.

• Backend: Local → K8s. LocalFleet drives a caliband over Unix sockets; K8sFleet realizes the same FleetProvider verbs as CRUD + watch on CalibanTask CRs, which the caliban-operator reconciles into sandboxed caliband pods. Both emit to the shared Store/EventBus. (ADR 0008)
• Scope: per-repo → workspace. A caliband manages a workspace of 1..N source checkouts; today's per-repo identity (hash(repo_root)) is being generalized in lockstep with caliban #281 and prospero #72.
• Topology: standalone → clustered. Standalone runs sqlite + an in-process bus + self-owned streams; clustered runs a Postgres store/config, a LISTEN/NOTIFY event bus, and leased stream ownership so replicas fail over without double-writing. Both sit behind the Store / EventBus / Ownership seams. (ADR 0004)

License

Prospero is licensed AGPL-3.0-only. (ADR 0009)

Architecture Decision Records

• ADR 0001 · Record architecture decisions
• ADR 0002 · Prospero is a control plane over caliband, not a re-implementation
• ADR 0003 · Couple to caliban only through its NDJSON wire format
• ADR 0004 · Hybrid live + durable observability behind a Store trait
• ADR 0005 · Worktree isolation by default for agent spawns
• ADR 0006 · Layered crate boundaries: cli/daemon → api → core
• ADR 0007 · Test the control plane against an in-process fake caliban
• ADR 0008 · K8sFleet — a Kubernetes FleetProvider backend
• ADR 0009 · License prospero under AGPL-3.0-only

ADR 0001 · Record architecture decisions

• Status: accepted
• Date: 2026-06-13

Context

Prospero's architectural decisions — the coupling boundary to caliban, the observability model, the crate layout, the testing strategy — have so far lived in PR descriptions, commit messages, design docs under docs/superpowers/, and chat. Those sources answer what the system does but scatter the why. Design docs cover whole features at once and go stale; commit messages are hard to discover after the fact. As the project grows and more people touch it, reconstructing the rationale behind a decision means archaeology across several places.

We need a durable, discoverable, append-only record of significant decisions that survives independently of any one feature's design doc.

Decision

We will keep Architecture Decision Records (ADRs) in this repository under docs/adr/, one decision per file, named docs/adr/####-topic.md (zero-padded, monotonically increasing number + kebab-case slug).

Each ADR records the context, the decision, and its consequences in a short, lightweight format (see template.md). ADRs are immutable once Accepted; a decision is changed by writing a new ADR that supersedes the old one rather than by editing history. The process is documented in README.md.

We are seeding the directory with records for architectural decisions already made and documented elsewhere (ADRs 0002–0007), so the practice starts with real content rather than an empty convention.

Consequences

• Positive: the rationale behind significant decisions has a single, version-controlled home that outlives individual design docs and PRs. Reviewers gain a lightweight place to record "why" during normal development. ADRs are additive and immutable, so the decision log only grows; superseded records stay in place with a pointer forward, preserving the full history.
• Negative: one short file per significant decision, and the team must remember to write an ADR when a decision is architecturally significant. The README.md gives the "when to write one" bar to keep this from degrading into either noise or neglect.
• Revisit if: the practice degrades into noise (trivial decisions getting ADRs) or neglect (significant decisions going unrecorded) — a sign the "when to write one" bar needs tightening or the format is too heavy to sustain.

ADR 0002 · Prospero is a control plane over caliband, not a re-implementation

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §1

Context

Caliban already ships caliband, a per-repo supervisor daemon that spawns, lists, kills, respawns, and attaches to background agents over a Unix-socket NDJSON protocol. Prospero's job is to launch, manage, and observe many agents across many repositories at once.

We could either (a) re-implement process supervision ourselves to own the full stack, or (b) build a layer above the existing calibands that delegates supervision to them and adds only the fleet-wide concerns caliban lacks.

Decision

Prospero is a control plane. It discovers and drives the existing per-repo caliband daemons and does not re-implement process supervision. Prospero sits above many calibands and adds what they individually lack:

• a fleet-wide model aggregating agents across repos and hosts,
• durable run history (caliband exposes only live state),
• a normalized event type independent of caliban's wire frames,
• the observability/control surfaces: CLI, HTTP/JSON API, SSE, and a minimal dashboard.

Consequences

• Positive: we avoid duplicating — and diverging from — caliban's supervision logic; spawn/kill/respawn/attach semantics stay defined in one place. Prospero's value is concentrated on the genuinely new concerns (fleet aggregation, history, normalization) rather than re-litigating process management.
• Negative: Prospero depends on a running caliband per managed repo and inherits its behavior and limitations (Discovery can autostart a caliband on demand to soften this), and the boundary between the two systems must be defined precisely — see 0003.
• Revisit if: caliban's supervision proves too limited for fleet needs, or a concern we treat as fleet-wide turns out to belong inside the per-repo daemon — either would move the control-plane / supervisor boundary.

ADR 0003 · Couple to caliban only through its NDJSON wire format

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §3, §4

Context

Prospero needs to talk to caliband: send control requests (List, Spawn, Attach, Kill, …) and read per-agent stream-json frames. Caliban implements this in Rust crates (e.g. caliban-supervisor) that Prospero could depend on directly to reuse the request, reply, and SpawnSpec types.

Depending on caliban's crates would tie Prospero to caliban's internal Rust API, version cadence, and transitive dependencies — coupling far wider than the bytes actually exchanged on the socket.

Decision

The caliband wire format is the only contract. Prospero owns a thin NDJSON client in prospero-core (CalibandClient) with its own mirrored serde types (CtlRequest / CtlReply / AgentRecord / SpawnSpec) and newline-delimited framing over tokio::net::UnixStream. Prospero does not depend on caliban-supervisor or any other caliban crate.

Consequences

• Positive: Prospero and caliban evolve independently; the only thing that must stay compatible is the bytes on the wire, which is also the surface real deployments depend on. Because the coupling is just a socket protocol, the entire control plane can be tested against a fake that speaks the same protocol — see 0007. Unknown/forward-compatible frames are tolerated by the normalizer (skip-and-log), so caliban can add frame types without breaking Prospero.
• Negative: the wire types are mirrored, so a protocol change in caliban requires a corresponding edit in Prospero's client — an intentional, explicit seam rather than a silent transitive break, but a seam someone must remember to keep in sync.
• Revisit if: the wire protocol churns fast enough that hand-mirroring the types becomes a recurring source of drift — a generated client or a shared schema crate might then earn its coupling cost.

ADR 0004 · Hybrid live + durable observability behind a Store trait

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §1, §4, §5

Context

Caliban exposes only live state: it can List agents and stream a stream-json tail while an agent is active, but it keeps no history — once an agent finishes, its story is gone. Prospero must show both a cheap fleet-wide status overview and a per-agent detail stream, and that detail must survive after the agent (and caliband's memory of it) is gone.

Options ranged from pure polling (cheap, but no streaming detail and no history) to attaching to every agent's stream continuously (rich, but expensive and still no history).

Decision

Adopt a hybrid model:

• Poll each caliband's List on an interval for cheap, fleet-wide status reconciliation.
• Attach to a per-agent stream on demand — while the agent is active or a client is watching — and stop when it is terminal and unwatched (work stays proportional to active + watched agents).
• Normalize both onto an in-memory FleetSnapshot and a normalized FleetEvent type, and also append every event to durable storage behind a Store trait. The first implementation is JsonlStore (append-only JSONL log + registry persistence).

A client that starts watching gets replay from the Store then a live tail from the broadcast bus, joined on a monotonic seq — so "observe" means live + history, unified, and the full story persists on disk even after caliban forgets the agent.

Consequences

• Positive: Prospero fills caliban's history gap — runs are durable and replayable, and seq survives prosperod restarts. Streaming cost is bounded to active/watched agents rather than the whole fleet. Putting persistence behind a Store trait keeps the door open for a sqlite backend later without touching the rest of the system; JsonlStore is deliberately the simple first step.
• Negative: durability is best-effort relative to liveness — a failed Store.append is logged and metered but does not stop live SSE, so we favor a never-down fleet view over guaranteed persistence (first stab). Log retention/rotation is deferred.
• Revisit if: dropped-event durability becomes unacceptable (history is relied on as a system of record), or JsonlStore outgrows append-only files — either would promote the sqlite backend the Store trait was designed to allow.

ADR 0005 · Worktree isolation by default for agent spawns

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §1, §2, §5

Context

A core use case is running several parallel agents on the same codebase at once — multiple streams of work under one repo's caliband. If those agents share a single working tree, their concurrent edits collide and corrupt each other's work. Caliban's SpawnSpec exposes an isolation_worktree flag that gives each agent its own git worktree.

The question is the default: do agents share the tree unless told otherwise, or get an isolated worktree unless told otherwise?

Decision

Worktree isolation is the default for every spawn. Each agent gets its own git worktree so concurrent edits on one codebase don't collide. Sharing the working tree is an explicit opt-out via the --shared-tree flag (which sets isolation_worktree: false).

This default is enforced at the API boundary: POST /api/repos/{repo}/agents defaults isolation to worktree, so the CLI, the dashboard, and any future client inherit the safe behavior without each re-deciding it.

Consequences

• Positive: the common case — parallel agents on one repo — is safe by default; a user has to go out of their way (--shared-tree) to opt into shared-tree behavior and its collision risk. Enforcing the default at the API boundary, not per client, keeps the policy in one place and makes "spawn defaults to a worktree" a testable invariant of the control plane.
• Negative: each isolated agent consumes a git worktree (disk + setup cost) — acceptable for the parallelism it buys, but real cost for many or short-lived agents.
• Revisit if: worktree setup cost dominates for high-churn or single-agent workloads, or a use case emerges where shared-tree is the safe common case — either would argue for a different default or a per-repo policy.

ADR 0006 · Layered crate boundaries: cli/daemon → api → core

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §4

Context

Prospero is a CLI, a long-running daemon, an HTTP/SSE API, and an orchestration engine. Put in one crate, the web framework, transport, and process concerns would bleed into the domain logic, and tests of the core engine would drag in axum and a running server.

We need crate boundaries that keep the orchestration brain independent of how it is exposed.

Decision

Split the workspace into four crates with one-directional dependencies:

prospero-cli (prospero)  ─┐
                          ├─▶ prospero-api ─▶ prospero-core
prospero-daemon (prosperod)┘

• prospero-core — the orchestration brain: domain model, CalibandClient, discovery, registry, Store, FleetManager. No web framework in its public API.
• prospero-api — an axum adapter (REST + SSE + dashboard assets) over FleetManager; depends only on core.
• prospero-daemon (prosperod) — process entry: owns the tokio runtime, config, logging, shutdown; wires core + api into a server.
• prospero-cli (prospero) — a thin client that talks to prosperod over HTTP, not a second protocol. Nothing depends on the daemon.

Consequences

• Positive: the core engine is testable with no HTTP server and no web types in scope, and api is testable in-process over a fake-backed FleetManager. One control surface, not two: the CLI and the dashboard both go through the HTTP API, so there is a single place where control/observe semantics live. The acyclic, one-way dependency graph keeps responsibilities from leaking upward (e.g. transport concerns can't seep into the domain model) and makes the boundaries easy to reason about as the system grows.
• Negative: four crates plus the one-way rule impose a structure cost — types that span layers must be placed deliberately, and an in-process call becomes an HTTP round-trip for the CLI rather than a direct function call.
• Revisit if: the crate split adds more ceremony than it prevents leakage (e.g. constant re-exports across boundaries), or a client genuinely needs a control path that HTTP can't serve well — either would pressure the layering.

ADR 0007 · Test the control plane against an in-process fake caliban

• Status: accepted
• Date: 2026-06-05
• Source: docs/superpowers/specs/2026-06-05-prospero-framework-design.md §7

Context

End-to-end testing of Prospero would normally require a real caliband, real agents, API keys, and live LLM calls — slow, non-deterministic, expensive, and awkward in CI. Yet the behavior worth testing (spawn defaults, poll reconciliation, attach/normalize, replay-then- tail, resilience to dropped streams and refused sockets) is exactly the control-plane logic that sits between Prospero and caliban.

Because the only coupling to caliban is the NDJSON wire format (0003), we can substitute anything that speaks that protocol.

Decision

Build an in-process fake caliban as the cornerstone of the test strategy: a harness (shipped in prospero-core behind a testkit feature) that listens on a real Unix socket and speaks the same NDJSON control + per-agent stream protocol as the real daemon. Tests drive the real FleetManager / CalibandClient against this fake.

This enables deterministic, end-to-end testing of the whole control plane — including the CLI-through-HTTP path — with no real caliban, no API keys, and no LLM calls.

Consequences

• Positive: the full stack is tested fast and deterministically in CI; scripted frames make event sequences, resilience cases, and seq recovery reproducible. Pure units (framing, normalizer, discovery resolution, store, reconciliation) are tested directly; the fake is reserved for integration-level behavior that needs the protocol.
• Negative: the fake must track the wire protocol — if caliban's protocol drifts, the fake and the mirrored client must be updated together (the same explicit seam noted in 0003). Tests against a real caliban binary + live model remain out of scope for the first stab (manual / CI-gated later), so the fake covers the wire contract, not caliban's own correctness.
• Revisit if: the fake and the real caliban diverge in behavior the wire contract doesn't capture (green tests, broken integration) — that gap would justify a real-caliban smoke test in CI alongside the fake.

ADR 0008 · K8sFleet — a Kubernetes FleetProvider backend

• Status: accepted
• Date: 2026-07-04
• Source: k8s system-design spec (§"prospero changes", §"The two planes") in the caliban-ai docs hub · prospero #64 · epic caliban#274 · builds on prospero #71/#75 (caliband network transport) · relates to 0003, 0006, 0007

Context

ADR 0006 put fleet control behind the FleetProvider trait (prospero #63); LocalFleet (caliband-over-Unix) is the only backend. The k8s epic needs a second backend, K8sFleet, that drives a fleet by CRUD + watch on CalibanTask custom resources — the caliban-operator (caliban #283) reconciles each CalibanTask into a sandboxed caliband pod exposing a stable DNS endpoint — and connects the live session plane to that pod over the network.

The network transport this needs already landed in prospero #71/#75 (caliband/transport.rs): CalibandClient can now dial a caliband over TCP + rustls TLS + a bearer-token preamble (connect_tcp), spawn/attach return an Endpoint, and AgentHandle.endpoint: Endpoint carries a Unix path or a host:port. So K8sFleet composes an existing transport; it does not build one.

What remains for K8sFleet:

• a client-side CalibanTask type and a kube client to CRUD/watch it;
• the four FleetProvider methods mapped onto CR operations;
• a session-plane bridge that dials each agent's Endpoint::Tcp (Sandbox DNS) over #75's transport and feeds prospero's existing event bus + store, so the dashboard/SSE work unchanged;
• reuse of the 0007 conformance suite, which is Unix-FakeCaliband-coupled and must be generalized to a fake backend.

Decision

1. Mirror a minimal CalibanTask type; do not depend on the caliban-operator crate. Per ADR 0003's "couple only through the wire" principle (here, the CRD's serialized form), declare a minimal kube::CustomResource (caliban.caliban-ai.dev/v1alpha1) in prospero-core carrying only the fields K8sFleet sets (workspace.sources, task.prompt, optional isolation) and reads (status.phase, status.calibandEndpoint, status.sandboxRef). A golden test pins it against a sample CR. The operator's CRD is the source of truth; the mirror is kept minimal to limit drift.

2. K8sFleet implements FleetProvider over CalibanTask CRs. New prospero-core module behind a k8s cargo feature (so LocalFleet-only builds pull no kube).

   • ensure_agent(spec) → server-side-apply a CalibanTask (deterministic name from a hash of the spec, so it is idempotent); await status.phase = Running + status.calibandEndpoint; return AgentHandle { endpoint: Endpoint::Tcp(calibandEndpoint) }.
   • watch_fleet() → a kube::runtime::watcher on CalibanTask → translate applied/deleted + phase transitions into FleetChange::{Discovered,StatusChanged,Gone} (map Phase → AgentStatus), seeded by an initial list.
   • stop_agent(id, drain) → delete the CalibanTask (the operator's owner-ref GC tears down the Sandbox); Graceful best-effort awaits deletion within the timeout.
   • restart_agent(id) → delete + re-apply (fresh name → fresh id).

3. The session plane dials the agent Endpoint over #75's transport and feeds the existing bus + store. K8sFleet carries its own attach task built on CalibandClient::connect_tcp + the shared stream normalizer + Emitter, so /stream SSE and history work unchanged (they read the bus/store, never a socket — ADR 0004). TLS root + bearer token come from operator-injected config (env/Secret; Sandbox DNS is the host). The attach-loop core is refactored out of FleetManager into a provider-agnostic helper if cheaper than duplicating.

4. Generalize the conformance suite behind a FakeBackend trait. Replace fleet_provider_conformance(provider, fake: &FakeCaliband) with (provider, backend: &dyn FakeBackend) where FakeBackend { received_any_spec(); simulate_reap(id) }. FakeCaliband implements it trivially (its existing received_specs/remove_agent); a new in-memory FakeK8s implements it for K8sFleet. LocalFleet's existing conformance run is unchanged. This keeps ADR 0007's "test the control plane against a fake" property for both backends.

5. Backend selection at the daemon edge. prosperod chooses LocalFleet vs K8sFleet by config/env (mirroring caliban's --database-url topology switch — e.g. PROSPERO_FLEET=local|k8s + namespace/kubeconfig). The API layer's remaining direct FleetManager calls (kill/respawn/steer/snapshot) are an ADR 0006 P1 limitation tracked separately; K8sFleet MVP wires the four provider methods + the session plane.

Consequences

• prospero gains a Kubernetes fleet backend — kubectl-less fleet control via CalibanTask CRs, with live streaming over the pod network. This completes the epic's "two planes" for the k8s path (declarative CRs + real-time session over #75's transport).
• prospero takes its first kube/k8s-openapi dependency, scoped to prospero-core behind the k8s feature; LocalFleet builds and runs with no cluster.
• A second mirrored seam (the CalibanTask type vs the operator's CRD) joins the wire mirror of ADR 0003 — the same manual-sync tradeoff, kept minimal and golden-pinned.
• The conformance suite becomes backend-agnostic, so future backends (remote — prospero #1) reuse it for free.
• Deferred: gRPC (caliban #314); rerouting the API's direct FleetManager calls through the provider seam; warm pools / multi-tenant (epic P4). The finalizer-drain / checkpoint pairing waits on caliban checkpoint gRPC.

ADR 0009 · License prospero under AGPL-3.0-only

• Status: accepted
• Date: 2026-07-05
• Source: prospero #73 · in force since the prosperod image work (P0 #62, workspace.package.license = "AGPL-3.0-only") · relates to 0002

Context

Prospero shipped its workspace with license = "AGPL-3.0-only" (set alongside the prosperod container image, P0 #62) but never recorded why — the decision lived only in a Cargo.toml field and a commit message. Prospero is a network-facing control plane (0002): its primary deployment shape is a long-running daemon (prosperod) plus a dashboard that users reach over the network, increasingly in-cluster (the k8s epic, 0008). A license needs recording as an ADR so the choice — and its obligations — are explicit for contributors and operators.

Options weighed:

• AGPL-3.0-only. Strong copyleft whose §13 network clause extends the source-availability obligation to users who interact with a modified version over a network, not only to those who receive a binary. Matches the rest of the caliban-ai stack (caliban, gonzalo), which is AGPL.
• A permissive license (MIT/Apache-2.0). Maximizes adoption and frictionless embedding, but lets a network operator run a modified prospero as a service without ever sharing changes — the exact gap AGPL closes, and the wrong default for a network control plane.
• GPL-3.0-only. Copyleft but without the network clause, so a hosted (SaaS) prospero could diverge privately — strictly weaker than AGPL for this deployment shape.

Decision

We will license all prospero crates under AGPL-3.0-only, matching the caliban-ai stack. workspace.package.license = "AGPL-3.0-only" is the source of truth; every crate inherits it via license.workspace = true. Network use of a modified prospero carries the AGPL §13 source-availability obligation.

This ADR records a decision already in force; it changes no code.

Consequences

• Positive: the copyleft obligation reaches network users of a modified prospero (its dominant deployment), keeping hosted forks source-available; the whole stack (caliban/gonzalo/prospero) shares one coherent license story; the rationale is now discoverable, not buried in a manifest field.
• Negative: AGPL deters some downstream adopters (notably orgs with policies banning AGPL dependencies), narrowing embedding/commercial-integration reach; contributors must be comfortable with copyleft.
• Revisit if: a dual-licensing or commercial-exception model is needed for adoption, or the stack's license stance changes — at which point this ADR is superseded rather than edited (ADRs are immutable once accepted).