Crate gonzalo_soak 

HA soak harness for gonzalo (#52): drive N stateless gonzalod replicas over a shared S3 backend (RustFS) under replica-kill chaos, and assert gonzalo’s core invariant — concurrent edits are never silently lost.

The crate is split into pure-logic pieces (unit-tested without any backend) and integration pieces that need a live S3 backend + gonzalod binary:

• oracle — the safety/liveness invariant checker (pure).
• dispatch — round-robin + failover across live replicas (pure, mockable).
• target — the S3 endpoint from the env, or skip (pure).

Integration pieces (replica, workload) are added on top and exercised by tests/ha_soak.rs (the bounded per-PR gate) and the gonzalo-soak binary (the deep soak), both of which skip/error unless a S3 target is set.

Modules

dispatch

The replica dispatcher — the harness’s stand-in for the k8s Service.

harness

Ties the pieces together: spawn gonzalod replicas over the S3 backend, build a Dispatcher over them, run the workload while injecting replica-kill chaos, and check the oracle. Used by the bounded gate (tests/ha_soak.rs) and, with a longer chaos loop, by the gonzalo-soak binary.

oracle

The soak safety/liveness oracle.

replica

A set of real gonzalod subprocesses, all backed by the same S3 store — the “N stateless replicas behind a Service” of the k8s HA model.

target

The S3 target for the soak, resolved from the environment.

workload

The concurrent write workload and result collection for the oracle.