Struct KnowledgeStore 

pub struct KnowledgeStore<S, V, E> { /* private fields */ }

Composes a [Store], a [VectorIndex], and an [Embedder] into one retrieval surface keyed by RecordKey.

Implementations

impl<S: Store, V: VectorIndex, E: Embedder> KnowledgeStore<S, V, E>

pub fn new(store: S, index: V, embedder: E) -> Self

pub async fn open(store: S, index: V, embedder: E) -> Result<Self>

Open over an index that may already hold vectors, rebuilding the chunk counts from it.

new starts those counts empty, which is right for a fresh in-memory index and wrong for a durable one: a re-ingest that shrinks a record would not know how many chunks to remove, leaving orphans that still match queries (#150). KeyPrefix cannot narrow below a collection, so this scans once here rather than per ingest.

pub fn store(&self) -> &S

Borrow the underlying store (e.g. to put records before ingesting them).

pub async fn ingest(&self, key: &RecordKey) -> Result<bool>

Ingest the record at key: split it into per-kind chunks, embed each, and index it under a derived per-chunk key. Returns Ok(false) (without indexing) if the record is absent or its kind is definitionally not knowledge-bearing; returns Err if a knowledge-bearing body fails to parse — a corrupt record is a real failure, not “not indexable” (#139).

One batch per record, not one call per chunk (gonzalo#326). On a durable index every upsert is a full OCC commit — re-serialise the shard, upload it as a blob, write a new manifest revision — so a three-chunk record used to advance the manifest three times and leave two superseded shard blobs for gc. Every chunk is now embedded first, then written with one [VectorIndex::upsert_many]. A consequence of embedding first: if an embedding fails part way, nothing is written at all, where before the earlier chunks were already committed.

The orphan removals below are not batched yet. Both candidate shapes — a remove_many on the trait, or folding removals into the upsert commit — add a VectorIndex method, and cargo publish --workspace --dry-run verifies each packaged crate against the published gonzalo-vector, so a new trait method cannot be called from this crate until the workspace version is bumped. Orphans only occur when a record shrinks, so the common path commits once either way. Tracked separately.

pub async fn remove(&self, key: &RecordKey) -> Result<()>

De-index the record at key: remove every chunk vector it contributed to the index and forget its chunk count. This is the counterpart to a record being deleted from the store — without it, the record’s chunk vectors are orphaned, still matching in [query] but de-duping to a parent that no longer resolves, so they silently shrink the result set (#150).

Idempotent: removing a record that was never ingested (chunk count 0) removes nothing. Uses the same [chunk_key] derivation as [ingest].

pub async fn query( &self, text: &str, k: usize, filter: &KeyPrefix, ) -> Result<Vec<Hit>>

Semantic query, one hit per record. Embed text, over-fetch chunk matches (restricted to filter), collapse them to their parent records keeping each record’s best chunk score, and return the top-k records.

Because a record with many matching chunks can crowd the over-fetch window, pathological cases may return fewer than k hits.

pub async fn query_chunks( &self, text: &str, k: usize, filter: &KeyPrefix, ) -> Result<Vec<ChunkHit>>

Chunk-granular query: the top-k matching chunks (restricted to filter), not de-duped to their parents. Each hit carries the parent record, the chunk’s ordinal, and the chunk’s text.