pub struct RecordVectorIndex<S> { /* private fields */ }Expand description
A VectorIndex whose contents live in store rather than only in
process memory (ADR 0027): vectors are bucketed into content-addressed
shard blobs named by one [VectorManifest] record, so a query is served
from an in-memory MemoryVectorIndex hydrated at open,
and a write goes through to store under optimistic concurrency before it
is reflected in memory.
One writer per index. A handle serializes its own upsert/remove/
upsert_many calls, but a second handle on the same key is only
detected, not merged: its commit conflicts and retries, reloading whatever
the winner changed. A second handle’s queries can still be served from a
shard it hasn’t refreshed until it reopens.
Requires a tokio runtime. open reads shard blobs
concurrently via tokio::task::JoinSet::spawn, which panics outside one.
Implementations§
Source§impl<S: Store + BlobStore + Send + Sync + 'static> RecordVectorIndex<S>
impl<S: Store + BlobStore + Send + Sync + 'static> RecordVectorIndex<S>
Sourcepub async fn open(
store: S,
key: RecordKey,
space: &str,
dim: usize,
) -> Result<Self>
pub async fn open( store: S, key: RecordKey, space: &str, dim: usize, ) -> Result<Self>
Open the index at key, hydrating it from store.
Errors if a manifest exists whose space or dim differs from the
declared one, naming both values, so a swapped embedder surfaces once at
startup rather than as quietly wrong rankings on every later query. A
missing manifest starts an empty index created on the first commit.
Errors if the index was deleted and its tombstone still holds a
restore window (gonzalo#340). The error names the key and the two ways
out — gonzalo undelete to restore it, gonzalo purge to discard the
window — and there is deliberately no flag here that skips them: a
caller starting over purges first, which keeps discarding a restore
window a single explicit operator action (ADR 0029/0030).
Sourcepub async fn open_with_shards(
store: S,
key: RecordKey,
space: &str,
dim: usize,
shards: NonZeroU16,
) -> Result<Self>
pub async fn open_with_shards( store: S, key: RecordKey, space: &str, dim: usize, shards: NonZeroU16, ) -> Result<Self>
As open, but choosing the shard count for a new
index. An existing manifest’s shard count always wins.
Trait Implementations§
Source§impl<S> Debug for RecordVectorIndex<S>
impl<S> Debug for RecordVectorIndex<S>
Source§impl<S: Store + BlobStore + Send + Sync + 'static> VectorIndex for RecordVectorIndex<S>
impl<S: Store + BlobStore + Send + Sync + 'static> VectorIndex for RecordVectorIndex<S>
Source§fn upsert_many<'life0, 'async_trait>(
&'life0 self,
items: Vec<(RecordKey, Vec<f32>)>,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn upsert_many<'life0, 'async_trait>(
&'life0 self,
items: Vec<(RecordKey, Vec<f32>)>,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Overrides the trait’s looping default: one batch becomes one commit
call, so a bulk load stages every shard once and writes one manifest
revision instead of one per vector. Dimensions are validated up front,
before anything is staged, so a bad batch fails without touching the
store at all.
Source§fn upsert<'life0, 'async_trait>(
&'life0 self,
key: RecordKey,
vector: Vec<f32>,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn upsert<'life0, 'async_trait>(
&'life0 self,
key: RecordKey,
vector: Vec<f32>,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
key.Source§fn remove<'life0, 'life1, 'async_trait>(
&'life0 self,
key: &'life1 RecordKey,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn remove<'life0, 'life1, 'async_trait>(
&'life0 self,
key: &'life1 RecordKey,
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
key if present (no error if absent).Source§fn query<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
query: &'life1 [f32],
k: usize,
filter: &'life2 KeyPrefix,
) -> Pin<Box<dyn Future<Output = Result<Vec<Match>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
fn query<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
query: &'life1 [f32],
k: usize,
filter: &'life2 KeyPrefix,
) -> Pin<Box<dyn Future<Output = Result<Vec<Match>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
k matches to query, restricted to keys matching
filter, ordered by descending score.Source§fn keys<'life0, 'life1, 'async_trait>(
&'life0 self,
filter: &'life1 KeyPrefix,
) -> Pin<Box<dyn Future<Output = Result<Vec<RecordKey>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn keys<'life0, 'life1, 'async_trait>(
&'life0 self,
filter: &'life1 KeyPrefix,
) -> Pin<Box<dyn Future<Output = Result<Vec<RecordKey>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
filter. Order is unspecified. Read more