Skip to main content

RecordVectorIndex

Struct RecordVectorIndex 

Source
pub struct RecordVectorIndex<S> { /* private fields */ }
Expand description

A VectorIndex whose contents live in store rather than only in process memory (ADR 0027): vectors are bucketed into content-addressed shard blobs named by one [VectorManifest] record, so a query is served from an in-memory MemoryVectorIndex hydrated at open, and a write goes through to store under optimistic concurrency before it is reflected in memory.

One writer per index. A handle serializes its own upsert/remove/ upsert_many calls, but a second handle on the same key is only detected, not merged: its commit conflicts and retries, reloading whatever the winner changed. A second handle’s queries can still be served from a shard it hasn’t refreshed until it reopens.

Requires a tokio runtime. open reads shard blobs concurrently via tokio::task::JoinSet::spawn, which panics outside one.

Implementations§

Source§

impl<S: Store + BlobStore + Send + Sync + 'static> RecordVectorIndex<S>

Source

pub async fn open( store: S, key: RecordKey, space: &str, dim: usize, ) -> Result<Self>

Open the index at key, hydrating it from store.

Errors if a manifest exists whose space or dim differs from the declared one, naming both values, so a swapped embedder surfaces once at startup rather than as quietly wrong rankings on every later query. A missing manifest starts an empty index created on the first commit.

Errors if the index was deleted and its tombstone still holds a restore window (gonzalo#340). The error names the key and the two ways out — gonzalo undelete to restore it, gonzalo purge to discard the window — and there is deliberately no flag here that skips them: a caller starting over purges first, which keeps discarding a restore window a single explicit operator action (ADR 0029/0030).

Source

pub async fn open_with_shards( store: S, key: RecordKey, space: &str, dim: usize, shards: NonZeroU16, ) -> Result<Self>

As open, but choosing the shard count for a new index. An existing manifest’s shard count always wins.

Source

pub fn store(&self) -> &S

Borrow the underlying store.

Trait Implementations§

Source§

impl<S> Debug for RecordVectorIndex<S>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl<S: Store + BlobStore + Send + Sync + 'static> VectorIndex for RecordVectorIndex<S>

Source§

fn upsert_many<'life0, 'async_trait>( &'life0 self, items: Vec<(RecordKey, Vec<f32>)>, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Overrides the trait’s looping default: one batch becomes one commit call, so a bulk load stages every shard once and writes one manifest revision instead of one per vector. Dimensions are validated up front, before anything is staged, so a bad batch fails without touching the store at all.

Source§

fn upsert<'life0, 'async_trait>( &'life0 self, key: RecordKey, vector: Vec<f32>, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Insert or replace the vector for key.
Source§

fn remove<'life0, 'life1, 'async_trait>( &'life0 self, key: &'life1 RecordKey, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Remove key if present (no error if absent).
Source§

fn query<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, query: &'life1 [f32], k: usize, filter: &'life2 KeyPrefix, ) -> Pin<Box<dyn Future<Output = Result<Vec<Match>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Return the top-k matches to query, restricted to keys matching filter, ordered by descending score.
Source§

fn keys<'life0, 'life1, 'async_trait>( &'life0 self, filter: &'life1 KeyPrefix, ) -> Pin<Box<dyn Future<Output = Result<Vec<RecordKey>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Every key in the index matching filter. Order is unspecified. Read more

Auto Trait Implementations§

§

impl<S> !Freeze for RecordVectorIndex<S>

§

impl<S> !RefUnwindSafe for RecordVectorIndex<S>

§

impl<S> !UnwindSafe for RecordVectorIndex<S>

§

impl<S> Send for RecordVectorIndex<S>
where S: Sync + Send,

§

impl<S> Sync for RecordVectorIndex<S>
where S: Sync + Send,

§

impl<S> Unpin for RecordVectorIndex<S>

§

impl<S> UnsafeUnpin for RecordVectorIndex<S>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.