Expand description
Shard assignment and the on-blob format for a durable vector index.
Vectors are bucketed by a hash of their key so a write touches one shard rather than the whole index, and a shard’s bytes are deterministic so an unchanged shard content-addresses to the blob already stored.
Constants§
- DEFAULT_
SHARDS - Default number of shards for a new index. At ~100k chunks of 384 f32s this is ~600 KB per shard: 256 reads to open, one ~600 KB rewrite per upsert.
Functions§
- shard_
of - Which shard
keybelongs to.