cockroachdb / cockroachdb/cockroach
sql/vecindex: push `Store.SearchPartitions` into KV/Pebble layer
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Searching a vector index partition requires scanning over ~100 high-dimensional vectors that are stored as contiguous KV rows. For each vector, its estimated distance to the query vector needs to be computed (preferably using SIMD instructions). Currently, we fetch all 100 KV rows back to the gateway node and then perform the scan there. This requires serializing and deserializing 100 rows of data for each partition, which is CPU-hungry and also consumes network bandwidth.
It would be much better to push down the partition scan into the KV layer, so that it can run directly in the KV server process. Ideally, it could be pushed down to run against the raw Pebble buffers, so that no extra copies need to be made. Only the estimated distance (float32), error bound (float32), child key (usually 8-12 bytes), and value bytes (usually just a few bytes) would need to be returned to the gateway node, rather than each complete row.
NOTE: if we did this, we would also need to solve the problem of how to account for CPU consumption in the KV server process, so it could be charged back in multi-tenant scenarios.
Jira issue: CRDB-53646
Contributor guide
Research direction
Start by tracing sql/vecindex's Store.SearchPartitions path and how partition rows move between the gateway and the KV/Pebble layer. Determine the design needed for server-side scanning, compact result transfer, and CPU accounting; done requires an agreed implementation plan that addresses all three concerns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100