elastic / elastic/roadmap

Auto-calibration for DiskBBQ vector search

Open
#281 0 comments 0 reactions 1 assignee Claimed by @giladgal View on GitHub
Component: Elasticsearch product-area:search
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Users would benefit from vector search configurations that maintain relevance ranking quality based on how easily vectors can be told apart, but without needing to run manual experiments on their data or understand underlying configuration details.

Auto-calibration is an automatic configuration mechanism that reads the index data to determine the optimal quantization level for DiskBBQ, the use of preconditioning, and the oversampling factor. If vectors are similar, Elasticsearch will quantize them to a less lossy data type and increase oversampling. If vectors are highly distinct, these settings are reduced. This mechanism dynamically balances better query latency and lower computing resource consumption against relevance quality based on the actual indexed data.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.