chroma-core / chroma-core/chroma
[Feature Request]: Set ef_search per query
- Dominant language
- Rust
- Stars
- 29.3k
- Forks
- 2.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 38
Description
### Describe the problem
Currently we only set the `ef_search` parameter when we init the index, usually on creation of a `Collection`. However, this parameter is actually per-query at the level of HNSW.
Generally we only need to set this parameter once since in principle there is a 'best' setting for the average case on any collection, but there are cases where we may want to let the user set it dynamically, for example to conduct a parameter sweep against a particular dataset.
Additionally, EF should be at least as large as the number of elements we want to return, which means it may have to be dynamic anyway.
### Describe the proposed solution
Allow EF search to be set on a per-query basis.
### Alternatives considered
_No response_
### Importance
would make my life easier
### Additional Information
This is somewhat of a pain in the ass. Besides piping in this parameter on query (which now creates the need for a `query_parameters` argument...) the way the HNSW implementation handles this is, also, bad, since it's set as a property of the index: https://github.com/chroma-core/hnswlib/blob/d52dd85106c5a1955a7dbd745a76ba2a8259451a/hnswlib/hnswalg.h#L33
even though it's only a property of each *query*.
To do this 'right' (for example, to handle concurrency correctly), it would take a refactor of our HNSW impl too.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing how Collection initialization currently sets ef_search and how query requests reach the HNSW implementation. Read hnswlib's hnswalg.h at the linked location, then determine the changes needed for a per-query parameter and concurrency-safe handling; done means callers can set ef_search dynamically, including values based on requested result counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, rust
- Domain
- databases, performance, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100