chroma-core / chroma-core/chroma

[Feature Request]: Set ef_search per query

Open
#2,176 3 comments 0 reactions 0 assignees View on GitHub
by-chroma enhancement needs-cip
Dominant language
Rust
Stars
29.3k
Forks
2.5k
Avg merge
1d 4h
Merged PRs (30d)
38

Description

### Describe the problem

Currently we only set the `ef_search` parameter when we init the index, usually on creation of a `Collection`. However, this parameter is actually per-query at the level of HNSW.

Generally we only need to set this parameter once since in principle there is a 'best' setting for the average case on any collection, but there are cases where we may want to let the user set it dynamically, for example to conduct a parameter sweep against a particular dataset.

Additionally, EF should be at least as large as the number of elements we want to return, which means it may have to be dynamic anyway.

### Describe the proposed solution

Allow EF search to be set on a per-query basis.

### Alternatives considered

_No response_

### Importance

would make my life easier

### Additional Information

This is somewhat of a pain in the ass. Besides piping in this parameter on query (which now creates the need for a `query_parameters` argument...) the way the HNSW implementation handles this is, also, bad, since it's set as a property of the index: https://github.com/chroma-core/hnswlib/blob/d52dd85106c5a1955a7dbd745a76ba2a8259451a/hnswlib/hnswalg.h#L33

even though it's only a property of each *query*.

To do this 'right' (for example, to handle concurrency correctly), it would take a refactor of our HNSW impl too.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how Collection initialization currently sets ef_search and how query requests reach the HNSW implementation. Read hnswlib's hnswalg.h at the linked location, then determine the changes needed for a per-query parameter and concurrency-safe handling; done means callers can set ef_search dynamically, including values based on requested result counts.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, rust
Domain
databases, performance, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.