[Feature]: Integrate OMEGA adaptive early termination into zvec
- Dominant language
- C++
- Stars
- 15.9k
- Forks
- 998
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 34
Description
### Problem / Motivation
HNSW usually depends on a fixed `ef_search` budget. In real workloads, that budget often has to be tuned
conservatively, which means many easy queries still pay unnecessary traversal and comparison cost.
[OMEGA](https://arxiv.org/abs/2603.06159) is designed to turn this fixed budget into adaptive early termination: it monitors the current HNSW search
state and decides whether the target recall has already been reached.
More importantly, OMEGA is not just another learned stopping heuristic. Its key advantage is **one-model for multi-K**: it reduces an arbitrary top-K ANN query into repeated top-1 ANN decisions, so a single learned model can be reused across different K values and different recall targets. This avoids the need to retrain separate models or repeatedly retune search parameters for each K.
### Proposed Solution
```python
Integrate OMEGA into zvec as a built-in capability on top of HNSW.
This includes:
- OMEGA index and query parameters
- integration with the HNSW search loop through hooks
- offline training and model persistence
- support for one-model-for-multi-K query serving
- runtime fallback to plain HNSW when OMEGA is unavailable or not applicable
```
### Alternatives Considered
- Keep using only fixed-budget HNSW tuning:
this is simple, but it still relies on a single global budget such as `ef_search`, which does not address query-level variation and cannot provide one-model-for-multi-K behavior.
- Build a separate OMEGA-specific search path:
this would duplicate the existing HNSW search implementation and increase long-term maintenance cost. We prefer integrating OMEGA as a query-time control layer on top of the current HNSW path.
- Train or tune separate learned/search configurations for different K values:
this increases offline cost and operational complexity. One of OMEGA's main advantages is that a single model can serve multiple K values and recall targets.
### Affected Area
{"label" => "C++ Core (storage, indexing)"}, {"label" => "Python API / Bindings"}, {"label" => "Testing / CI / Coverage"}
Contributor guide
Assessment
This issue has not been assessed yet.