chroma-core / chroma-core/chroma
[New Feature][Accuracy] Reranker
- Dominant language
- Rust
- Stars
- 29.3k
- Forks
- 2.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 38
Description
# Reranker
Re-ranking is a quick accuracy win, and as cheap computationally as sentence transformers.
Re-ranking can take account of the query as well as additional metadata when evaluating the relevance of results. For example, it’s fairly straightforward to add a weighted term to account for data recency.
It also provides a normalized measure of relevancy, allowing users to more easily filter on the relevancy of results.
## API Design
```python
# Get a re-ranker
reranker = chromadb.utils.rerankers.Reranker()
# API 1.
# Pass it as an additional argument to query:
result = collection.query(...., reranker=reranker)
# Return a new field, 'rank_scores'
result.reranker_scores: List[List[Double]] # Score per retrived document per query
# API 2.
reranker.rerank(results, query_text) # return the query back?
# API 3.
results.rerank(reranker)
```
## [Complexity] Subtask
- [Low] Use a [sentence transformer cross-encoder](https://sbert.net/examples/training/cross-encoder/README.html) as a basic re-ranker.
- [Low] Implement [Cohere re-ranker](https://cohere.com/rerank) as a third-party re-ranker.
- [Low] Evaluate the effectiveness of the default re-ranker we ship with the default embedding model to ensure we actually get wins.
- [Med] Ship a metadata-aware re-ranker (use RRF)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.