chroma-core / chroma-core/chroma

[New Feature][Accuracy] Reranker

Open
#2,283 2 comments 1 reaction 1 assignee Claimed by @atroyn View on GitHub
by-chroma Local Chroma
Dominant language
Rust
Stars
29.3k
Forks
2.5k
Avg merge
1d 4h
Merged PRs (30d)
38

Description

# Reranker

Re-ranking is a quick accuracy win, and as cheap computationally as sentence transformers.

Re-ranking can take account of the query as well as additional metadata when evaluating the relevance of results. For example, it’s fairly straightforward to add a weighted term to account for data recency.

It also provides a normalized measure of relevancy, allowing users to more easily filter on the relevancy of results.

## API Design

```python
# Get a re-ranker
reranker = chromadb.utils.rerankers.Reranker()

# API 1.

# Pass it as an additional argument to query:
result = collection.query(...., reranker=reranker)

# Return a new field, 'rank_scores'
result.reranker_scores: List[List[Double]] # Score per retrived document per query

# API 2.
reranker.rerank(results, query_text) # return the query back?

# API 3.
results.rerank(reranker)
```

## [Complexity] Subtask

- [Low] Use a [sentence transformer cross-encoder](https://sbert.net/examples/training/cross-encoder/README.html) as a basic re-ranker.
- [Low] Implement [Cohere re-ranker](https://cohere.com/rerank) as a third-party re-ranker.
- [Low] Evaluate the effectiveness of the default re-ranker we ship with the default embedding model to ensure we actually get wins.
- [Med] Ship a metadata-aware re-ranker (use RRF)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.