Azure / Azure/azure-search-vector-samples
How to Calculate Search Scores and Rankings for Hybrid Searches
- Dominant language
- Jupyter Notebook
- Stars
- 910
- Forks
- 377
- PR merge metrics
- No merged PRs in 30d
Description
Hi team,
I am wondering how search scores and rankings are calculated for hybrid searches.
I did a search with keyword search(top=10) and vector search(vector.k=10) and got the following ranking.
RRFscore(d) = Σ (1 / (_k_ + rank_d_i)) with constant _k_ = 59.
**1, Simple Vector Search Result**
| FileName | Vector-Rank | Vector-RRF |
| ------------- | ------------- |------------- |
| A.txt | 2 | 1/(_k_+2)=0.016393443 |
| B.txt | 1 | 1/(_k_+1)=0.016666667 |
| C.txt | 3 | 1/(_k_+3)=0.016129032 |
**2, Simple Keyword Search Result**
| FileName | Keyword-Rank | Keyword-RRF |
| ------------- | ------------- |------------- |
| A.txt | 8 | 1/(_k_+8)=0.014925373 |
| B.txt | 10 | 1/(_k_+10)=0.014492754 |
| C.txt | None | 1/(_k_+?)=None |
**3, Σ RRF**
| FileName | calclated-RRF | displayed-RRF |
| ------------- | ------------- |------------- |
| A.txt | 1/(_k_+2)+1/(_k_+8)=0.031318816 | ≈0.03131881356239319 |
| B.txt | 1/(_k_+1)+1/(_k_+10)=0.031159420 | ≈0.031159421429038048 |
| C.txt | 1/(_k_+3)=0.016129032 | **≠0.029286926612257957** |
**Question 1.**
[The docs](https://github.com/Azure/cognitive-search-vector-pr/blob/main/docs/vector-search-overview.md#reciprocal-rank-fusion) describe _k_= like 60, but my calculation seems to be 59, how much is it actually?
**Question 2.**
"C.txt" was not found in the Simple Keyword Search (top=10) results. How is the RRF calculated in this case? My guess is that internally a larger value than I specified in my top query is specified and its ranking is calculated.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked Reciprocal Rank Fusion section in docs/vector-search-overview.md and compare its description with the reported keyword and vector rankings. Determine the actual constant and how documents missing from one result set are scored; done means the documentation answers both questions accurately, with the example reconciled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- documentation, search
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100