NVIDIA-Merlin / NVIDIA-Merlin/Merlin
[QST] Difference Similarity Metrics between Two tower training and QueryFaiss
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 907
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
❓ Questions & Help
Details
I try to follow the example of Two Tower model training model and ensemble pipeline. However, The top-k prediction lists from two tower training and top-k search from QueryFaiss are significantly different. Which I believe due to the use of different similarity functions during training (Dot Product) and QueryFaiss (L2 Distance).
I would appreciate any advice you may have regarding this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Two Tower model training example and the ensemble pipeline notebook linked in the issue. Compare the retrieval and similarity configurations used to produce each top-k list, then reproduce the discrepancy with the reported workflow. Done means the cause is documented and the two retrieval paths are either aligned or their expected difference is clearly explained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100