facebookresearch / facebookresearch/isc2021

Reproducing Multigrain subset results

Open
#10 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
205
Forks
41
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm trying to reproduce the baseline following the step provided in the below link.
( https://github.com/facebookresearch/isc2021/blob/main/docs/4_multigrain_subset.md )
I strictly followed the steps without any modification of the given code.
However, for my case, the result seems far different from the doc.

It supposed to be:
Average Precision: 0.56314
Recall at P90 : 0.41374
Threshold at P90 : -1.51692
Recall at rank 1: 0.60970
Recall at rank 10: 0.70827

But I got:
Track 2 running matching of 4991 queries in 4991 database (1500D descriptors), max_results=500000.
Evaluating 499999 predictions (4991 GT matches)
Average Precision: 0.09594
Recall at P90 : 0.07113
Threshold at P90 : -1.52351
Recall at rank 1: 0.10258
Recall at rank 10: 0.12022

I really have no idea on the incorrect result.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.