deepinsight / deepinsight/insightface
IJB-B and C Evaluation Problem
- Dominant language
- Python
- Stars
- 29.7k
- Forks
- 6.1k
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to evaluate the performance of my model on IJB-B dataset. I am using the code in the link below:
https://github.com/deepinsight/insightface/blob/master/evaluation/IJB/IJB_11.py
However, i can see that in get_image_feature, there is no call to forward_db function, so it gives me an output with the shape (227630,2) which is wrong. It should be (227630,512) as 512 is the embedding dimension. Then, I tried the following code:
https://github.com/deepinsight/insightface/blob/master/recognition/partial_fc/pytorch/IJB/IJB_11_Batch.py
It generates the embedding with correct dimension, but the evaluation results are very poor.
Methods | 1e-06 | 1e-05 | 0.0001 | 0.001 | 0.01 | 0.1 |
+------------------+-------+-------+--------+-------+------+-------+
| insightface-IJBB | 0.00 | 0.00 | 0.05 | 0.30 | 2.19 | 19.59 |
+------------------+-------+-------+--------+-------+------+-------+
Can someone tell me how to solve this problem? Thank you..
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with evaluation/IJB/IJB_11.py, especially get_image_feature and the missing forward_db call, then compare it with recognition/partial_fc/pytorch/IJB/IJB_11_Batch.py. Reproduce the reported embedding shapes and evaluation metrics; done means the embeddings have shape (227630,512) and the IJB-B evaluation produces valid results rather than the reported poor scores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100