deepinsight / deepinsight/insightface

IJB-B and C Evaluation Problem

Open
#1,407 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

I am trying to evaluate the performance of my model on IJB-B dataset. I am using the code in the link below:
https://github.com/deepinsight/insightface/blob/master/evaluation/IJB/IJB_11.py

However, i can see that in get_image_feature, there is no call to forward_db function, so it gives me an output with the shape (227630,2) which is wrong. It should be (227630,512) as 512 is the embedding dimension. Then, I tried the following code:
https://github.com/deepinsight/insightface/blob/master/recognition/partial_fc/pytorch/IJB/IJB_11_Batch.py
It generates the embedding with correct dimension, but the evaluation results are very poor.

Methods | 1e-06 | 1e-05 | 0.0001 | 0.001 | 0.01 | 0.1 |
+------------------+-------+-------+--------+-------+------+-------+
| insightface-IJBB | 0.00 | 0.00 | 0.05 | 0.30 | 2.19 | 19.59 |
+------------------+-------+-------+--------+-------+------+-------+

Can someone tell me how to solve this problem? Thank you..

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with evaluation/IJB/IJB_11.py, especially get_image_feature and the missing forward_db call, then compare it with recognition/partial_fc/pytorch/IJB/IJB_11_Batch.py. Reproduce the reported embedding shapes and evaluation metrics; done means the embeddings have shape (227630,512) and the IJB-B evaluation produces valid results rather than the reported poor scores.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.