MOT20 leaderboard ranks a submission with no evaluation scores as #1
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 176
- Forks
- 74
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 21
Description
Hello Codabench team,
I found a ranking issue on the MOT20 Challenge leaderboard for MOT20 Test set submissions.
A submission with no evaluation scores is currently ranked #1, ahead of submissions with valid results.
Current leaderboard entries
Rank 1
Participant: s*****
Date: 2026-07-22 17:58
Submission ID: 857861
Method name: adt
Detection type: Public Detections
Tracker type: Online Tracker
HOTA: n/a
MOTA: n/a
IDF1: n/a
All other evaluation metrics: n/a
Expected behavior
Submissions without valid evaluation results should not be ranked above successfully evaluated submissions. They should either:
be excluded from the leaderboard ranking,
be marked as failed or pending, or
appear below submissions with valid scores.
Actual behavior
Submission 857861, which has n/a for all evaluation metrics, is ranked first on the leaderboard.
Could you please check the leaderboard sorting and handling of submissions with missing evaluation scores?
Thank you.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the MOT20 leaderboard and submission 857861, then trace the leaderboard sorting and handling of submissions whose evaluation metrics are all n/a. Reproduce the ranking behavior and verify that submissions without valid scores are excluded, marked failed or pending, or placed below evaluated submissions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100