Thinklab-SJTU / Thinklab-SJTU/Bench2Drive
About evaluation metric confusion
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 144
- PR merge metrics
- No merged PRs in 30d
Description
I tried to evaluate TCP with official ckpt file in my device. However, I find that the results of Merging and Overtaking reversed.
The official are Merging:8.89 and Overtaking:24.29.Mine are Merging:21.43 and Merging:11.11. That's really confusing.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the TCP evaluation on the device with the official checkpoint and compare the Merging and Overtaking outputs with the reported values. The issue names no source file or test, so trace the evaluation entry point used for the checkpoint and verify how these two metrics are produced; done when the discrepancy is explained or corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- autonomous-driving
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100