Thinklab-SJTU / Thinklab-SJTU/Bench2Drive
unmatching of dataset & evaluation script frame rate.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 144
- PR merge metrics
- No merged PRs in 30d
Description
Bench2Drive dataset is collect with 10 fps, so the trained model should be good for 10fps.
However, I noticed the frame_rate in LeaderboardEvaluator & ScenarioRunner are 20.0 Hz as default.
Will this unmatching affect the evaluation results ?
If I want to run evaluation with 10fps, shall I modify both of the frame_rate ?
Thanks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the LeaderboardEvaluator and ScenarioRunner frame_rate defaults and compare them with the Bench2Drive dataset's stated 10 fps. Run or inspect the evaluation path with the relevant frame-rate settings to determine whether the mismatch changes results. Done means documenting the impact and identifying whether both settings must be changed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- autonomous-driving
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100