open-compass / open-compass/VLMEvalKit
Mathverse vision only score different from leaderboard
@FangXinyu-0913 is already working on this.
Since Mar 28, 2025.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 17
Description
I tested qwen2.5-vl-7b but the result is much lower than leaderboard, is there any special command?
Command:
CUDA_VISIBLE_DEVICES=1 python run.py --data MathVerse_MINI_Vision_Only --model Qwen2.5-VL-7B
Result:
split Vision Only
Overall 30.96446700507614
Solid Geometry 11.76470588235294
Functions 37.10691823899371
Plane Geometry 33.52941176470588
Area 21.978021978021978
Angle 35.233160621761655
Analytic 32.55813953488372
Length 36.81318681318682
Volume 7.8431372549019605
Applied 28.440366972477065
Coordinate 56.25
Expression 12.5
Property 38.028169014084504
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.