huggingface / huggingface/lighteval

[EVAL] Add MathVista

Open
#975 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

## Evaluation short description
- Why is this evaluation interesting?
MathVista is one of the most popular and commonly reported benchmark in MultiModal LLM like Qwen2.5_VL and InternVL3_5
- How used is it in the community?
To evaluate multimodal vision reasoning capabilities on Math questions

## Evaluation metadata
Provide all available
- Paper url:https://arxiv.org/pdf/2310.02255
- Github url: https://github.com/lupantech/MathVista
- Dataset url: https://huggingface.co/datasets/AI4Math/MathVista

https://mathvista.github.io/

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.