huggingface / huggingface/lighteval
[EVAL] Add TUMLU benchmark
Open
good first issue
help wanted
new-task
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
Hello!
We just released the benchmark for Turkic languages. Does it make sense if I add it to lighteval?
## Evaluation short description
- Why is this evaluation interesting?
First native-language MMLU benchmark for low-resource Turkic languages.
- How is it used in the community?
Just released, MC high-school exam questions
## Evaluation metadata
Provide all available
- Paper url: https://arxiv.org/abs/2502.11020
- Github url: https://github.com/ceferisbarov/TUMLU
- Dataset url:
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.