huggingface / huggingface/lighteval
[EVAL] Add Multi-IF
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Evaluation short description
Multi-IF has become one the multilingual analogue of the popular IFEval benchmark and is widely used in most recent model releases like Qwen3, SmolLM3 etc.
It requires LLMs to follow verifiable instructions across 3 turns of user prompts and thus has the added benefit of testing LLMs' ability to follow more than one turn.
## Evaluation metadata
Provide all available
- Paper url: https://arxiv.org/abs/2410.15553
- Github url: https://github.com/facebookresearch/Multi-IF
- Dataset url: https://huggingface.co/datasets/facebook/Multi-IF
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.