huggingface/lighteval
View on GitHubLighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
- Stars
- 2.5k
- Forks
- 555
- Open beginner issues
- 1
- Indexed issues
- 233
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
- Dominant language
- Python
- License
- MIT
- Last GitHub push
- Sep 9, 2026
- Latest indexed
- Sep 17, 2026
- Contributing guide
- No contributing guide
- Code of conduct
- No code of conduct
- Beginner labels
- good first issue help wanted
-
huggingface/lighteval#796 · 1 comment · 2 reactions · 0 assignees ·
-
huggingface/lighteval#797 · 2 comments · 1 reaction · 0 assignees ·
-
huggingface/lighteval#798 · 0 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#799 · 1 comment · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#801 · 0 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#802 · 10 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#803 · 2 comments · 0 reactions · 0 assignees ·
-
feature good first issue
huggingface/lighteval#804 · 5 comments · 1 reaction · 0 assignees ·
-
bug
huggingface/lighteval#805 · 3 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#806 · 1 comment · 1 reaction · 0 assignees ·
-
huggingface/lighteval#808 · 2 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#813 · 0 comments · 0 reactions · 0 assignees ·
-
[EVAL] MRCR Opennew-task science-team
huggingface/lighteval#815 · 2 comments · 0 reactions · 0 assignees ·
-
[EVAL] LVEval Opennew-task science-team
huggingface/lighteval#816 · 0 comments · 0 reactions · 0 assignees ·
-
prio science-team
huggingface/lighteval#819 · 2 comments · 2 reactions · 2 assignees ·
-
huggingface/lighteval#821 · 0 comments · 0 reactions · 1 assignee ·
-
huggingface/lighteval#822 · 2 comments · 0 reactions · 0 assignees ·
-
refacto
huggingface/lighteval#823 · 0 comments · 0 reactions · 1 assignee ·
-
prio science-team
huggingface/lighteval#824 · 0 comments · 0 reactions · 1 assignee ·
-
science-team
huggingface/lighteval#825 · 1 comment · 0 reactions · 1 assignee ·
-
science-team
huggingface/lighteval#826 · 0 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#834 · 0 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#844 · 4 comments · 0 reactions · 0 assignees ·
-
[BUG] Error in lighteval/metrics/metrics_sample.py:513: _rouge_score_with_bootsrap, invalid argument Openbug
huggingface/lighteval#854 · 0 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#855 · 2 comments · 7 reactions · 0 assignees ·
-
bug documentation
huggingface/lighteval#857 · 2 comments · 1 reaction · 0 assignees ·
-
bug
huggingface/lighteval#858 · 0 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#864 · 3 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#867 · 0 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#868 · 0 comments · 0 reactions · 0 assignees ·
-
[EVAL] Add Multi-IF Openscience-team
huggingface/lighteval#871 · 0 comments · 0 reactions · 1 assignee ·
-
prio science-team
huggingface/lighteval#872 · 0 comments · 0 reactions · 0 assignees ·
-
[EVAL] Add BFCL-v3 Openscience-team
huggingface/lighteval#873 · 0 comments · 1 reaction · 0 assignees ·
-
[EVAL] Add tau-bench Openscience-team
huggingface/lighteval#874 · 4 comments · 3 reactions · 0 assignees ·
-
[EVAL] Add LiveBench Openscience-team
huggingface/lighteval#875 · 0 comments · 0 reactions · 0 assignees ·
-
science-team
huggingface/lighteval#876 · 0 comments · 0 reactions · 0 assignees ·
-
huggingface/lighteval#881 · 0 comments · 0 reactions · 1 assignee ·
-
bug
huggingface/lighteval#888 · 0 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#892 · 2 comments · 0 reactions · 1 assignee ·
-
bug
huggingface/lighteval#895 · 0 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#896 · 0 comments · 0 reactions · 0 assignees ·
-
[BUG] TransformersModel.from_model provides an incompletely initialized object compared to __init__ Openbug
huggingface/lighteval#897 · 1 comment · 0 reactions · 0 assignees ·
-
huggingface/lighteval#918 · 2 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#933 · 0 comments · 2 reactions · 0 assignees ·
-
science-team
huggingface/lighteval#940 · 0 comments · 0 reactions · 1 assignee ·
-
bug
huggingface/lighteval#949 · 1 comment · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#950 · 1 comment · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#957 · 0 comments · 1 reaction · 0 assignees ·
-
bug
huggingface/lighteval#959 · 10 comments · 0 reactions · 0 assignees ·
-
bug
huggingface/lighteval#960 · 1 comment · 0 reactions · 0 assignees ·