huggingface / huggingface/lighteval
[New Task] Add AlpacaEval LC
Open
feature
new-task
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
Great library, a light library for all the main evals was really needed!💯
I just came across this [line](https://github.com/huggingface/lighteval/blob/af24080ea4f16eaf1683e353042a2dfc9099f038/src/lighteval/tasks/extended/tiny_benchmarks/main.py#L79): is there any interest of adding [length-controlled AlpacaEval ](https://github.com/tatsu-lab/alpaca_eval)to lighteval? If so I'm happy to help, e.g., if you want a minimal script that doesn't depend on `alpaca_eval`.
Let me know
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.