huggingface / huggingface/evaluate
[Question]Shall we adding a faster BLEU score calculator?
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
## Shall we?
Hi, I found the [performance request](https://huggingface.co/spaces/evaluate-metric/bleu/discussions/8) about the bleu score. And I'm wondering if we should add a `fast-bleu` or replace the `bleu` implementation with a faster one, like [bleuscore](https://github.com/shenxiangzhuang/bleuscore) which is aimed for better performance for large data size.
## About bleuscore library
### Correctness
I implement bleuscore library with exactly same way as evaluate library and do [serious test with hypothesis](https://github.com/shenxiangzhuang/bleuscore/blob/master/tests/test_bleu_score.py) to make sure they return same result.
### Performance
I wrote the library in Rust and used multi-threading for parallel execution, so it has better performance when the data size gets larger, you can check the [benchmark](https://github.com/shenxiangzhuang/bleuscore?tab=readme-ov-file#benchmark) section for more details.
Contributor guide
Assessment
This issue has not been assessed yet.