huggingface / huggingface/evaluate
Rouge evaluator produced `0` score for the same prediction and reference in Chinese.
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
I am using rouge evaluator for assessing my translation outputs, but the score is not correct as I know.
An example is like this:
both the `prediction` and `reference` are `length=9` strings. They are totally the same, but the `rouge-score` is all **zero``, while the correct one should be `{'rouge1': 1.0, 'rouge2': 0.0, 'rougeL': 1.0, 'rougeLsum': 1.0}`.
Contributor guide
Assessment
This issue has not been assessed yet.