huggingface / huggingface/evaluate

Rouge evaluator produced `0` score for the same prediction and reference in Chinese.

Open
#465 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

I am using rouge evaluator for assessing my translation outputs, but the score is not correct as I know.
An example is like this:
image
both the `prediction` and `reference` are `length=9` strings. They are totally the same, but the `rouge-score` is all **zero``, while the correct one should be `{'rouge1': 1.0, 'rouge2': 0.0, 'rougeL': 1.0, 'rougeLsum': 1.0}`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.