huggingface / huggingface/evaluate
Add support for other languages for rouge
Open
Metric discussion
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
I calculate rouge with
```
from datasets import load_metric
rouge = load_metric("rouge")
rouge_output = rouge.compute(predictions=['тест тест привет'], references=['тест тест пока'], rouge_types=[
"rouge2"])["rouge2"].mid
print(rouge_output)
```
the result is
`Score(precision=0.0, recall=0.0, fmeasure=0.0)`
It seems like the `rouge_score` library that this metric uses filters all non-alphanueric latin characters
in `rouge_scorer/tokenize.py` with `text = re.sub(r"[^a-z0-9]+", " ", six.ensure_str(text))`.
Please add support for other languages.
Contributor guide
Assessment
This issue has not been assessed yet.