huggingface / huggingface/evaluate

Add support for other languages for rouge

Open
#108 2 comments 4 reactions 0 assignees View on GitHub
Metric discussion
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

I calculate rouge with
```
from datasets import load_metric
rouge = load_metric("rouge")
rouge_output = rouge.compute(predictions=['тест тест привет'], references=['тест тест пока'], rouge_types=[
"rouge2"])["rouge2"].mid
print(rouge_output)
```
the result is
`Score(precision=0.0, recall=0.0, fmeasure=0.0)`
It seems like the `rouge_score` library that this metric uses filters all non-alphanueric latin characters
in `rouge_scorer/tokenize.py` with `text = re.sub(r"[^a-z0-9]+", " ", six.ensure_str(text))`.
Please add support for other languages.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.