huggingface / huggingface/evaluate
bug: huggingface-hub 1.0.0 release breaks existing evaluate releases
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
We encountered the following issues today with `pip3 install evaluate==0.4.5 rouge_score==0.1.2`. The issue is
```lib/python3.12/site-packages/evaluate/utils/file_utils.py", line 236, in get_authentication_headers_for_url
token = hf_api.HfFolder.get_token()
^^^^^^^^^^^^^^^
AttributeError: module 'huggingface_hub.hf_api' has no attribute 'HfFolder'
```
The script is simple:
```
import evaluate
def calculate_rouge_score(predictions, references):
"""
Compute Rouge score between predictions and references.
Args:
predictions: List of predictions.
references: List of references.
Returns:
Rouge score. Format: {
"rouge1": float,
"rouge2": float,
"rougeL": float,
"rougeLsum": float
}
"""
rouge = evaluate.load("rouge")
return rouge.compute(predictions=predictions, references=references)
```
This issue can be resolved after we fix `huggingface-hub==0.36.0`. It is due to https://github.com/huggingface/evaluate/blob/v0.4.5/setup.py#L54 does not check major version < 1.0.0 and pip uses latest available version. This affects previous stable versions. Could you enable stricter dependency checks?
Contributor guide
Assessment
This issue has not been assessed yet.