huggingface / huggingface/evaluate
Documentation claims that `label_mapping` is an accepted input to `TokenClassificationEvaluator.compute()`; it is not
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
[This line](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/docs/source/base_evaluator.mdx#L133C2-L133C99) in the documentation states that `label_mapping` is an accepted parameter of [`TokenClassificationEvaluator.compute()`](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/src/evaluate/evaluator/token_classification.py#L220), when it in fact is not.
I suspect this line was absent-mindedly copied over from the `TextClassificationEvaluator` documentation (that class's [`compute()`](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/src/evaluate/evaluator/text_classification.py#L91) method actually *does* accept this parameter).
In my opinion this would be a useful feature to implement (easily allowing users to evaluate models on datasets that don't use exactly the same labels), but at the very least this line should be removed from the documentation as long as the method doesn't support this parameter.
Contributor guide
Assessment
This issue has not been assessed yet.