huggingface / huggingface/evaluate

Documentation claims that `label_mapping` is an accepted input to `TokenClassificationEvaluator.compute()`; it is not

Open
#545 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

[This line](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/docs/source/base_evaluator.mdx#L133C2-L133C99) in the documentation states that `label_mapping` is an accepted parameter of [`TokenClassificationEvaluator.compute()`](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/src/evaluate/evaluator/token_classification.py#L220), when it in fact is not.

I suspect this line was absent-mindedly copied over from the `TextClassificationEvaluator` documentation (that class's [`compute()`](https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/src/evaluate/evaluator/text_classification.py#L91) method actually *does* accept this parameter).

In my opinion this would be a useful feature to implement (easily allowing users to evaluate models on datasets that don't use exactly the same labels), but at the very least this line should be removed from the documentation as long as the method doesn't support this parameter.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.