huggingface / huggingface/lighteval
[BUG] incorrect type hints such as `callable`
Open
bug
documentation
feature
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Describe the bug
Reading the code, it's clear this code base can really benefit from type checking, running vanilla `mypy` reveals over 1.2k type errors.
For example, `callable` is not a valid type hint: https://github.com/huggingface/lighteval/blob/v0.9.2/src/lighteval/metrics/utils/metric_utils.py#L62-L63
## To Reproduce
```bash
uv sync --extra dev
uv pip install mypy==0.15.0
mypy src
```
## Expected behavior
Type hints to be correct.
## Version info
`lighteval==0.9.2`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.