huggingface / huggingface/lighteval
[BUG] sync `LightevalTaskConfig` docstring with types/defaults
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Describe the bug
The docstring in `LightevalTaskConfig` ([link](https://github.com/huggingface/lighteval/blob/v0.9.2/src/lighteval/tasks/lighteval_task.py#L68-L89)) lags the type hints:
- `evaluation_splits` says `list[str]` but it's actually `ListLike` so it can be `tuple`. Also, the default of `validation` isn't documented
- Imo a default of `validation` should not be there, as most people just have `train`/`test`
- `few_shots_split` says `str`, but it's actually optional with a default of `None`
There's more cases (e.g. `generation_size`) but I didn't state them here.
## To Reproduce
N/a
## Expected behavior
Up-to-date docstrings. For typed code, it's a best practice to not to restate types in the docstring for this very reason.
## Version info
```none
lighteval==0.9.2
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.