huggingface / huggingface/lighteval
[BUG] FileNotFoundError with newest lighteval 0.11.0
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Describe the bug
I receive a FileNotFoundError:
```
│ /home/jxm/anaconda3/lib/python3.12/site-packages/datasets/load.py:1033 in dataset_module_factory │
│ │
│ 1030 │ │ │ │ │ ) from None │
│ 1031 │ │ │ │ raise e1 from None │
│ 1032 │ else: │
│ ❱ 1033 │ │ raise FileNotFoundError(f"Couldn't find any data file at {relative_to_absolute_p │
│ 1034 │
│ 1035 │
│ 1036 def load_dataset_builder( │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: Couldn't find any data file at /home/jxm/lighteval/openai/gsm8k.
```
## To Reproduce
```bash
lighteval accelerate "model_name=gpt2" "lighteval|gsm8k|0|0"
```
## Expected behavior
I expect it to work and run evaluation.
## Version info
0.11.0 on linux with conda/uv pip
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.