huggingface / huggingface/lighteval
[BUG] Cuda OOM with gpt-oss-20b
Open
bug
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Describe the bug
A clear and concise description of what the bug is.
`lighteval accelerate "model_name=openai/gpt-oss-20b,max_length=4096,skip_special_tokens=False,batch_size=1" "leaderboard|mmlu|5" --save-details --output-dir "openai_scores"`
goes OOM (lighteval installed from main, kernels 0.10.1, accelerate 1.6.0, transformers 4.56.1)
using a RTX 5090
can this be fixed with some other settings?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.