Running the benchmark suite on very big models
Aperta
- Lingua principale
- Python
- Stelle
- 395
- Fork
- 105
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Hi, thank you very much for the OLMES work, it is great!
I am having some troubles in running the run_eval.py script on a 340B Megatron model. This model typically requires 16 A100 GPUs for the forward pass. in particular, I keep getting an out of memory error while I load the checkpoint files. I am running the script on a SLURM cluster, and it seems that simply increasing the number of assigned compute nodes does not solve the issue. Any tips on how to make the script work on very big models?
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.