huggingface / huggingface/alignment-handbook
Having troubles changing save_steps parameter for a resumed job.
- Dominant language
- Python
- Stars
- 5.7k
- Forks
- 490
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
I'm having next issue. Let say I'm starting a job
```
ACCELERATE_LOG_LEVEL=info accelerate launch --config_file recipes/accelerate_configs/deepspeed_zero3.yaml scripts/run_sft.py recipes/zephyr-7b-beta/sft/config_full.yaml
```
In `config_full.yaml` I would have `save_steps: 1000`. At some point I would realize that 1000 is too frequent of a step to save, so I stop the job, edit `config_full.yaml` to have `save_steps: 10000` and restart the job. The resume from checkpoint goes as planned, however I would still have checkpoints saved every 1000 steps (original parameter). What do I do wrong?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.