what is the baseline for your md <RL/docs/guides/muon-optimizer.md>
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
**The value change of muon_extra_scale_factor in post-training**
Recently, I try to use muon opt in my experiments, then I try to get the correct hyperparameters for my muon optimizer. I try to get these hyperparameters from the existed experiment, and I find that your experiment in muon for post-training in muon-optimizer.md use muon_extra_scale_factor = 0.5 rather then muon_extra_scale_factor = 0.2 for SFT. I am confuse about your change for muon_extra_scale_factor, does it means muon need muon_extra_scale_factor to be larger in post-training then in SFT?
**what I need**
+ what is the baseline of post-training in muon-optimizer.md, please give me the launch scripts or commands.
+ why changed muon_extra_scale_factor from 0.2 to 0.5. Does it make sense?
Contributor guide
Assessment
This issue has not been assessed yet.