Question: Current thinking on Muon/Dion optimizer support for OLMo-core training runs

Hi all — I was looking through optimizer support in OLMo-core and noticed the Muon/Dion wrapper, including the Moonshot RMS scalar. I also noticed that the public OLMo-3 / OLMo Hybrid training configs appear to use `SkipStepAdamWConfig` rather than Muon.

I’m curious how the team is currently thinking about Muon for larger pre-training or mid-training runs. Was AdamW mainly chosen for consistency with prior runs / comparison hygiene, or were there benchmarking results around stability, throughput, memory footprint, or distributed communication behavior that made Muon less attractive for the "hero runs"?

I had previously [reproduced](https://huggingface.co/blog/bird-of-paradise/reproducing-and-validating-distributed-muon) a small-scale distributed Muon setup and found the tradeoff interesting: higher optimizer-step overhead, but lower optimizer-state memory compared with AdamW. Since the in-tree support now appears to use Microsoft’s optimized Dion implementation, I’m wondering whether there’s a useful contribution path around benchmarking or documenting the tradeoffs on smaller OLMo-core configs.