Kind Request for Open-Sourcing Ablation Study Checkpoints

Hello OLMoE team, thank you for your wonderful work and for your contributions to open-source research! I am a researcher focusing on MoE and currently analyzing different MoE architecture designs.

I noticed that your paper provides extensive experiments on MoE design choices in the ablation study section and generously shares various training metrics. I was wondering if you would be willing to take it a step further by releasing the corresponding checkpoints from these ablation studies? I believe that having access to these checkpoints would greatly help researchers validate their ideas in a larger scale and further advance research in this area.

Once again, I sincerely appreciate your outstanding work!