Request for 10B / 50B long-ctx training configs or hyperparameters

Hi OlmPool team,

Thanks for releasing OlmPool! We are very interested in this project and are currently looking into reproducing and extending some of the experiments.

I was wondering whether you could provide the training configs (or a description of the key hyperparameters) used for the **10B- and 50B-token long-ctx experiments**.

In particular, I'm curious about how the training horizon and learning-rate schedule were configured. For example, were both runs configured similarly to pretraining with the schedule defined for **50B tokens**, while the 10B run was obtained by a **hard stop at 10B tokens**? Or were separate schedules/hyperparameters used for the 10B and 50B runs?

Having the corresponding configs, or even just a brief description of the differences from the released configs, would be very helpful for reproduction.

Thanks again for releasing this work!