Release Qwen2.5-3B and Llama-3.1-8B 512K context checkpoints on Hugging Face

Hi @iofu728 🤗

I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work through Hugging Face's daily papers as yours got featured: https://huggingface.co/papers/2510.18830.
The paper page lets people discuss about your paper and lets them find artifacts about it (your models for instance),
you can also claim the paper as yours which will show up on your public profile at HF, add Github and project page URLs.

I saw that you successfully trained variants of Qwen2.5-3B and Llama-3.1-8B with expanded 512K context windows using the MTraining framework. While the code is available in your GitHub repository, I couldn't find the weights for these 512K context checkpoints.

Would you like to host these pre-trained model checkpoints on https://huggingface.co/models?
Hosting on Hugging Face will give your work much more visibility and enable better discoverability. We can add metadata tags (like "long-context") so that people find the models easier and link them directly to the paper page.

If you're down, leaving a guide here. Since these are based on the Qwen and Llama architectures, they would be very easy for the community to use directly with the transformers library.

After uploaded, we can also link the models to the paper page (read here) so people can discover your work.

Let me know if you're interested or need any guidance :)

Kind regards,

Niels

Start with the Hugging Face model-uploading guide and the model-card guidance linked in the issue. Done means the Qwen2.5-3B and Llama-3.1-8B 512K-context checkpoints are hosted on Hugging Face with metadata and linked to the paper page.