[Feature] Support for Deepseek Arch (including Kimi-K2)

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Based on the roadmap, try to enable the draft model training of ones with Deepseek Arch, like Deepseek V3/R1/ Kimi k2. Considering that Kimi-K2 has no released MTP layer, it could be a natural starting point.

Milestones:

Please note that the following items are still in a draft state, and i may change them later.

• Generated dataset preparation, as the instructions of llama3. Note that the dataset cache cannot be built until the following steps done.
• Dataset cache built.
  • involve Deepseek backbone for target model (will consider three ways mentioned in this issue
  • involve Deepseek nextn (eagle3) backbone as the draft model (thanks for the support from (@FrankLeeeee )
• Train.
• Evaluate the accuracy.

Note

• Target model: Kimi K2 0905

Related resources

• instructions of llama3.
• the discussion about how to introduce new target model in spec
• one draft PR from another contributor

Start with docs/examples/llama3-eagle3.md and the target model config linked in the issue, then read issue #255 and draft PR #108 to understand the proposed integration paths. Done means the DeepSeek/Kimi-K2 dataset preparation and cache work, training, and accuracy evaluation milestones are completed.