Add CP to Inkling

Goal

Add United Ulysses support to the Inkling model family, then progressively extend CP support to other applicable models, prioritizing commonly used MoE models.

Priority and timeline

Phase 1: Inkling — P0

Target completion: August 10, 2026.

• Add end-to-end Ulysses CP support for Inkling.
• Support the attention variants and distributed training configurations used by Inkling.
• Add representative training recipes and automated coverage.
• Validate a 64K sequence-length training configuration.
• Attempt 128K sequence length and document the maximum validated configuration if 128K is not feasible.

Phase 2: Other models

After Inkling is complete, extend CP support incrementally to other applicable models, prioritizing important MoE models.

This phase does not have the one-week deadline. Each model should be handled in a separate follow-up PR and include the same correctness, convergence, and performance evidence described below.

Validation requirements

Convergence + Performance comparison.

Deliverables

• Inkling CP implementation
• Reproducible training recipes on full Inkling-small model
• Convergence plots and raw metrics
• Performance and memory comparison

Assignee

Assigned to: @ghadiaravi13