[Question] How to add a custom auxiliary SFT loss to the GRPO training loop （Chord）?

Your Question

How to add a custom auxiliary SFT loss to the GRPO training loop?

I'm training DeepSeek-V3.2 on 128×H200 with GRPO and would like to mix in an SFT loss term during training, similar to CHORD (https://arxiv.org/abs/2508.11408):

$$\mathcal{L} = (1 - \mu) \cdot \mathcal{L}{\text{GRPO}} + \mu \cdot \mathcal{L}{\text{SFT}}$$

Could you point me to where in the codebase the GRPO loss is computed so I can add the SFT forward pass and combine the two losses?

Also, one question about MoE: routing replay forces the same expert routing from rollout during training, but SFT data has no corresponding rollout. Should I just skip routing replay for the SFT forward pass and use live routing?

Thanks!

What I've Tried

Verified CHORD on smaller models using ms-swift, but ms-swift only supports vLLM for GRPO rollout and lacks routing replay / off-policy sequence masking, so it doesn't work for DeepSeek-V3.2 at this scale.

Environment (if relevant)

• slime version:
• Python version:
• PyTorch version:
• CUDA/ROCm version:
• GPU type and count:
• OS:

Additional Context

No response

Pre-submission Checklist

• I have read the CONTRIBUTING.md and understand the collaboration scope.
• I have read the documentation and FAQ and my question is not answered there.
• I have searched for existing issues and my question has not been asked before.

No source file, test, or entry point is named in the issue. Start by locating the GRPO loss computation and routing-replay implementation, then determine how an SFT forward pass would interact with both; done would be a maintainer-confirmed design for combining the losses and handling SFT routing.