Request for training recipe of on-policy KD
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Hello! Thank you for the excellent work. I’m trying to **reproduce Table 1** from the post (https://github.com/NVIDIA-NeMo/RL/discussions/1445) and would be very grateful for any guidance. A few targeted questions:
1. Could you share the training recipe used for on-policy KD in the post? (Qwen3-32B teacher, Qwen3-4B student, how many gpus did you used?)
2. If it's sharable.. could you share training logs? I'd like to refer the training curves of losses/evaluation metrics to reproduce the experiment. It would be a great help.
3. Could you tell some practical tips for on-policy kd training? for example ..
- The sensitivity for learning rate
- KL loss type (mixed for revered, kl weight)
- The appropriate performance gap between student and teacher?
- Tricks to avoid model collapse
Sorry for many questions and thanks again for sharing the great work on open-source community!
Best regards,
Jihwan
Contributor guide
Assessment
This issue has not been assessed yet.