NVIDIA-NeMo / NVIDIA-NeMo/RL

Request for training recipe of on-policy KD

Open
#1,527 11 comments 0 reactions 0 assignees View on GitHub
community-request enhancement example external t-onpolicydistillation waiting-on-customer x-kakao
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Hello! Thank you for the excellent work. I’m trying to **reproduce Table 1** from the post (https://github.com/NVIDIA-NeMo/RL/discussions/1445) and would be very grateful for any guidance. A few targeted questions:

1. Could you share the training recipe used for on-policy KD in the post? (Qwen3-32B teacher, Qwen3-4B student, how many gpus did you used?)

2. If it's sharable.. could you share training logs? I'd like to refer the training curves of losses/evaluation metrics to reproduce the experiment. It would be a great help.

3. Could you tell some practical tips for on-policy kd training? for example ..
- The sensitivity for learning rate
- KL loss type (mixed for revered, kl weight)
- The appropriate performance gap between student and teacher?
- Tricks to avoid model collapse

Sorry for many questions and thanks again for sharing the great work on open-source community!

Best regards,
Jihwan

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.