layer-by-layer comparison tool between policy/generation
Open
enhancement
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Automodel has added this nice util to compare HF and custom models layer by layer https://github.com/NVIDIA-NeMo/Automodel/pull/787
We should add a utility that does this.
We already have one util https://github.com/NVIDIA-NeMo/RL/blob/main/tools/refit_verifier.py that does e2e forward pass comparison, but most of the time we need to compare layer-by-layer once we see the difference
Contributor guide
Assessment
This issue has not been assessed yet.