NVIDIA-NeMo / NVIDIA-NeMo/RL

[Feature] Explicit failure for unmatched model and checkpoints

Open
#415 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Currently nemo-rl always tries to resume from the last checkpoint in the checkpoint path. When we change the policy model, the new model will fail silently at loading old checkpoints, resulting in two negative consequences:

1. New checkpoints will overwrite old checkpoints from a different model.
2. The training step is counted from the old checkpoint, even if the new model is actually trained from scratch.

I feel it's better to fail explicitly when the policy model doesn't match the checkpoints, to prevent such undefined behaviors.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.