NVIDIA-NeMo / NVIDIA-NeMo/RL

CLI overrides don't get propagated for inherited configs

Open
#1,483 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

**Describe the bug**

When a config inherits from another object like with the [teacher config for distillation](https://github.com/NVIDIA-NeMo/RL/blob/main/examples/configs/distillation_math.yaml#L196), overriding the parent config's settings don't get reflected in the child's config.

For example, the `teacher` config in distillation uses the same config as the `policy` but changes the `model_name` and a few parallelism sizes. If you were to change something in the policy by overriding it in the CLI, like `policy.max_total_sequence_length`, it will only update the policy setting, but not the teacher. This will cause problems when using larger sequence lengths as it can throw errors with the mismatch.

While settings like these could instead be overriden for both configs, it can be confusing for users as glancing at the YAML file only shows the settings in one place, so it is not obvious that it needs to be updated again, or that it isn't updated in both locations.

**Steps/Code to reproduce bug**

1. Setup the NeMo-RL container and run a RayCluster.
2. Run the `examples/run_distillation_math.py` example with `uv run examples/run_distillation_math.py policy.max_total_sequence_length`.
3. Observe the `max_total_sequence_length` for the teacher.

**Expected behavior**

If I override a config value that another config inherits from, I would expect both of them to be updated. Otherwise, it is confusing what value it would be using.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.