Megatron has left-over memory after refit completed
Open
bug
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Ideally the memory should be exactly equal to 0 after refit completed, but megatron will have some left.
```
> assert current_allocated == 0.0, "Memory should be 0 after refit completed"
E AssertionError: Memory should be 0 after refit completed
E assert 0.00244140625 == 0.0
```
Repro:
Set `train_backend == megatron` in the below unit test and run it.
https://github.com/NVIDIA-NeMo/RL/blob/ba46741f081b6a71a68af1d884c71f65b4da80f4/tests/unit/models/generation/test_vllm_generation.py#L1667-L1669.
Contributor guide
Assessment
This issue has not been assessed yet.