Qwen3-30B-A3B: Checkpoint Save Failures on Large-Scale GPU Configurations (H100) and Small-Scale GB200 Systems
Open
bug
research
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
**Describe the bug**
Currently, the checkpoint fails to save for Qwen3-30B-A3B when using a higher number of GPUs under certain configurations.
For example:
64 nodes (512 GPUs):
TP=4, CP=8, PP=1, ETP=1, EMP=8
TP=4, CP=4, PP=1, ETP=1, EMP=8
32 nodes (256 GPUs):
TP=4, CP=16, PP=1, ETP=1, EMP=4
Contributor guide
Assessment
This issue has not been assessed yet.