8*A800 训练Qwen3.5-35-a3b OOM

Checklist / 检查清单

• I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues，确认这是一个新的问题与讨论。

Question Description / 问题描述

报错内容
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 5 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 7 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 4 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 1 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 6 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 0 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 2 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:05 [parallel_state.py:1715] rank 3 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [base.py:106] Offloader set to NoopOffloader
INFO 03-12 11:58:06 [gpu_model_runner.py:4281] Starting to load model /data_nvme0n1/models/Qwen3.5-35B-A3B...
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
INFO 03-12 11:58:07 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
INFO 03-12 11:58:07 [unquantized.py:186] Using TRITON backend for Unquantized MoE
INFO 03-12 11:58:07 [cuda.py:405] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
INFO 03-12 11:58:07 [flash_attn.py:587] Using FlashAttention version 2
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
INFO 03-12 11:58:10 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:10 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:10 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
INFO 03-12 11:58:12 [gpu_model_runner.py:4364] Model loading took 65.53 GiB memory and 3.672805 seconds
INFO 03-12 11:58:12 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:12 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:12 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:12 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:13 [gpu_model_runner.py:5280] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
INFO 03-12 11:58:17 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_4_0/model
INFO 03-12 11:58:17 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_6_0/model
INFO 03-12 11:58:17 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_1_0/model
INFO 03-12 11:58:19 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_0_0/model
INFO 03-12 11:58:19 [backends.py:916] Using cache directory: /root/.cache/vllm/torch_compile_cache/6afc389cdd/rank_0_0/backbone for vLLM's torch.compile
INFO 03-12 11:58:19 [backends.py:976] Dynamo bytecode transform time: 1.97 s
INFO 03-12 11:58:19 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_7_0/model
INFO 03-12 11:58:19 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_5_0/model
INFO 03-12 11:58:19 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_2_0/model
INFO 03-12 11:58:19 [decorators.py:465] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/ab025569b1a0bf5f28bd377302fdd972a26afff2a536b0b6df422eaacea20c78/rank_3_0/model
WARNING 03-12 11:58:20 [fused_moe.py:1093] Using default MoE config. Performance might be sub-optimal! Config file not found at /data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/model_executor/layers/fused_moe/configs/E=256,N=512,device_name=NVIDIA_A800-SXM4-80GB.json
INFO 03-12 11:58:21 [backends.py:266] Directly load the compiled graph(s) for compile range (1, 2048) from the cache, took 1.588 s
INFO 03-12 11:58:21 [monitor.py:35] torch.compile takes 3.91 s in total
INFO 03-12 11:58:22 [gpu_worker.py:424] Available KV cache memory: -27.82 GiB
[rank0]: Traceback (most recent call last):
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/rollout_mixin.py", line 579, in offload_context
[rank0]: yield
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/rollout_mixin.py", line 222, in _init_rollout_engine
[rank0]: self.engine = self._prepare_vllm_engine()
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/rollout_mixin.py", line 248, in _prepare_vllm_engine
[rank0]: engine = GRPOVllmEngine(
[rank0]: ^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/infer_engine/vllm_engine.py", line 161, in init
[rank0]: self._prepare_engine()
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/infer_engine/vllm_engine.py", line 183, in _prepare_engine
[rank0]: engine = llm_engine_cls.from_engine_args(self.engine_args)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py", line 177, in from_engine_args
[rank0]: return cls(
[rank0]: ^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py", line 111, in init
[rank0]: self.engine_core = EngineCoreClient.make_client(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 100, in make_client
[rank0]: return InprocClient(vllm_config, executor_class, log_stats)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 282, in init
[rank0]: self.engine_core = EngineCore(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 120, in init
[rank0]: num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
[rank0]: return func(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 263, in _initialize_kv_caches
[rank0]: kv_cache_configs = get_kv_cache_configs(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1572, in get_kv_cache_configs
[rank0]: _check_enough_kv_cache_memory(
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 623, in _check_enough_kv_cache_memory
[rank0]: raise ValueError(
[rank0]: ValueError: No available memory for the cache blocks. Try increasing gpu_memory_utilization when initializing the engine. See https://docs.vllm.ai/en/latest/configuration/conserving_memory/ for more details.

[rank0]: During handling of the above exception, another exception occurred:

[rank0]: Traceback (most recent call last):
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/cli/_megatron/rlhf.py", line 7, in
[rank0]: megatron_rlhf_main()
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/pipelines/train/rlhf.py", line 73, in megatron_rlhf_main
[rank0]: return MegatronRLHF(args).main()
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/pipelines/base.py", line 52, in main
[rank0]: result = self.run()
[rank0]: ^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/pipelines/train/sft.py", line 65, in run
[rank0]: trainer = self.prepare_trainer()
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/pipelines/train/rlhf.py", line 34, in prepare_trainer
[rank0]: return trainer_cls(args, self.template, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/grpo_trainer.py", line 54, in init
[rank0]: self._init_rollout_engine()
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/grpo_trainer.py", line 122, in _init_rollout_engine
[rank0]: super()._init_rollout_engine()
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/rollout_mixin.py", line 220, in init_rollout_engine
[rank0]: with context():
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/contextlib.py", line 155, in exit
[rank0]: self.gen.throw(value)
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/rollout_mixin.py", line 582, in offload_context
[rank0]: load_megatron_model_to_gpu(self.wrapped_models)
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/torch/utils/contextlib.py", line 124, in decorate_context
[rank0]: return func(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/data_nvme0n1/zpk/ms-swift/swift/megatron/trainers/utils.py", line 128, in load_megatron_model_to_gpu
[rank0]: buffer.param_data.storage().resize(buffer.param_data_size)
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/torch/storage.py", line 1257, in resize
[rank0]: self.resize(size)
[rank0]: File "/data_nvme0n1/conda/envs/ms-swift/lib/python3.12/site-packages/torch/storage.py", line 1261, in resize
[rank0]: self.untyped_storage.resize(size * self._element_size())
[rank0]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.50 GiB. GPU 0 has a total capacity of 79.25 GiB of which 975.94 MiB is free. Including non-PyTorch memory, this process has 78.25 GiB memory in use. Of the allocated memory 77.28 GiB is allocated by PyTorch, with 26.83 MiB allocated in private pools (e.g., CUDA Graphs), and 66.45 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
环境配置 ：
Package Version Editable project location

abnf 2.2.0
absl-py 2.4.0
accelerate 1.12.0
addict 2.4.0
aiobotocore 3.2.0
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.13.3
aioitertools 0.13.0
aiosignal 1.4.0
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
aniso8601 10.0.1
annotated-doc 0.0.4
annotated-types 0.7.0
anthropic 0.84.0
antlr4-python3-runtime 4.9.3
anyio 4.12.1
apache-tvm-ffi 0.1.8.post2
apex 0.1
astor 0.8.1
attrdict 2.0.1
attrs 25.4.0
av 15.1.0
backoff 2.2.1
binpacking 2.0.1
bitarray 3.8.0
bitstring 4.3.1
blake3 1.0.8
blinker 1.9.0
botocore 1.42.55
braceexpand 0.1.7
bracex 2.6
brotli 1.2.0
cachetools 7.0.1
causal_conv1d 1.6.1
cbor2 5.8.0
certifi 2026.2.25
cffi 2.0.0
chardet 7.0.1
charset-normalizer 3.4.4
cint 1.0.0
click 8.3.1
cloudpickle 3.1.2
compressed-tensors 0.13.0
contourpy 1.3.3
cpm-kernels 1.0.11
crcmod 1.7
cryptography 46.0.5
cuda-bindings 12.9.4
cuda-pathfinder 1.4.0
cuda-python 12.9.4
cupy-cuda12x 14.0.1
cycler 0.12.1
dacite 1.9.2
datasets 3.6.0
decord 0.6.0
deepspeed 0.18.7
defusedxml 0.7.1
Deprecated 1.3.1
depyf 0.20.0
dill 0.3.8
diskcache 5.6.3
distro 1.9.0
dnspython 2.8.0
docstring_parser 0.17.0
ebmlite 3.4.1
einops 0.8.2
email-validator 2.3.0
fastapi 0.133.1
fastapi-cli 0.0.24
fastapi-cloud-cli 0.14.0
fastar 0.8.0
ffmpy 1.0.0
fickling 0.1.9
filelock 3.24.3
filetype 1.2.0
flash_attn 2.8.3
flash-linear-attention 0.4.2
flashinfer-python 0.6.4
Flask 3.1.3
Flask-RESTful 0.3.10
fonttools 4.61.1
frozenlist 1.8.0
fsspec 2025.3.0
gguf 0.17.1
gitdb 4.0.12
GitPython 3.1.46
googleapis-common-protos 1.72.0
gql 4.0.0
gradio 5.50.0
gradio_client 1.14.0
graphql-core 3.2.8
graphviz 0.21
groovy 0.1.2
grpcio 1.78.1
grpcio-reflection 1.78.1
h11 0.16.0
hf-xet 1.3.2
hjson 3.1.0
httpcore 1.0.9
httptools 0.7.1
httpx 0.28.1
httpx-sse 0.4.3
huggingface_hub 1.6.0
idna 3.11
ijson 3.5.0
importlib_metadata 8.6.1
interegular 0.3.3
intervaltree 3.2.1
itsdangerous 2.2.0
jieba 0.42.1
Jinja2 3.1.6
jiter 0.13.0
jmespath 0.10.0
joblib 1.5.3
json_repair 0.58.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
kaitaistruct 0.11
kaldi-native-fbank 1.22.3
kiwisolver 1.4.9
lark 1.2.2
liger_kernel 0.7.0
llguidance 1.3.0
llvmlite 0.44.0
lm-format-enforcer 0.11.3
loguru 0.7.3
mamba_ssm 2.3.0
Markdown 3.10.2
markdown-it-py 4.0.0
MarkupSafe 3.0.3
matplotlib 3.10.8
mcp 1.26.0
mdurl 0.1.2
megatron-core 0.15.3
megatron-energon 6.0.1
mistral_common 1.9.1
ml_dtypes 0.5.4
model-hosting-container-standards 0.1.13
modelscope 1.34.0
mpmath 1.3.0
ms_swift 4.1.0.dev0 /data_nvme0n1/zpk/ms-swift
msgpack 1.1.2
msgspec 0.20.0
multi-storage-client 0.42.0
multidict 6.7.1
multiprocess 0.70.16
networkx 3.6.1
ninja 1.13.0
nltk 3.9.3
numba 0.61.2
numpy 2.2.6
nv_grouped_gemm 1.1.4.post8
nvdlfw_inspect 0.2.2
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cudnn-frontend 1.18.0
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-cutlass-dsl 4.4.1
nvidia-cutlass-dsl-libs-base 4.4.1
nvidia-ml-py 13.590.48
nvidia-modelopt 0.41.0
nvidia-nccl-cu12 2.27.5
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvshmem-cu12 3.4.5
nvidia-nvtx-cu12 12.8.90
nvidia-resiliency-ext 0.4.1
nvtx 0.2.14
omegaconf 2.3.0
onnx 1.20.1
onnx-ir 0.2.0
onnxscript 0.6.2
openai 2.24.0
openai-harmony 0.0.8
opencv-python-headless 4.13.0.92
opentelemetry-api 1.39.1
opentelemetry-exporter-otlp 1.39.1
opentelemetry-exporter-otlp-proto-common 1.39.1
opentelemetry-exporter-otlp-proto-grpc 1.39.1
opentelemetry-exporter-otlp-proto-http 1.39.1
opentelemetry-proto 1.39.1
opentelemetry-sdk 1.39.1
opentelemetry-semantic-conventions 0.60b1
opentelemetry-semantic-conventions-ai 0.4.14
orjson 3.11.7
oss2 2.19.1
outlines_core 0.2.11
packaging 25.0
pandas 2.3.3
partial-json-parser 0.2.1.1.post7
pdfminer.six 20260107
peft 0.18.1
pillow 11.3.0
pip 26.0.1
platformdirs 4.9.2
polyfile-weave 0.5.9
prettytable 3.17.0
prometheus_client 0.24.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.4.1
protobuf 6.33.5
psutil 7.2.2
PuLP 3.3.0
py-cpuinfo 9.0.0
pyarrow 23.0.1
pybase64 1.4.3
pybind11 3.0.2
pycountry 26.2.16
pycparser 3.0
pycryptodome 3.23.0
pydantic 2.12.3
pydantic_core 2.41.4
pydantic-extra-types 2.11.0
pydantic-settings 2.13.1
pydub 0.25.1
pyecharts 2.1.0
Pygments 2.19.2
PyJWT 2.11.0
pynvml 13.0.1
pyparsing 3.3.2
python-dateutil 2.9.0.post0
python-dotenv 1.2.1
python-json-logger 4.0.0
python-multipart 0.0.22
pytz 2025.2
PyYAML 6.0.3
pyzmq 27.1.0
quack-kernels 0.3.2
qwen-vl-utils 0.0.14
ray 2.54.0
referencing 0.37.0
regex 2026.2.19
requests 2.32.5
rich 13.9.4
rich-toolkit 0.19.7
rignore 0.7.6
rouge 1.0.1
rpds-py 0.30.0
ruff 0.15.2
s3fs 2026.2.0
safehttpx 0.1.7
safetensors 0.7.0
scipy 1.17.1
semantic-version 2.10.0
sentencepiece 0.2.1
sentry-sdk 2.53.0
setproctitle 1.3.7
setuptools 79.0.1
shellingham 1.5.4
simplejson 3.20.2
six 1.17.0
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
soundfile 0.13.1
sse-starlette 3.2.0
starlette 0.52.1
supervisor 4.3.0
swanlab 0.7.11
sympy 1.14.0
tabulate 0.9.0
tenacity 9.1.4
tensorboard 2.20.0
tensorboard-data-server 0.7.2
tensorstore 0.1.81
tiktoken 0.12.0
tokenizers 0.22.2
tomlkit 0.13.3
torch 2.10.0
torch_c_dlpack_ext 0.1.5
torchaudio 2.10.0+cu129
torchvision 0.25.0+cu129
tqdm 4.67.3
transformer_engine 2.9.0
transformer_engine_cu12 2.9.0
transformer_engine_torch 2.9.0
transformers 5.3.0
transformers-stream-generator 0.0.5
triton 3.6.0
trl 0.28.0
typer 0.24.1
typer-slim 0.24.0
typing_extensions 4.15.0
typing-inspection 0.4.2
tzdata 2025.3
urllib3 2.6.3
uvicorn 0.41.0
uvloop 0.22.1
vllm 0.17.1
wandb 0.25.0
watchfiles 1.1.1
wcmatch 10.1
wcwidth 0.6.0
weave 0.52.31
webdataset 1.0.2
websockets 15.0.1
Werkzeug 3.1.6
wget 3.2
wheel 0.46.3
wrapt 2.1.1
xattr 1.3.0
xgrammar 0.1.29
xxhash 3.6.0
yarl 1.22.0
zipp 3.23.0
zstandard 0.25.0

训练脚本
SYSTEM_PROMPT="""You are a helpful math assistant. Solve the problem step by step and put your final answer within \boxed{}."""
SKIP_MULTIMODAL_MTP_VALIDATION=1 \

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
NPROC_PER_NODE=8
megatron rlhf
--rlhf_type grpo
--loss_type sapo
--tau_pos 1
--tau_neg 1.05
--model /data_nvme0n1/models/Qwen3.5-35B-A3B
--expert_model_parallel_size 4
--add_non_thinking_prefix true
--optimizer_offload_fraction 1
--sequence_parallel true
--tensor_model_parallel_size 2
--micro_batch_size 1
--global_batch_size 16
--freeze_vit true
--freeze_aligner true
--finetune true
--use_precision_aware_optimizer true
--moe_expert_capacity_factor 2
--recompute_granularity full
--recompute_method uniform
--recompute_num_layers 4
--pipeline_model_parallel_size 2
--external_plugins examples/train/grpo/plugin/gsm8k/gsm8k_plugin.py
--reward_funcs gsm8k_accuracy gsm8k_format
--columns '{"answer": "solution"}'
--save_safetensors true
--enable_thinking false
--use_vllm true
--vllm_mode colocate
--vllm_gpu_memory_utilization 0.5
--vllm_max_model_len 8192
--sleep_level 1
--tuner_type full
--bf16 true
--importance_sampling_level token
--dataset 'gsm8k'
--load_from_cache_file true
--max_length 2048
--max_completion_length 8192
--num_train_epochs 1
--save_steps 10
--save_total_limit 100
--logging_steps 1
--dataloader_num_workers 4
--num_generations 4
--temperature 1.0
--system "$SYSTEM_PROMPT"
--report_to tensorboard swanlab
--epsilon 0.2
--epsilon_high 0.28
--scale_rewards none
--offload_model true
--offload_optimizer true
--no_save_optim
--no_save_rng
--attention_backend flash

Start with the reported entry points in swift/megatron/trainers/rollout_mixin.py and swift/infer_engine/vllm_engine.py, then inspect the gpu_worker.py:424 log where available KV cache memory becomes negative. Reproduce the 8-GPU Qwen3.5-35B-A3B setup and determine the conditions causing the OOM; completion requires an identified cause and a verified resolution or documented configuration.