[Bug]: Qwen3.5-397B-A17B (bf16) OOMs host RAM on the PyTorch backend; AutoDeploy loads fine on the same node

System Info

• GPU: 8× NVIDIA H200 (SM90), 2,015 GB host RAM
• TensorRT-LLM: 1.3.0rc16 (verified against the source tree)
• Container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc16
• Model: Qwen3.5-397B-A17B (bf16; hybrid full-attention + Gated DeltaNet)
• Serving: trtllm-serve, TP8

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

trtllm-serve Qwen3.5-397B-A17B --backend pytorch --tp_size 8

Expected behavior

The model loads within available host RAM (it fits in GPU across 8×H200). The AutoDeploy backend loads the same checkpoint on the same node using ~69 GB committed host RAM (mmap-shared).

actual behavior

The PyTorch backend exhausts host RAM during weight load and OOMs the node. Measured peak committed host RAM (the OOM-relevant metric):

additional notes

Root cause (read against the 1.3.0rc16 tree):

1. The shared reader HfWeightLoader._load_safetensors_file (tensorrt_llm/_torch/models/checkpoints/hf/weight_loader.py) uses safetensors.torch.load_file (no mmap) → each of the 8 TP ranks materializes a private host copy.
2. Per-model load_weights diverges:
   • deepseek_v3 (modeling_deepseekv3.py) drains the host dict as it streams to GPU via .cuda() + weights.mark_consumed(...) → host stays bounded and releases.
   • qwen3_5 / qwen3_next (modeling_qwen3_next.py) does not call .cuda()/mark_consumed
     during load → the host weight dict is never drained → it accumulates → ~8× checkpoint →
     ~2 TB → OOM.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.