Trying to run wan_14b_text_to_video.py example on a 12G card, tries to allocate 160 GiB of VRAM and fails

I edited the "wan_14b_text_to_video.py" example script to set "num_persistent_param_in_dit=0", and I'm using torch_dtype=torch.float8_e4m3fn, so per the README it should only use 10G VRAM. My GPU has 12GB so it should work - but when I try to execute I get the error "torch.OutOfMemoryError: HIP out of memory. Tried to allocate 159.92 GiB. GPU 0 has a total capacity of 11.98 GiB of which 5.95 GiB is free" and the script exits. I've no idea why the model is trying to allocate nearly 160G of VRAM?

Note the GPU is a 6700 XT and I'm using the ROCM version of torch. The same model is working fine in ComfyUI (though slowly, and I notice it gives a warning saying it has to use tiled VAE), so I could carry on using that, but I'd rather use DiffSynth-Studio if possible.

Any suggestions?

Full run log:

Downloading Model to directory: /home/karl/DiffSynth-Studio/examples/wanvideo/models/Wan-AI/Wan2.1-T2V-14B 2025-03-18 18:34:49,170 - modelscope - INFO - Target directory already exists, skipping creation. Loading models from: ['models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00001-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00002-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00003-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00004-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00005-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00006-of-00006.safetensors'] model_name: wan_video_dit model_class: WanModel This model is initialized with extra kwargs: {'has_image_input': False, 'patch_size': [1, 2, 2], 'in_dim': 16, 'dim': 5120, 'ffn_dim': 13824, 'freq_dim': 256, 'text_dim': 4096, 'out_dim': 16, 'num_heads': 40, 'num_layers': 40, 'eps': 1e-06} The following models are loaded: ['wan_video_dit']. Loading models from: models/Wan-AI/Wan2.1-T2V-14B/models_t5_umt5-xxl-enc-bf16.pth model_name: wan_video_text_encoder model_class: WanTextEncoder The following models are loaded: ['wan_video_text_encoder']. Loading models from: models/Wan-AI/Wan2.1-T2V-14B/Wan2.1_VAE.pth model_name: wan_video_vae model_class: WanVideoVAE The following models are loaded: ['wan_video_vae']. Using wan_video_text_encoder from models/Wan-AI/Wan2.1-T2V-14B/models_t5_umt5-xxl-enc-bf16.pth. Using wan_video_dit from ['models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00001-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00002-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00003-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00004-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00005-of-00006.safetensors', 'models/Wan-AI/Wan2.1-T2V-14B/diffusion_pytorch_model-00006-of-00006.safetensors']. Using wan_video_vae from models/Wan-AI/Wan2.1-T2V-14B/Wan2.1_VAE.pth. No wan_video_image_encoder models available. /home/karl/DiffSynth-Studio/diffsynth/pipelines/wan_video.py:232: UserWarning: expandable_segments not supported on this platform (Triggered internally at /pytorch/c10/hip/HIPAllocatorConfig.h:29.) noise = noise.to(dtype=self.torch_dtype, device=self.device) 0%| | 0/50 [00:01<?, ?it/s] Traceback (most recent call last): File "/home/karl/DiffSynth-Studio/examples/wanvideo/./wan_14b_text_to_video.py", line 30, in <module> video = pipe( ^^^^^ File "/home/karl/DiffSynth-Studio/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/diffsynth/pipelines/wan_video.py", line 268, in __call__ noise_pred_posi = model_fn_wan_video(self.dit, latents, timestep=timestep, **prompt_emb_posi, **image_emb, **extra_input, **tea_cache_posi) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/diffsynth/pipelines/wan_video.py", line 379, in model_fn_wan_video x = block(x, context, t_mod, freqs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/diffsynth/models/wan_video_dit.py", line 197, in forward x = x + gate_msa * self.self_attn(input_x, freqs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/diffsynth/models/wan_video_dit.py", line 129, in forward x = flash_attention( ^^^^^^^^^^^^^^^^ File "/home/karl/DiffSynth-Studio/diffsynth/models/wan_video_dit.py", line 56, in flash_attention x = F.scaled_dot_product_attention(q, k, v) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.OutOfMemoryError: HIP out of memory. Tried to allocate 159.92 GiB. GPU 0 has a total capacity of 11.98 GiB of which 5.95 GiB is free. Of the allocated memory 5.39 GiB is allocated by PyTorch, and 345.27 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_HIP_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Environment variables (same as I used successfully with ComfyUI to run the same model):

export HSA_OVERRIDE_GFX_VERSION=10.3.0 export HIP_VISIBLE_DEVICES=1 export PYTORCH_HIP_ALLOC_CONF=expandable_segments:True

Start with examples/wanvideo/wan_14b_text_to_video.py and the README settings for num_persistent_param_in_dit and torch_dtype. Reproduce the failure on the reported ROCm environment, then trace the call through diffsynth/pipelines/wan_video.py into diffsynth/models/wan_video_dit.py, especially flash_attention. Done means the cause of the excessive allocation is identified and the supported 12G-card behavior or required limitation is documented or fixed.