modelscope / modelscope/DiffSynth-Studio

Failed to achieve official vram usage when trying wan_1.3b_vace.py, which reaches 17G vram usage with num_persistent_param_in_dit=0.

Open
#590 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

I followed the official guidance to set up my environment, and ran wan_1.3b_vace.py on one A6000 gpu with 48G vram.
According to the official statement, the vram usage should be 10G if num_persistent_param_in_dit=0 when running wan_14b_text_to_video.py and 48G if num_persistent_param_in_dit=None.

However, my gpu vram usage reached 17G when running wan_1.3b_vace.py with num_persistent_param_in_dit=0 and 20G when num_persistent_param_in_dit=None. This doesn't seem right. I saw others can get a vram usage in 6 to 8 G.
Then I tried to install flash-attn and sageattention as thoese previous issues mentioned and I made sure that they were used during inference, but this still didn't work.

I'm wondering whether this is right? I checked the vram management code, it seems that each time before using one model like text_encoder, call load_models_to_device, for example, self.load_models_to_device(["dit", "motion_controller", "vace"]). This will move all models except dit, motion_controller, and vace to the CPU and load the required models onto the onload device. Since the portion exceeding num_persistent_param_in_dit on the onload device is set to the CPU, it is uploaded to the GPU only during the forward pass. However, this does not reduce the peak VRAM, as the final dit model parameters are still loaded onto the GPU before denoise ends. A more reasonable logic would be to unload this part of the parameters to the CPU after the forward pass?

I was wondering whether I understand it correctly, can anybody help me out?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported usage with wan_1.3b_vace.py for both num_persistent_param_in_dit=0 and None. Trace load_models_to_device and the denoise flow to compare the observed peak VRAM with the official guidance. Done means determining whether the peak is expected and, if not, identifying the specific VRAM-management behavior responsible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.