aigc-apps / aigc-apps/VideoX-Fun

Speeding up inference for Wan2.2-VACE-fun-A14B, sage-attention and batch inference?

Open
#414 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.3k
Forks
188
Avg merge
2d 22h
Merged PRs (30d)
3

Description

I have been running multi-gpu inference for Wan-2.2-VACE for resolution of 512x512 videos with 121 frames.
It takes about 50 seconds to do 20 diffusion steps on 8xH200s, using this config:

ulysses_degree      = 4
ring_degree         = 2
fsdp_dit            = False
fsdp_text_encoder   = True
compile_dit         = True

enable_teacache     = True
teacache_threshold  = 0.10
num_skip_start_steps = 5
teacache_offload    = False

cfg_skip_ratio      = 0.1

# Riflex config
enable_riflex       = False
# Index of intrinsic frequency
riflex_k            = 6

# Config and model path
config_path         = "config/wan2.2/wan_civitai_t2v.yaml"
# model path
model_name          = "models/Diffusion_Transformer/Wan2.2-VACE-Fun-A14B"

# Choose the sampler in "Flow", "Flow_Unipc", "Flow_DPM++"
sampler_name        = "Flow"
# [NOTE]: Noise schedule shift parameter. Affects temporal dynamics. 
# Used when the sampler is in "Flow_Unipc", "Flow_DPM++".
shift               = 12.0

# Other params
sample_size         = [512, 512]
video_length        = 121
fps                 = 60

weight_dtype        = torch.bfloat16
control_video       = "outputs-1/input_videos/1.mp4"
start_image         = None
end_image           = None
# Use inpaint video instead of start image and end image.
inpaint_video       = None
inpaint_video_mask  = None
subject_ref_images  = None
vace_context_scale  = 1.00
padding_in_subject_ref_images   = True

guidance_scale          = 5.0
seed                    = 43
num_inference_steps     = 20

It is possible to use sage-attention with the model?
I tried flash attention but it did not offer any speedups compared to torch SDPA,
I installed sage attention but the model does not seem to be using it.

Is it possible to do batch inference (i.e. generating two videos using 2 differenct control videos on the same GPU since, max memory util is 70GB on H200?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the supplied Wan2.2-VACE configuration and the current torch SDPA, flash-attention, and sage-attention setup. Measure the existing 20-step, 512x512, 121-frame run on 8xH200s, then investigate whether attention selection and two-video batch inference are supported. Done means documenting or implementing a verified speedup and confirming memory and output behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.