thu-ml / thu-ml/TurboDiffusion
5090 Video Generation is around 79s (reported 38s).
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.7k
- Forks
- 277
- Avg merge
- 2h 57m
- Merged PRs (30d)
- 2
Description
Hi author,
Thanks for your great work! I ran the code on my 5090, and the average time assumption for SLA and SageSLA is 83s and 78s, respectively, which is double what you reported in your code. Any ideas on what is wrong with the problem? I only added some logs to record inference time.
INFERENCE CODE:
python turbodiffusion/inference/wan2.2_i2v_infer.py \
--model Wan2.2-A14B \
--low_noise_model_path checkpoints/TurboWan2.2-I2V-A14B-low-720P-quant.pth \
--high_noise_model_path checkpoints/TurboWan2.2-I2V-A14B-high-720P-quant.pth \
--resolution 720p \
--adaptive_resolution \
--image_path test_image.jpg \
--prompt "Starts with a boy looking at the box of cereal box in front of him. Boy takes the spoon in his hand and scoops up some cereal from his bowl and eats it. He then gives a thumbs up with his other hand. " \
--num_samples 1 \
--num_steps 4 \
--quant_linear \
--attention_type sagesla \
--sla_topk 0.1 \
--ode
OUTPUT LOGS:
[12-26 05:27:40|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:99:<module>] Computing embedding for prompt: Starts with a boy looking at the box of cereal box in front of him. Boy takes the spoon in his hand and scoops up some cereal from his bowl and eats it. He then gives a thumbs up with his other hand.
[12-26 05:27:40|INFO|turbodiffusion/rcm/utils/umt5.py:495:__init__] loading checkpoints/models_t5_umt5-xxl-enc-bf16.pth
[12-26 05:27:53|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:104:<module>] Loading and preprocessing image from: /home/user/img2video_online//2.webp
[12-26 05:27:53|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:107:<module>] Adaptive resolution mode enabled.
[12-26 05:27:53|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:110:<module>] Target area is based on 720p 16:9 (~921600 pixels).
[12-26 05:27:53|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:124:<module>] Input image aspect ratio: 1.7778. Adaptive resolution set to: 720x1280
[12-26 05:27:53|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:134:<module>] Preprocessing image to 720x1280...
[12-26 05:27:58|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:160:<module>] Generating with prompt: Starts with a boy looking at the box of cereal box in front of him. Boy takes the spoon in his hand and scoops up some cereal from his bowl and eats it. He then gives a thumbs up with his other hand.
Sampling: 0%| | 0/4 [00:00<?, ?it/s]
Sampling: 25%|██▌ | 1/4 [00:06<00:20, 6.87s/it]
Sampling: 50%|█████ | 2/4 [00:15<00:15, 7.81s/it][12-26 05:28:27|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:202:<module>] Switched to low noise model.
Sampling: 75%|███████▌ | 3/4 [00:32<00:12, 12.17s/it]
Sampling: 100%|██████████| 4/4 [00:41<00:00, 10.71s/it]
Sampling: 100%|██████████| 4/4 [00:41<00:00, 10.29s/it]
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[12-26 05:28:59|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:231:<module>] +==========================++
[12-26 05:28:59|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:232:<module>] results/2_1234232_sagesla_turbodiffusion.mp4: time: 79.28578853607178
[12-26 05:28:59|INFO|turbodiffusion/inference/wan2.2_i2v_infer.py:233:<module>] +==========================++
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with turbodiffusion/inference/wan2.2_i2v_infer.py and reproduce the supplied 5090 command, checking the logged preprocessing, sampling, model-switch, and total timings. Done means the difference between the reported 38 seconds and the observed 79 seconds is explained and any required correction or clarification is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100