modelscope / modelscope/DiffSynth-Studio

video background become dark using Wan2.1 , which is different from origin wan2.1 repo

Open
#560 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

I used DiffSynth-Studio to generate video with wan2.1 480p i2v model

I found the background of generated video became dark, but when I used origin wan repo the background seems good.

I ran in A100 with these settings same with settings used in origin wan repo

width = 480
height = 832
seed = 0
negative_prompt = '镜头晃动,色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走'

this is printed info when I ran origin wan2.1 :

[2025-04-29 03:15:25,275] INFO: offload_model is not specified, set to True.
[2025-04-29 03:15:25,275] INFO: Generation job args: Namespace(task='i2v-14B', size='832*480', frame_num=81, ckpt_dir='./Wan2.1-I2V-14B-480P/', offload_model=True, ulysses_size=1, ring_size=1, t5_fsdp=False, t5_cpu=False, dit_fsdp=False, save_file=None, prompt='A woman in a stylish beige dress stands confidently, adjusting her sunglasses with one hand while the other rests on her hip. She exudes confidence and elegance, her posture relaxed yet poised. The dress features a draped neckline and a fitted silhouette, accentuating her figure. The background is a plain, neutral color, ensuring all attention remains on her. The camera slowly pans to the right, capturing her from the side, highlighting the fluidity of her movements and the subtle details of her attire. The overall scene is captured in a smooth, steady shot, emphasizing her poised demeanor and the sophistication of her outfit.', use_prompt_extend=False, prompt_extend_method='local_qwen', prompt_extend_model=None, prompt_extend_target_lang='zh', base_seed=8157352883078725481, image='test.png', first_frame=None, last_frame=None, sample_solver='unipc', sample_steps=40, sample_shift=3.0, sample_guide_scale=5.0)
[2025-04-29 03:15:25,275] INFO: Generation model config: {'__name__': 'Config: Wan I2V 14B', 't5_model': 'umt5_xxl', 't5_dtype': torch.bfloat16, 'text_len': 512, 'param_dtype': torch.bfloat16, 'num_train_timesteps': 1000, 'sample_fps': 16, 'sample_neg_prompt': '镜头晃动,色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走', 't5_checkpoint': 'models_t5_umt5-xxl-enc-bf16.pth', 't5_tokenizer': 'google/umt5-xxl', 'clip_model': 'clip_xlm_roberta_vit_h_14', 'clip_dtype': torch.float16, 'clip_checkpoint': 'models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth', 'clip_tokenizer': 'xlm-roberta-large', 'vae_checkpoint': 'Wan2.1_VAE.pth', 'vae_stride': (4, 8, 8), 'patch_size': (1, 2, 2), 'dim': 5120, 'ffn_dim': 13824, 'freq_dim': 256, 'num_heads': 40, 'num_layers': 40, 'window_size': (-1, -1), 'qk_norm': True, 'cross_attn_norm': True, 'eps': 1e-06}
[2025-04-29 03:15:25,275] INFO: Input prompt: A woman with blonde hair stands against a plain white background, wearing a white sleeveless top and mustard-colored shorts. She has a relaxed posture, with her arms hanging loosely by her sides. The camera slowly zooms in to focus on her upper body and face, capturing her serene expression. The lighting is soft and even, highlighting her features without harsh shadows. The overall scene exudes a calm and peaceful atmosphere.

The image I used is :

Image

generated videos :

origin repo:
https://github.com/user-attachments/assets/09d3afcd-000d-4a66-9e96-64bf38bc88cf
Diffsync-studio
https://github.com/user-attachments/assets/f2c9cc5f-76ea-44f4-b256-80fe9011fbbc

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Wan2.1 480p I2V generation in DiffSynth-Studio with the reported A100 settings, input image, prompt, and seed, then compare it with the origin Wan2.1 run. Trace the model configuration, sampling settings, and video decoding or VAE path for differences. Done means identifying and correcting the cause of the darker background, with output matching the expected brightness.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.