modelscope / modelscope/DiffSynth-Studio
video background become dark using Wan2.1 , which is different from origin wan2.1 repo
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
I used DiffSynth-Studio to generate video with wan2.1 480p i2v model
I found the background of generated video became dark, but when I used origin wan repo the background seems good.
I ran in A100 with these settings same with settings used in origin wan repo
width = 480
height = 832
seed = 0
negative_prompt = '镜头晃动,色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走'
this is printed info when I ran origin wan2.1 :
[2025-04-29 03:15:25,275] INFO: offload_model is not specified, set to True.
[2025-04-29 03:15:25,275] INFO: Generation job args: Namespace(task='i2v-14B', size='832*480', frame_num=81, ckpt_dir='./Wan2.1-I2V-14B-480P/', offload_model=True, ulysses_size=1, ring_size=1, t5_fsdp=False, t5_cpu=False, dit_fsdp=False, save_file=None, prompt='A woman in a stylish beige dress stands confidently, adjusting her sunglasses with one hand while the other rests on her hip. She exudes confidence and elegance, her posture relaxed yet poised. The dress features a draped neckline and a fitted silhouette, accentuating her figure. The background is a plain, neutral color, ensuring all attention remains on her. The camera slowly pans to the right, capturing her from the side, highlighting the fluidity of her movements and the subtle details of her attire. The overall scene is captured in a smooth, steady shot, emphasizing her poised demeanor and the sophistication of her outfit.', use_prompt_extend=False, prompt_extend_method='local_qwen', prompt_extend_model=None, prompt_extend_target_lang='zh', base_seed=8157352883078725481, image='test.png', first_frame=None, last_frame=None, sample_solver='unipc', sample_steps=40, sample_shift=3.0, sample_guide_scale=5.0)
[2025-04-29 03:15:25,275] INFO: Generation model config: {'__name__': 'Config: Wan I2V 14B', 't5_model': 'umt5_xxl', 't5_dtype': torch.bfloat16, 'text_len': 512, 'param_dtype': torch.bfloat16, 'num_train_timesteps': 1000, 'sample_fps': 16, 'sample_neg_prompt': '镜头晃动,色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走', 't5_checkpoint': 'models_t5_umt5-xxl-enc-bf16.pth', 't5_tokenizer': 'google/umt5-xxl', 'clip_model': 'clip_xlm_roberta_vit_h_14', 'clip_dtype': torch.float16, 'clip_checkpoint': 'models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth', 'clip_tokenizer': 'xlm-roberta-large', 'vae_checkpoint': 'Wan2.1_VAE.pth', 'vae_stride': (4, 8, 8), 'patch_size': (1, 2, 2), 'dim': 5120, 'ffn_dim': 13824, 'freq_dim': 256, 'num_heads': 40, 'num_layers': 40, 'window_size': (-1, -1), 'qk_norm': True, 'cross_attn_norm': True, 'eps': 1e-06}
[2025-04-29 03:15:25,275] INFO: Input prompt: A woman with blonde hair stands against a plain white background, wearing a white sleeveless top and mustard-colored shorts. She has a relaxed posture, with her arms hanging loosely by her sides. The camera slowly zooms in to focus on her upper body and face, capturing her serene expression. The lighting is soft and even, highlighting her features without harsh shadows. The overall scene exudes a calm and peaceful atmosphere.
The image I used is :
generated videos :
origin repo:
https://github.com/user-attachments/assets/09d3afcd-000d-4a66-9e96-64bf38bc88cf
Diffsync-studio
https://github.com/user-attachments/assets/f2c9cc5f-76ea-44f4-b256-80fe9011fbbc
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Wan2.1 480p I2V generation in DiffSynth-Studio with the reported A100 settings, input image, prompt, and seed, then compare it with the origin Wan2.1 run. Trace the model configuration, sampling settings, and video decoding or VAE path for differences. Done means identifying and correcting the cause of the darker background, with output matching the expected brightness.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100