huggingface / huggingface/diffusers
Support training where width does not equal height for Qwen-Image
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
To support training where width does not equal height for Qwen-Image, the following code:
```
img_shapes = [
(1, args.resolution // vae_scale_factor // 2, args.resolution // vae_scale_factor // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, args.resolution, args.resolution, vae_scale_factor
)
```
should be modified to:
```
img_shapes = [
(1, model_input.shape[3] // 2, model_input.shape[4] // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, model_input.shape[3] * vae_scale_factor, model_input.shape[4] * vae_scale_factor, vae_scale_factor
)
```
Contributor guide
Assessment
This issue has not been assessed yet.