huggingface / huggingface/diffusers

Support training where width does not equal height for Qwen-Image

Offen
#12,102 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
stale
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

To support training where width does not equal height for Qwen-Image, the following code:

```
img_shapes = [
(1, args.resolution // vae_scale_factor // 2, args.resolution // vae_scale_factor // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, args.resolution, args.resolution, vae_scale_factor
)
```
should be modified to:
```
img_shapes = [
(1, model_input.shape[3] // 2, model_input.shape[4] // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, model_input.shape[3] * vae_scale_factor, model_input.shape[4] * vae_scale_factor, vae_scale_factor
)
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Locate the training code containing QwenImagePipeline._pack_latents and compare its current rectangular-shape handling with the proposed changes. Verify that width and height are taken from model_input throughout packing and unpacking, and that training with unequal dimensions completes successfully.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, pytorch
Bereich
machine-learning
Issue-Typ
Feature
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
42/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.