huggingface / huggingface/diffusers
Support training where width does not equal height for Qwen-Image
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
To support training where width does not equal height for Qwen-Image, the following code:
```
img_shapes = [
(1, args.resolution // vae_scale_factor // 2, args.resolution // vae_scale_factor // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, args.resolution, args.resolution, vae_scale_factor
)
```
should be modified to:
```
img_shapes = [
(1, model_input.shape[3] // 2, model_input.shape[4] // 2)
] * bsz
noisy_model_input = noisy_model_input.permute(0, 2, 1, 3, 4)
packed_noisy_model_input = QwenImagePipeline._pack_latents(
noisy_model_input,
batch_size=model_input.shape[0],
num_channels_latents=model_input.shape[1],
height=model_input.shape[3],
width=model_input.shape[4],
)
model_pred = transformer(
hidden_states=packed_noisy_model_input,
encoder_hidden_states=prompt_embeds,
encoder_hidden_states_mask=prompt_embeds_mask,
timestep=timesteps / 1000,
img_shapes=img_shapes,
txt_seq_lens=prompt_embeds_mask.sum(dim=1).tolist(),
return_dict=False,
)[0]
model_pred = QwenImagePipeline._unpack_latents(
model_pred, model_input.shape[3] * vae_scale_factor, model_input.shape[4] * vae_scale_factor, vae_scale_factor
)
```
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Locate the training code containing QwenImagePipeline._pack_latents and compare its current rectangular-shape handling with the proposed changes. Verify that width and height are taken from model_input throughout packing and unpacking, and that training with unequal dimensions completes successfully.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 42/100