huggingface / huggingface/diffusers

Fix skipped `test_model_xattn_padding` test by updating mask padding logic

Offen
#14,699 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug models needs-code-example needs-env-info
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

### **Title:**
Fix skipped `test_model_xattn_padding` test by updating mask padding logic

### **Description:**
Currently, there is a skipped test in `tests/models/unets/test_models_unet_2d_condition.py` regarding cross-attention mask padding (`test_model_xattn_padding`).

The test is currently skipped with the following reason:
> *"we currently pad mask by target_length tokens (what unclip needs), whereas stable-diffusion's cross-attn needs to instead pad by remaining_length."*

This is reflected by a dangling `TODO` in `src/diffusers/models/attention_processor.py` inside `prepare_attention_mask` (around line 740):

```python
# TODO: for pipelines such as stable-diffusion, padding cross-attn mask:
# we want to instead pad by (0, remaining_length), where remaining_length is:
# remaining_length: int = target_length - current_length
# TODO: re-enable tests/models/test_models_unet_2d_condition.py#test_model_xattn_padding
attention_mask = F.pad(attention_mask, (0, target_length), value=0.0)
```

### **Expected Behavior / Proposed Solution:**
1. Update `prepare_attention_mask` in `attention_processor.py` (and any other relevant `attention.py` files) to pad the mask by `remaining_length = target_length - current_length`.
2. Ensure the MPS `torch.zeros()` hack is also updated to pad by `remaining_length` rather than `target_length`.
3. Verify that these changes do not break existing `unclip` tests/pipelines.
4. Remove `@pytest.mark.skip` from `test_model_xattn_padding` in `tests/models/unets/test_models_unet_2d_condition.py` to re-enable it.

### **Relevant Files:**
- `src/diffusers/models/attention_processor.py`
- `tests/models/unets/test_models_unet_2d_condition.py`

@sayakpaul If this is indeed technical debt that needs fixing, I would be happy to put together a PR to update the logic and re-enable the test myself!

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Lies zuerst prepare_attention_mask in src/diffusers/models/attention_processor.py und den übersprungenen Test in tests/models/unets/test_models_unet_2d_condition.py; verfolge die Pfade für current_length, target_length und das MPS-Padding. Führe test_model_xattn_padding und die relevanten unclip-Tests oder -Pipelines aus; abgeschlossen ist die Aufgabe, wenn der Test wieder aktiviert ist und ohne Regressionen besteht.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, pytorch
Bereich
machine-learning, testing
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Aktiv
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
55/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.