huggingface / huggingface/diffusers

PR GPU CI skips new-model GPU tests: Cosmos3 memory tests failed on an unrelated PR

Aperta
#14,247 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug models needs-code-example
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

## Summary

GPU-only model tests added in a PR can silently never run on that PR, then surface as failures on an **unrelated** PR later. This just happened with the Cosmos3 transformer: two `TestCosmos3OmniTransformerMemory` tests failed in the ace-step-lora PR's CI ([run](https://github.com/huggingface/diffusers/actions/runs/29635377742/job/88057065236)), even though ace-step-lora has nothing to do with Cosmos3.

This is relevant to the ongoing work on **selecting which tests to run for a given PR** — the fix here is really "make the PR GPU workflow run the tests that the PR's changes actually affect."

## What happened

| PR | Added | Ran its Cosmos3 GPU tests? |
|---|---|---|
| #13818 "Adding Cosmos 3" | `transformer_cosmos3.py` + AE + pipeline + docs + examples, but **zero `tests/` files** | Workflow triggered (via `examples/**`), but no transformer tests existed yet |
| #14181 "Cosmos3 edge support" | **First** added `tests/models/transformers/test_models_transformer_cosmos3.py` incl. `TestCosmos3OmniTransformerMemory` | **No** — GPU workflow never triggered (see below) |
| #14193 "ace-step lora" | Touched a loader file | GPU workflow triggered → ran the full model suite → Cosmos3 memory tests ran for the first time → **failed**, blocking an unrelated PR |

## Root cause — two compounding gaps

**1. `pr_tests_gpu.yml`'s `paths:` filter is out of sync with `push_tests.yml`'s.**

- `push_tests.yml` (Fast GPU Tests on **main**) gates on `src/diffusers/**.py`, `examples/**.py`, **`tests/**.py`**.
- `pr_tests_gpu.yml` (Fast GPU Tests on **PR**) gates only on a narrow list of core loader/modeling files + `examples/**/*.py`. It **excludes** `src/diffusers/models/**` and `tests/**`.

So a PR that adds a new model + its tests (touching only `src/diffusers/models/**` and `tests/models/**`, as #14181 did) never triggers GPU CI on the PR. The tests first execute post-merge on `main`, or accidentally when some unrelated PR trips the narrow filter.

Net: GPU-only tests were introduced in a PR that structurally couldn't run them.

## Suggested fixes (Claude)

1. **Align the PR trigger with the push trigger** — add `src/diffusers/models/**.py`, `src/diffusers/pipelines/**.py`, `tests/models/**.py`, `tests/pipelines/**.py` to `pr_tests_gpu.yml`'s `paths:`.
2. **Scope the pytest run to changed files** so broadening the trigger doesn't run the entire `tests/models` suite (~17 min) on every model PR. Derive changed test files from `git diff origin/main...HEAD` and pass those (still combined with the mixin `-k` pattern). *This overlaps directly with the "fetch tests to run per PR" work.*
3. **Stopgap:** until the above lands, run `/diffusers-bot pytest ` (via `pr_comment_gpu_tests.yml`) on any new-model / new-transformer PR, and make it a review-checklist item.

@DN6 mentioning you since 2. above is probably also best implemented after your take on improving the test fetcher?

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with .github/workflows/pr_tests_gpu.yml and compare its paths filter with push_tests.yml. Review how changed files could be derived from git diff origin/main...HEAD and how the existing pytest mixin -k pattern is used. Done means new model and test changes trigger PR GPU CI without unnecessarily running the full tests/models suite, with the Cosmos3 case covered.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
github-actions, python, pytorch
Ambito
ci-cd, testing-qa
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
48/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.