huggingface / huggingface/diffusers
process_encoder_hidden_states function is applied on each denoising step
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
https://github.com/huggingface/diffusers/blob/31058cdaef63ca660a1a045281d156239fba8192/src/diffusers/models/unets/unet_2d_condition.py#L1004
Hey! I noticed that `process_encoder_hidden_states` is applied on each denoising step that influences on an inference performance. This function can be executed only once before running UNet2DConditionModel.forward. The output of `process_encoder_hidden_states` should be combined with `encoder_hidden_states` and be provided to UNet2DConditionModel.forward
Beitragsleitfaden
Rechercherichtung
Start at src/diffusers/models/unets/unet_2d_condition.py around the referenced process_encoder_hidden_states call and inspect how its result enters UNet2DConditionModel.forward. Trace the denoising-step caller to determine where the processed encoder states can be prepared once; done means forward receives the combined states without repeating that processing on every step.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning, performance
- Issue-Typ
- Refactoring
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 35/100