huggingface / huggingface/diffusers

process_encoder_hidden_states function is applied on each denoising step

Open
#9,625 6 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

https://github.com/huggingface/diffusers/blob/31058cdaef63ca660a1a045281d156239fba8192/src/diffusers/models/unets/unet_2d_condition.py#L1004

Hey! I noticed that `process_encoder_hidden_states` is applied on each denoising step that influences on an inference performance. This function can be executed only once before running UNet2DConditionModel.forward. The output of `process_encoder_hidden_states` should be combined with `encoder_hidden_states` and be provided to UNet2DConditionModel.forward

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.