huggingface / huggingface/diffusers

Help us profile important pipelines and improve if needed

Open
#13,401 42 comments 2 reactions 0 assignees View on GitHub
diffusers-mvp performance
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

In https://github.com/huggingface/diffusers/pull/13356, we added a guide to comprehensively profile our pipelines with Claude. It, in turn, helped us get rid of issues that can get in the way of the benefits provided by `torch.compile`.

We cannot profile all our important pipelines alone, and this is where the community could really help us! In this thread, we want to gather interest from our community for such an initiative.

## How to take part?

* Take a pipeline of your choice (start with T2I or T2V first, as it's simpler than others). Then discuss with us here to gauge feasibility (not all pipelines are equally widely used). Tag `dg845` and `sayakpaul` while discussing.
* If there's mutual agreement, profile the pipeline following the steps outlined in the [guide](https://github.com/huggingface/diffusers/tree/main/examples/profiling).
* If there are fixes to be made, open a PR to ship the fixes. Also, include the results before and after those fixes.

## PRs merged with profiling-guided improvements

* QwenImage: https://github.com/huggingface/diffusers/pull/13406
* Z-Image: https://github.com/huggingface/diffusers/pull/13461
* LTX2: https://github.com/huggingface/diffusers/pull/13564

## PRs in progress

* None for now

## Pipelines worth profiling further

* QwenImage Edit
* SD3
* Flux.1-Kontext
* Ernie-Image
* Chroma
* JoyImage

Feel free to suggest yours, too!

Cc: @dg845

Contributor guide

Open the contributing guide

Research direction

Start by reading the guide in examples/profiling and review the listed candidate pipelines, beginning with a T2I or T2V pipeline. Discuss the selected pipeline with dg845 and sayakpaul before profiling it. Done means providing before-and-after profiling results and opening a PR for any fixes identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.