huggingface / huggingface/diffusers

Hardware auto-optimizing

Offen
#4,053 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
wip
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

**Is your feature request related to a problem? Please describe.**
The current Diffusers library does not provide an out-of-the-box functionality to _automatically_ apply system and model-level optimizations.

I often find myself manually tweaking system-level optimizations in PyTorch as well as model-specific ones such as VAE tiling and CPU offloading, which can be a tedious and error-prone task.

Over time, a lot of interface names or methods for optimization have come and gone, or no longer apply in certain situations. This leads to inconsistencies in the documentation, and published examples in tutorials. A single auto-optimize function could be applied in these cases, and the behaviour of that would transparently evolve over time.

**Describe the solution you'd like**
I would like the Diffusers library to include a new module or helper function that can intelligently select and apply the most effective optimizations to both the system itself (using the relevant torch.* methods) and to a given pipeline.

For instance, a helper function could take in an instance of a pipeline and kargs containing a set of parameters, and then apply the selected optimizations. The parameters could include settings for things like enabling or disabling VAE tiling, whole model or sequential CPU offloading, gradients, torch_dtype, and others. This would make the optimization process more streamlined and user-friendly.

**Describe alternatives you've considered**
One alternative is to manually apply these optimizations, as is currently done, but this can be complex and time-consuming, and it also risks inconsistent application of the optimizations across different parts of the code.

Another alternative could be the development of individual helper functions for each type of optimization, but this could lead to a bloated and less cohesive codebase.

**Additional context**
This feature would be beneficial to both newcomers to the library, who may not be fully versed in the intricacies of these optimizations, and to experienced users, who would appreciate the time saved and the consistency ensured by this feature.

It would contribute to making the Diffusers library more accessible and efficient to use, and could potentially improve the performance of the models built with it.

Any new optimizations that make obvious sense can be enabled by default in this tool, which would seamlessly improve the experience for users, as they update.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

The issue names no files, tests, or entry points. Start by mapping the existing PyTorch and pipeline optimization interfaces, including VAE tiling and CPU offloading, and determine the intended scope and defaults. Done would require an agreed design, implementation, and tests covering the selected optimizations.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, pytorch
Bereich
machine-learning, performance
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.