huggingface / huggingface/diffusers

Make the sampling loop of pipelines modular, with designated inputs that can be handled iteratively with functions and a default sampler step function

Aperta
#7,808 3 commenti 2 reazioni 0 assegnatari Vedi su GitHub
stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Model/Pipeline/Scheduler description

Related to #7761 .

This is an effective replacement for the existing sampling loop function and the many, many kwargs that were made to allow the user to control it or inject callbacks into it.

```py
class SamplingInput:
def __init__(self, img, text_embedding, unet, timestep=None, **kwargs):
self.img = img
self.text_embedding = text_embedding
self.unet = unet
self.timestep = timestep

# ... lots of other code ...

inp = SamplingInput(img, text_embedding, unet)
with self.progress_bar(total=num_inference_steps) as progress_bar:
for i, t in enumerate(timesteps):
inp.timestep = t
for sampling_function in self.sampling_functions:
inp = sampling_function(inp)

output_img = inp.img
```

This will give the end user complete control of the sampling loop, allow the repo to add "official inline sampling functions" like report an image to an endpoint so that the user can view intermediate steps, etc.

We can add an argument sampling_functions: `list[Callable]=[default_sampling_function]` into the `__call__` as a new, backwards compatible kwarg.

This requires a rewrite of all of the pipelines, but as it is a backwards compatible change it can be introduced to any of the more popular pipelines first.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Review the existing sampling loop in the pipelines and their __call__ entry points, then compare how sampling-related kwargs and callbacks are handled. Define the SamplingInput, sampling_functions, and default_sampling_function interfaces against the proposed example. Done means the modular loop is introduced compatibly and the affected popular pipelines are updated without changing existing behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
machine-learning
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.