huggingface / huggingface/diffusers
[Feature Request] Add native ResilPhase cache support for FLUX and HunyuanVideo
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
## Motivation
[ResilPhase](https://github.com/zqc214/ResilPhase) is a training-free diffusion acceleration method that predicts the residual produced by the transformer block stack with barycentric Lagrange interpolation on a normalized phase axis. I would like to integrate it with the existing Diffusers cache Hook/`CacheMixin` APIs instead of replacing model `forward` methods.
## Proposed integration
- Add `ResilPhaseCacheConfig` and `apply_resilphase_cache`.
- Register a stateful head/middle/tail block Hook over the existing transformer block registry.
- Keep conditional and unconditional histories isolated through `cache_context`.
- Support balanced and Chebyshev phase mappings.
- Initially cover `FluxTransformer2DModel` and `HunyuanVideoTransformer3DModel`.
- Fall back to full block computation when FLUX ControlNet residuals are supplied, because those residuals are injected between blocks.
- Expose the method through `transformer.enable_cache(...)` / `disable_cache()`.
Proposed usage:
```python
from diffusers import FluxPipeline, ResilPhaseCacheConfig
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev")
pipe.transformer.enable_cache(
ResilPhaseCacheConfig(
cache_interval=6,
warmup_steps=3,
max_order=1,
mapping_method="balanced",
)
)
```
## Validation prepared
A prototype is available on [`zqc214:add-resilphase-cache`](https://github.com/zqc214/diffusers/tree/add-resilphase-cache). It includes:
- numerical tests for balanced and Chebyshev barycentric prediction;
- Hook scheduling, context isolation, refresh, and ControlNet fallback tests;
- model-level tests for FLUX and HunyuanVideo;
- small pipeline inference tests for both pipelines;
- API and optimization documentation;
- passing `make quality`.
Would this scope and Hook-based approach be acceptable for an upstream PR?
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reviewing the prototype on the zqc214:add-resilphase-cache branch and the existing Diffusers Hook/CacheMixin APIs. Check its numerical, scheduling, context-isolation, ControlNet fallback, model-level, and pipeline tests, then run make quality. Done means the proposed FLUX and HunyuanVideo integration and its API and optimization documentation meet upstream expectations.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning, performance
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Tranquilla
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100