huggingface / huggingface/diffusers
[Feature Request] Add native ResilPhase cache support for FLUX and HunyuanVideo
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
## Motivation
[ResilPhase](https://github.com/zqc214/ResilPhase) is a training-free diffusion acceleration method that predicts the residual produced by the transformer block stack with barycentric Lagrange interpolation on a normalized phase axis. I would like to integrate it with the existing Diffusers cache Hook/`CacheMixin` APIs instead of replacing model `forward` methods.
## Proposed integration
- Add `ResilPhaseCacheConfig` and `apply_resilphase_cache`.
- Register a stateful head/middle/tail block Hook over the existing transformer block registry.
- Keep conditional and unconditional histories isolated through `cache_context`.
- Support balanced and Chebyshev phase mappings.
- Initially cover `FluxTransformer2DModel` and `HunyuanVideoTransformer3DModel`.
- Fall back to full block computation when FLUX ControlNet residuals are supplied, because those residuals are injected between blocks.
- Expose the method through `transformer.enable_cache(...)` / `disable_cache()`.
Proposed usage:
```python
from diffusers import FluxPipeline, ResilPhaseCacheConfig
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev")
pipe.transformer.enable_cache(
ResilPhaseCacheConfig(
cache_interval=6,
warmup_steps=3,
max_order=1,
mapping_method="balanced",
)
)
```
## Validation prepared
A prototype is available on [`zqc214:add-resilphase-cache`](https://github.com/zqc214/diffusers/tree/add-resilphase-cache). It includes:
- numerical tests for balanced and Chebyshev barycentric prediction;
- Hook scheduling, context isolation, refresh, and ControlNet fallback tests;
- model-level tests for FLUX and HunyuanVideo;
- small pipeline inference tests for both pipelines;
- API and optimization documentation;
- passing `make quality`.
Would this scope and Hook-based approach be acceptable for an upstream PR?
Contributor guide
Research direction
Start by reviewing the prototype on the zqc214:add-resilphase-cache branch and the existing Diffusers Hook/CacheMixin APIs. Check its numerical, scheduling, context-isolation, ControlNet fallback, model-level, and pipeline tests, then run make quality. Done means the proposed FLUX and HunyuanVideo integration and its API and optimization documentation meet upstream expectations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100