huggingface / huggingface/diffusers

[Feature Request] Add native ResilPhase cache support for FLUX and HunyuanVideo

Open
#14,378 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

## Motivation

[ResilPhase](https://github.com/zqc214/ResilPhase) is a training-free diffusion acceleration method that predicts the residual produced by the transformer block stack with barycentric Lagrange interpolation on a normalized phase axis. I would like to integrate it with the existing Diffusers cache Hook/`CacheMixin` APIs instead of replacing model `forward` methods.

## Proposed integration

- Add `ResilPhaseCacheConfig` and `apply_resilphase_cache`.
- Register a stateful head/middle/tail block Hook over the existing transformer block registry.
- Keep conditional and unconditional histories isolated through `cache_context`.
- Support balanced and Chebyshev phase mappings.
- Initially cover `FluxTransformer2DModel` and `HunyuanVideoTransformer3DModel`.
- Fall back to full block computation when FLUX ControlNet residuals are supplied, because those residuals are injected between blocks.
- Expose the method through `transformer.enable_cache(...)` / `disable_cache()`.

Proposed usage:

```python
from diffusers import FluxPipeline, ResilPhaseCacheConfig

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev")
pipe.transformer.enable_cache(
ResilPhaseCacheConfig(
cache_interval=6,
warmup_steps=3,
max_order=1,
mapping_method="balanced",
)
)
```

## Validation prepared

A prototype is available on [`zqc214:add-resilphase-cache`](https://github.com/zqc214/diffusers/tree/add-resilphase-cache). It includes:

- numerical tests for balanced and Chebyshev barycentric prediction;
- Hook scheduling, context isolation, refresh, and ControlNet fallback tests;
- model-level tests for FLUX and HunyuanVideo;
- small pipeline inference tests for both pipelines;
- API and optimization documentation;
- passing `make quality`.

Would this scope and Hook-based approach be acceptable for an upstream PR?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the prototype on the zqc214:add-resilphase-cache branch and the existing Diffusers Hook/CacheMixin APIs. Check its numerical, scheduling, context-isolation, ControlNet fallback, model-level, and pipeline tests, then run make quality. Done means the proposed FLUX and HunyuanVideo integration and its API and optimization documentation meet upstream expectations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.