qwen_image_controlnet_inpaint 使用cache加速后controlnet失效

Open
#482 4 comments 0 reactions 1 assignee View on GitHub

@DefTruth is already working on this.

Since Nov 26, 2025.

Assessment

This issue has not been assessed yet.

Description

root@autodl-container-7de840bdc0-068c6662:~/autodl-tmp/cache-dit-main/cache-dit-main/examples/pipeline# python run_qwen_image_controlnet_inpaint.py --cache
Skipping import of cpp extensions due to incompatible torch version 2.8.0+cu128 for torchao version 0.14.1 Please see pytorch/ao#2919 for more info
Namespace(cache=True, compile=False, fuse_lora=False, steps=None, Fn=8, Bn=0, rdt=0.08, max_warmup_steps=8, warmup_interval=1, max_cached_steps=-1, max_continuous_cached_steps=-1, taylorseer=False, taylorseer_order=1, height=None, width=None, quantize=False, quantize_type='float8_weight_only', parallel_type=None, attn=None, perf=False)
The config attributes {'guidance_embeds': False, 'pooled_projection_dim': 768} were passed to QwenImageControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████| 9/9 [00:12<00:00, 1.36s/it]
Loading pipeline components...: 40%|███████████████████████████████▏ | 2/5 [00:13<00:16, 5.61s/it]torch_dtype is deprecated! Use dtype instead!
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████| 4/4 [00:03<00:00, 1.01it/s]
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████████████████| 5/5 [00:17<00:00, 3.54s/it]
INFO 11-20 09:42:52 [cache_adapter.py:49] QwenImageControlNetInpaintPipeline is officially supported by cache-dit. Use it's pre-defined BlockAdapter directly!
INFO 11-20 09:42:52 [block_adapters.py:220] Auto fill blocks_name: transformer_blocks.
WARNING 11-20 09:42:52 [functor_qwen_image_controlnet.py:47] Patched QwenImageTransformer2DModel for cache-dit.
INFO 11-20 09:42:52 [functor_qwen_image_controlnet.py:57] Applied QwenImageControlNetPatchFunctor for QwenImageTransformer2DModel, Patch: True.
WARNING 11-20 09:42:52 [block_adapters.py:159] _hf_hook is not None, force skip pattern check!
WARNING 11-20 09:42:52 [block_adapters.py:469] Skipped Forward Pattern Check: ForwardPattern.Pattern_1
INFO 11-20 09:42:52 [cache_adapter.py:148] Use custom 'enable_separate_cfg' from cache context kwargs: True. Pipeline: QwenImageControlNetInpaintPipeline.
INFO 11-20 09:42:52 [cache_adapter.py:307] Collected Context Config: DBCache_F8B0_W8I1M0MC0_R0.08, Calibrator Config: None
WARNING 11-20 09:42:52 [pattern_base.py:78] Skipped Forward Pattern Check: ForwardPattern.Pattern_1
INFO 11-20 09:42:52 [pattern_base.py:70] Match Blocks: CachedBlocks_Pattern_0_1_2, for transformer_blocks, cache_context: transformer_blocks_140475306594464, context_manager: QwenImageControlNetInpaintPipeline_140475775400944.
WARNING 11-20 09:42:52 [cache_adapter.py:372] _hf_hook is not None, so, we have to re-direct transformer's original_forward(140475431963200) to transformer's _old_forward(140475434539904)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [04:05<00:00, 4.91s/it]

🤗Context Options: QwenImageTransformerBlock

{'cache_config': DBCacheConfig(cache_type=<CacheType.DBCache: 'DBCache'>, Fn_compute_blocks=8, Bn_compute_blocks=0, residual_diff_threshold=0.08, max_accumulated_residual_diff_threshold=None, max_warmup_steps=8, warmup_interval=1, max_cached_steps=-1, max_continuous_cached_steps=-1, enable_separate_cfg=True, cfg_compute_first=False, cfg_diff_compute_separate=True, num_inference_steps=None, steps_computation_mask=None, steps_computation_policy='dynamic'), 'name': 'transformer_blocks_140475306594464'}
WARNING 11-20 09:47:00 [summary.py:284] Can't find Parallelism Config for: QwenImageTransformerBlock

⚡️Cache Steps and Residual Diffs Statistics: QwenImageTransformerBlock

Cache Steps | Diffs P00 | Diffs P25 | Diffs P50 | Diffs P75 | Diffs P95 | Diffs Min | Diffs Max -- | -- | -- | -- | -- | -- | -- | -- 15 | 0.043 | 0.054 | 0.088 | 0.117 | 0.155 | 0.043 | 0.357

Time cost: 248.46s
Saving image to qwen-image-controlnet-inpaint.C0_Q0_DBCache_F8B0_W8I1M0MC0_R0.08_T0O0_S15.png

能不能再看一下run_qwen_image_controlnet_inpaint.py, 我试了一下, 就是这个会导致controlnet会失效, 生成的图和原图没关系, 用的一样的案例, 生图效果如下

Image

Dominant language
Python
Stars
1.3k
Forks
88
Avg merge
1d 4h
Merged PRs (30d)
14

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vipshop/cache-dit

All issues in vipshop/cache-dit

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.