qwen_image_controlnet_inpaint 使用cache加速后controlnet失效
@DefTruth is already working on this.
Since Nov 26, 2025.
Assessment
This issue has not been assessed yet.
Description
root@autodl-container-7de840bdc0-068c6662:~/autodl-tmp/cache-dit-main/cache-dit-main/examples/pipeline# python run_qwen_image_controlnet_inpaint.py --cache
Skipping import of cpp extensions due to incompatible torch version 2.8.0+cu128 for torchao version 0.14.1 Please see pytorch/ao#2919 for more info
Namespace(cache=True, compile=False, fuse_lora=False, steps=None, Fn=8, Bn=0, rdt=0.08, max_warmup_steps=8, warmup_interval=1, max_cached_steps=-1, max_continuous_cached_steps=-1, taylorseer=False, taylorseer_order=1, height=None, width=None, quantize=False, quantize_type='float8_weight_only', parallel_type=None, attn=None, perf=False)
The config attributes {'guidance_embeds': False, 'pooled_projection_dim': 768} were passed to QwenImageControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████| 9/9 [00:12<00:00, 1.36s/it]
Loading pipeline components...: 40%|███████████████████████████████▏ | 2/5 [00:13<00:16, 5.61s/it]torch_dtype is deprecated! Use dtype instead!
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████| 4/4 [00:03<00:00, 1.01it/s]
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████████████████| 5/5 [00:17<00:00, 3.54s/it]
INFO 11-20 09:42:52 [cache_adapter.py:49] QwenImageControlNetInpaintPipeline is officially supported by cache-dit. Use it's pre-defined BlockAdapter directly!
INFO 11-20 09:42:52 [block_adapters.py:220] Auto fill blocks_name: transformer_blocks.
WARNING 11-20 09:42:52 [functor_qwen_image_controlnet.py:47] Patched QwenImageTransformer2DModel for cache-dit.
INFO 11-20 09:42:52 [functor_qwen_image_controlnet.py:57] Applied QwenImageControlNetPatchFunctor for QwenImageTransformer2DModel, Patch: True.
WARNING 11-20 09:42:52 [block_adapters.py:159] _hf_hook is not None, force skip pattern check!
WARNING 11-20 09:42:52 [block_adapters.py:469] Skipped Forward Pattern Check: ForwardPattern.Pattern_1
INFO 11-20 09:42:52 [cache_adapter.py:148] Use custom 'enable_separate_cfg' from cache context kwargs: True. Pipeline: QwenImageControlNetInpaintPipeline.
INFO 11-20 09:42:52 [cache_adapter.py:307] Collected Context Config: DBCache_F8B0_W8I1M0MC0_R0.08, Calibrator Config: None
WARNING 11-20 09:42:52 [pattern_base.py:78] Skipped Forward Pattern Check: ForwardPattern.Pattern_1
INFO 11-20 09:42:52 [pattern_base.py:70] Match Blocks: CachedBlocks_Pattern_0_1_2, for transformer_blocks, cache_context: transformer_blocks_140475306594464, context_manager: QwenImageControlNetInpaintPipeline_140475775400944.
WARNING 11-20 09:42:52 [cache_adapter.py:372] _hf_hook is not None, so, we have to re-direct transformer's original_forward(140475431963200) to transformer's _old_forward(140475434539904)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [04:05<00:00, 4.91s/it]
🤗Context Options: QwenImageTransformerBlock
{'cache_config': DBCacheConfig(cache_type=<CacheType.DBCache: 'DBCache'>, Fn_compute_blocks=8, Bn_compute_blocks=0, residual_diff_threshold=0.08, max_accumulated_residual_diff_threshold=None, max_warmup_steps=8, warmup_interval=1, max_cached_steps=-1, max_continuous_cached_steps=-1, enable_separate_cfg=True, cfg_compute_first=False, cfg_diff_compute_separate=True, num_inference_steps=None, steps_computation_mask=None, steps_computation_policy='dynamic'), 'name': 'transformer_blocks_140475306594464'}
WARNING 11-20 09:47:00 [summary.py:284] Can't find Parallelism Config for: QwenImageTransformerBlock
⚡️Cache Steps and Residual Diffs Statistics: QwenImageTransformerBlock
Cache Steps | Diffs P00 | Diffs P25 | Diffs P50 | Diffs P75 | Diffs P95 | Diffs Min | Diffs Max -- | -- | -- | -- | -- | -- | -- | -- 15 | 0.043 | 0.054 | 0.088 | 0.117 | 0.155 | 0.043 | 0.357Time cost: 248.46s
Saving image to qwen-image-controlnet-inpaint.C0_Q0_DBCache_F8B0_W8I1M0MC0_R0.08_T0O0_S15.png
能不能再看一下run_qwen_image_controlnet_inpaint.py, 我试了一下, 就是这个会导致controlnet会失效, 生成的图和原图没关系, 用的一样的案例, 生图效果如下
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 88
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 14
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from vipshop/cache-dit
-
bug documentation
-
enhancement performance
-
enhancement performance
All issues in vipshop/cache-dit
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100