Comfy-Org / Comfy-Org/ComfyUI

Wan22FunControlToVideo error when ref_image provided. Fix provided in Other. RuntimeError: apply_rope freqs shape is not broadcastable to input

Open Beginner friendly
#16,181 0 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

Wan2.2 Funcontrol generate video with start frame (ref_image)

### Actual Behavior

RuntimeError: apply_rope freqs shape is not broadcastable to input

### Steps to Reproduce

run wan2.2 funcontrol with start frame

### Debug Logs

```powershell
[ERROR] !!! Exception during processing !!! apply_rope freqs shape is not broadcastable to input
[ERROR] Traceback (most recent call last):
File "G:\ComfyUI\execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "G:\ComfyUI\execution.py", line 306, in process_inputs
result = f(**inputs)
File "G:\ComfyUI\comfy_api\internal\__init__.py", line 149, in wrapped_func
return method(locked_class, **inputs)
File "G:\ComfyUI\comfy_api\latest\_io.py", line 2046, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
File "G:\ComfyUI\custom_nodes\ComfyUI-gibby-nodes\ksampler_context\__init__.py", line 790, in execute
samples = comfy.sample.sample_custom(
model_obj, noise, cfg_value, sampler_obj, sigmas_tensor,
...<2 lines>...
disable_pbar=disable_pbar, seed=seed
)
File "G:\ComfyUI\comfy\sample.py", line 87, in sample_custom
samples = comfy.samplers.sample(model, noise, positive, negative, cfg, model.load_device, sampler, sigmas, model_options=model.model_options, latent_image=latent_image, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "G:\ComfyUI\comfy\samplers.py", line 1353, in sample
return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 1335, in sample
output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, latent_shapes=latent_shapes)
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 1262, in outer_sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, latent_shapes=latent_shapes)
File "G:\ComfyUI\comfy\samplers.py", line 1237, in inner_sample
samples = executor.execute(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 1005, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "G:\ComfyUI\venv\Lib\site-packages\torch\utils\_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "G:\ComfyUI\comfy\k_diffusion\sampling.py", line 1619, in sample_er_sde
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "G:\ComfyUI\comfy\samplers.py", line 640, in __call__
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
File "G:\ComfyUI\comfy\samplers.py", line 1208, in __call__
return self.outer_predict_noise(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 1215, in outer_predict_noise
).execute(x, timestep, model_options, seed)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 1218, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
File "G:\ComfyUI\comfy\samplers.py", line 620, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
File "G:\ComfyUI\comfy\samplers.py", line 211, in calc_cond_batch
return _calc_cond_batch_outer(model, conds, x_in, timestep, model_options)
File "G:\ComfyUI\comfy\samplers.py", line 219, in _calc_cond_batch_outer
return executor.execute(model, conds, x_in, timestep, model_options)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\samplers.py", line 335, in _calc_cond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\model_base.py", line 208, in apply_model
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...<2 lines>...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.APPLY_MODEL, transformer_options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
).execute(x, t, c_concat, c_crossattn, control, transformer_options, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\model_base.py", line 252, in _apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds)
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 680, in forward
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...<2 lines>...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.DIFFUSION_MODEL, transformer_options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
).execute(x, timestep, context, clip_fea, time_dim_concat, transformer_options, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 709, in _forward
return self.forward_orig(x, timestep, context, clip_fea=clip_fea, freqs=freqs, transformer_options=transformer_options, **kwargs)[:, :, :t, :h, :w]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 618, in forward_orig
x = block(x, e=e0, freqs=freqs, context=context, context_img_len=context_img_len, transformer_options=transformer_options)
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 253, in forward
y = self.self_attn(
torch.addcmul(repeat_e(e[0], x), self.norm1(x), 1 + repeat_e(e[1], x)),
freqs, transformer_options=transformer_options)
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 81, in forward
q = qkv_fn_q(x)
File "G:\ComfyUI\comfy\ldm\wan\model.py", line 72, in qkv_fn_q
return apply_rope1(q, freqs)
File "G:\ComfyUI\comfy\ldm\flux\math.py", line 58, in apply_rope1
return comfy.quant_ops.ck.apply_rope1(x, freqs_cis)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\comfy_kitchen\__init__.py", line 794, in apply_rope1
return torch.ops.comfy_kitchen.apply_rope1(x, freqs_cis)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\torch\_ops.py", line 1279, in __call__
return self._op(*args, **kwargs)
~~~~~~~~^^^^^^^^^^^^^^^^^
File "G:\ComfyUI\venv\Lib\site-packages\torch\_library\custom_ops.py", line 441, in backend_impl
result = self._backend_fns[device_type](*args, **kwargs)
File "G:\ComfyUI\venv\Lib\site-packages\torch\_compile.py", line 54, in inner
return disable_fn(*args, **kwargs)
File "G:\ComfyUI\venv\Lib\site-packages\torch\_dynamo\eval_frame.py", line 1446, in _fn
return fn(*args, **kwargs)
File "G:\ComfyUI\venv\Lib\site-packages\torch\_library\custom_ops.py", line 502, in wrapped_fn
return fn(*args, **kwargs)
File "G:\ComfyUI\venv\Lib\site-packages\comfy_kitchen\backends\eager\rope.py", line 441, in _op_apply_rope1
return impl(**kwargs)
File "G:\ComfyUI\venv\Lib\site-packages\comfy_kitchen\backends\cuda\__init__.py", line 2776, in apply_rope1
return _apply_rope1_cuda(x, freqs_cis, split_half=False, inplace=False)
File "G:\ComfyUI\venv\Lib\site-packages\comfy_kitchen\backends\cuda\__init__.py", line 2736, in _apply_rope1_cuda
_C.apply_rope(
~~~~~~~~~~~~~^
_wrap_for_dlpack(x),
^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
split_half,
^^^^^^^^^^^
)
^
RuntimeError: apply_rope freqs shape is not broadcastable to input
```

### Other

Fix - produced by qwen3.8 27b, tested, working.

One line in comfy/ldm/wan/model.py (base WanModel._forward, right where the freqs assumption is made) — pad the reference latent the same way the target is padded:

python

if self.ref_conv is not None and "reference_latent" in kwargs:
t_len += 1
kwargs["reference_latent"] = comfy.ldm.common_dit.pad_to_patch_size(kwargs["reference_latent"], self.patch_size[1:])

This mirrors what the SCAIL Wan variant already does for its reference. Even-dim inputs get zero padding, so existing workflows are byte-for-byte unaffected; odd-dim refs now run instead of crashing. It also covers AnimateWanModel, which shares the base _forward.

Contributor guide

Open the contributing guide

Research direction

Reproduce Wan2.2 Funcontrol generation with a start frame, then read comfy/ldm/wan/model.py at WanModel._forward where the reference latent and freqs assumptions are handled. Verify the reported change against that path and confirm that generation with ref_image completes without the apply_rope shape error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.