intel / intel/llm-scaler

SeedVR2 torch.compile() can't run with llm-scaler-omni:0.1.0-b6 on B70

Open
#333 1 comment 0 reactions 1 assignee Claimed by @xiangyuT View on GitHub
omni
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

Enable "SeedVR2 Torch Compile Settings" node, will report below error.

```
[05:45:15.150] ❌ [ERROR] Error in Phase 1 (Encoding): Data-dependent branching
Explanation: Detected data-dependent branching (e.g. `if my_tensor.sum() > 0:`). Dynamo does not support tracic control flow.
Hint: This graph break is fundamental - it is unlikely that Dynamo will ever be able to trace through your coder finding a workaround.
Hint: Use `torch.cond` to express dynamic control flow.

Developer debug context: attempted to jump with TensorVariable()

For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/.html

from user code:
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae. 815, in forward
sample = self.conv_in(sample, memory_state=memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 227, in forward
return self.slicing_forward(input, memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 296, in slicing_forward
input[i] = self.memory_limit_conv(
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 135, in memory_limit_conv
if memory_occupy < self.memory_limit or split_dim == x.ndim:

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PFor even more developer context, set TORCH_LOGS="+dynamo"

!!! Exception during processing !!! Data-dependent branching
Explanation: Detected data-dependent branching (e.g. `if my_tensor.sum() > 0:`). Dynamo does not support tracic control flow.
Hint: This graph break is fundamental - it is unlikely that Dynamo will ever be able to trace through your coder finding a workaround.
Hint: Use `torch.cond` to express dynamic control flow.

Developer debug context: attempted to jump with TensorVariable()

For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/.html

from user code:
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae. 815, in forward
sample = self.conv_in(sample, memory_state=memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 227, in forward
return self.slicing_forward(input, memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 296, in slicing_forward
input[i] = self.memory_limit_conv(
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 135, in memory_limit_conv
if memory_occupy < self.memory_limit or split_dim == x.ndim:

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PFor even more developer context, set TORCH_LOGS="+dynamo"

Traceback (most recent call last):
File "/llm/ComfyUI/execution.py", line 524, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, i_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/execution.py", line 333, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/execution.py", line 307, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/llm/ComfyUI/execution.py", line 295, in process_inputs
result = f(**inputs)
^^^^^^^^^^^
File "/llm/ComfyUI/comfy_api/internal/__init__.py", line 149, in wrapped_func
return method(locked_class, **inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/comfy_api/latest/_io.py", line 1748, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/interfaces/video_upscaler.py", line 589, in
raise e
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/interfaces/video_upscaler.py", line 471, in
ctx = encode_all_batches(
^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/core/generation_phases.py", line 489, in encatches
cond_latents = runner.vae_encode([transformed_video])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/core/infer.py", line 179, in vae_encode
latent = self.vae.encode(sample, tiled=self.encode_tiled, tile_size=self.encode_tile_size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae.p1685, in encode
p = super().encode(x, return_dict=return_dict, tiled=tiled, tile_size=tile_size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
return method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae.p1186, in encode
h = self.tiled_encode(x, tile_size=tile_size, tile_overlap=tile_overlap)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae.p1403, in tiled_encode
encoded_tile = self.slicing_encode(tile_sample)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae.p1259, in slicing_encode
self._encode(
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae.p1218, in _encode
h = self.encoder(_x, memory_state=memory_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py", line 468, in __call__
return super().__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py", line 1046, in compile_wrapper
raise e.with_traceback(None) from e.__cause__ # User compiler error
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch._dynamo.exc.Unsupported: Data-dependent branching
Explanation: Detected data-dependent branching (e.g. `if my_tensor.sum() > 0:`). Dynamo does not support tracic control flow.
Hint: This graph break is fundamental - it is unlikely that Dynamo will ever be able to trace through your coder finding a workaround.
Hint: Use `torch.cond` to express dynamic control flow.

Developer debug context: attempted to jump with TensorVariable()

For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/.html

from user code:
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/attn_video_vae. 815, in forward
sample = self.conv_in(sample, memory_state=memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 227, in forward
return self.slicing_forward(input, memory_state)
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 296, in slicing_forward
input[i] = self.memory_limit_conv(
File "/llm/ComfyUI/custom_nodes/ComfyUI-SeedVR2_VideoUpscaler/src/models/video_vae_v3/modules/causal_inflation line 135, in memory_limit_conv
if memory_occupy < self.memory_limit or split_dim == x.ndim:

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PFor even more developer context, set TORCH_LOGS="+dynamo"

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.