Comfy-Org / Comfy-Org/ComfyUI

H3 Ref2VA: safe max frame length drops sharply as reference image/video count increases; corrupted output reports success in the log

Open
#15,738 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

## Environment
- OS: Windows 11 Pro (10.0.26200)
- CPU: AMD Ryzen 9 7950X3D (16C/32T)
- RAM: 128GB
- GPU: RTX 4090 24GB, driver 32.0.16.1047
- Python: 3.12.11
- PyTorch: 2.13.0+cu130 (CUDA 13.0, cuDNN 9.2.0)
- ComfyUI: v0.33.1
- Model: MiniMax-H3-Ref2VA FP8 (minimax_h3_ref2va_pruned_fp8_scaled.safetensors)
+ minimax_h3_ref2v_turbo_4step LoRA, 6 steps, euler/normal
- Output resolution: 1344x768
- Reference video: 608x352, 97 frames, correctly wired to `ref_videos` (not `ref_images`)

## Bug
As the number of reference images/videos fed into `MiniMaxH3ReferenceToVideo` increases,
the maximum safe output `length` before the result becomes full-frame RGB noise corruption
drops. Critically, **the execution log reports success with correct metadata (frame count,
duration, resolution) even when the output is fully corrupted** — only visual inspection
reveals the failure.

## Reproduction data (seed fixed, all other params identical unless noted)

| Reference load | max safe length | failure beyond threshold |
|---|---|---|
| ~none (video mistakenly wired to `ref_images`, only 1st frame used) | 311 | 328: full RGB noise (bitrate 47.4 Mbps) / 345: silent blank corruption (log reports success) / 362: `CUDA error: illegal memory access` (hard crash, reproduced 3x across restarts) |
| ref_images=2, ref_videos=1 (correctly wired, 97 frames) | 294 | 311: full RGB noise (bitrate 24.4 Mbps) |
| ref_images=3, ref_videos=1 | 277 | 294: full RGB noise (bitrate 47.4 Mbps) |
| ref_images=4, ref_videos=1 | 277 (not pushed further) | — |

Note: the relationship is not strictly linear — going from 3 to 4 reference images did not
require lowering the length further in our tests (the 3rd/4th images were visually similar
frames cut from the same source video, so their added information content may have been low).

## Notes
- Distinct from #15426 (periodic dark frames every 17 frames) — this is *complete*
corruption of the entire output past a threshold, not periodic dimming.
- Corrupted outputs have anomalously high bitrate (~24-47 Mbps vs ~8-10 Mbps for a correct
output of the same resolution/length) — could serve as an automatic sanity check.
- A server-side check on output tensor statistics would help, since the execution log
gives no indication of failure.

Contributor guide

Open the contributing guide

Research direction

Start at the MiniMaxH3ReferenceToVideo node and reproduce the reported reference-load and output-length thresholds with the fixed seed and listed model setup. Compare successful and corrupted outputs, including tensor statistics and the CUDA illegal-memory-access case; done means the corruption is prevented or reliably detected instead of being logged as a successful result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.