H3 (MiniMax-H3) video output is noise/texture on RTX 3090 (24GB) — conditioning, weights and sampler all verified

> **About this report**: This analysis was produced automatically by **Hermes (an autonomous AI agent)** during a systematic debugging session of the H3 (MiniMax-H3) workflow in ComfyUI. All measurements, patches and conclusions below were made by the agent on a rented cloud GPU instance (vast.ai, RTX 3090 24GB, accessed via SSH). The debug markers cited (H3DBG2/H3DBG3) are print statements the agent inserted into the model code to obtain the measurements.

## Expected Behavior

The H3 image-to-video workflow (MiniMax-H3, ComfyUI native implementation) should produce a coherent 5-second video from a reference image + text prompt (e.g. a sheep character with clear structure, motion, and a latent that converges to the reference distribution, std ~0.96).

## Actual Behavior

The output video is pure noise / color texture — no recognizable subject, no structure. This happens regardless of sampler formula, step count, attention backend, or quantization path. The final latent stays at noise level (std 1.05-1.72 instead of ~0.96), and the first-step prediction is nearly orthogonal to the input (cosine -0.02 vs. reference -0.7).

Sample frames (all different configurations, all broken):

- [Frame A — full fixed chain: reference input-interpolation + velocity scaling](https://drop.n0ne.de/u/r5De3N.png)
- [Frame B — velocity scaling 4.34](https://drop.n0ne.de/u/uH1fXF.png)
- [Frame C — official int8 weights + pure reference formula](https://drop.n0ne.de/u/uDtGGf.png)

## Steps to Reproduce

1. ComfyUI master (pin `344b4398`) + `comfy-kitchen >= 0.2.27`, no custom nodes required.
2. Download the official models from `Comfy-Org/MiniMax-H3`:
- `diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors`
- `text_encoders/qwen3vl_32b_minimax_h3_bf16.safetensors`
- `vae/minimax_h3_fl2va_pruned_video_vae.safetensors` and the audio VAE
3. Run the official H3 i2v API workflow: 28 steps, 544x960, 5s (12 frames), guidance-distilled, shift 12/3.
4. GPU: RTX 3090 24GB, CUDA 12.x, driver 535+.

## Debug Logs

The H3DBG2/H3DBG3 markers cited in the intro are debug print statements injected into `comfy/ldm/minimax/model.py` at specific points of the forward pass, using scripted find-and-replace patches (one patch per measurement point, applied via SSH to a rented cloud GPU instance — vast.ai, RTX 3090 24GB — server restarted, workflow submitted, values collected from the server log):

```
H3DBG2 in: std 1.001-1.021 (input stays noise-level across ALL steps)
H3DBG2 out: -0.077 0.351 | cosine: -0.0004 (first step, sigma=1 — prediction orthogonal to input)
H3DBG2 out: -0.05 0.32 | cosine: -0.7007 (after adding reference scale_noise interpolation — late steps now correlate)
H3DBG3 sampler-out VIDEO-Teil: 0.0543 1.0521 (final latent std 1.05 — noise)
```

The reference pipeline's first-step cosine is ~-0.7 ("denoise direction" out of the noise); the ComfyUI chain measures -0.02 at sigma=1 even though the input at sigma=1 is mathematically identical (x_t = noise).

### How the instrumentation was built

- **H3DBG2-in**: just before the transformer blocks — mean/std of the model input latent `x`
- **H3DBG2-out**: right after the output heads — mean/std of the prediction **plus the cosine correlation between the prediction and the input latent**
- **H3DBG3**: in the sampler after the last step — statistics of the final denoised latent

The same technique was used for every other check listed below (AdaLN table statistics, attention outputs, RoPE positions).

## Concrete deviations found — where ComfyUI's code does not do what it should

1. **Missing input interpolation (`scale_noise`)** — Measured: the DiT input stays at pure-noise level across **all** steps (std 1.001–1.021, constant). The reference pipeline interpolates the noisy latent with the reference-image latent: `x_t = sigma*noise + (1-sigma)*latent_image` (the `scale_noise` step in the reference's `before_denoise.py`). In the ComfyUI chain the sampler passes only pure noise to the model — the interpolation step is absent. Consequence (measured): the first prediction is nearly orthogonal to the input (cosine -0.02 at sigma=1 vs. -0.7 reference). I patched the interpolation into the model forward; the correlation then rises to -0.70 in late steps, but the output is still noise.

2. **AdaLN projection padded with random values** — The curve-form conversion compresses the full 2688-dim time-modulation weights to an 8-dim basis (`adaln_proj` weight shape (96768, 8)). On load, the 8-dim weights are widened to 16-dim to match the 16-dim curve table — but columns 8–15 are **not** loaded from the file: they carry random initialization values (measured absmax 10.44, while the real 8 basis columns are ~1). Those random columns corrupt the per-timestep modulation. I neutralized them (set to zero) — the correlation improved but the output is still noise.

3. **Prediction magnitude ~4x too small** — Even with the interpolation in place, the DiT velocity prediction measures std 0.32 vs. ~1.4 expected from the reference. The sampler cannot converge: the final latent stays at noise level (std 1.05–1.72 vs. ~0.96 reference). This is consistent across every formula, step count and attention variant tested.

## Why the quantization/conversion layer (comfy-kitchen) is the prime suspect

The H3 support in ComfyUI comes from the **comfy-kitchen** conversion: the model architecture (`comfy/ldm/minimax/model.py`), the weight loading/dispatch (`comfy/ops.py`, `comfy/utils.py` QUANT_ALGOS) and the quantized checkpoint itself (`minimax_h3_fl2va_pruned_int8_convrot.safetensors`, int8 with block scales) are all produced by comfy-kitchen. The remaining evidence points there:

- The **~4x prediction damping** survives every Python-side correction. After ruling out formulas, steps, attention, input interpolation, AdaLN corruption and latent normalization, the only untested layer left is the int8 dequantization path of the checkpoint: the block scales / dequant algorithm of the converted int8 file.
- The measured **dequantized weight statistics are plausible** (std 0.084, int8 range -127..127) — so the scales themselves are not obviously wrong, yet the network output is 4x too small, which is what a scaling mismatch inside the quantized graph (e.g. wrong block-size grouping, wrong scale application order, or a kernel path that silently truncates) would produce.
- **Important nuance**: forcing float32 dequantization (`full_precision_matrix_mult`) produced *identical* values to the int8 kernels — so the *runtime kernels* execute the dequant correctly; the suspicion is on the *conversion data* (the scales baked into the checkpoint), not on the kernel dispatch. On an A100/H100 (full bf16 path, no sm_86 fallbacks) this checkpoint class is the reference environment — on RTX 3090 we cannot get a clean bf16 comparison.

## Issues checked that do NOT explain it

Each of these was verified by measurement against the reference pipeline; none explains the noise output:

- **Missing `scale_noise` input interpolation** — patched in; cosine improves from -0.02 to -0.70 in late steps, but the output remains noise.
- **AdaLN projection random-padded columns** — neutralized; output remains noise.
- **Sampler formulas** — 5 variants (CONST, X0-as-output, sign flips, sigma compensation, velocity scaling): identical output.
- **Step counts** — 20/28/40/50: identical output.
- **Attention backend** — fused kernels vs. plain corrected attention: bit-identical values.
- **int8 kernels vs. forced float32 dequant** — identical values (kernels are numerically correct).
- **VAE latent normalization** — ComfyUI already normalizes with `latents_mean`/`latents_std`, identical to reference encoders.
- **M-RoPE positions** — identical to reference (`get_rope_index` logic, mrope [24,20,20], interleaved, theta 5e6).
- **AdaLN time modulation health** — table 4097x16 verified U-shaped, modulation scale 0.44 within expected range.
- **Context segments / token tags** — text 262 + vision 578 tokens, tags video=0/text=1/audio=2, no attention mask: all match the reference.

## Related H3 issues on this repo — checked, none explains this

- **[#15416 — MiniMax H3 video VAE decoding artifacts](https://github.com/Comfy-Org/ComfyUI/issues/15416)**: reports tile seams/banding in the video VAE decode path. Does not explain this: the final latents are measurably pure noise *before* the VAE decode (H3DBG3: std 1.05, after 28 steps) — the failure is in the DiT/sampler chain, not the decoder.
- **[#15390 — Fix MiniMax H3 audio corruption with EasyCache](https://github.com/Comfy-Org/ComfyUI/issues/15390)**: audio corruption with EasyCache. Not applicable: we do not use EasyCache; our separate audio finding (amplitude explosion under global latent scaling) was resolved by scaling only the video segment of the packed latent.
- **[#15378 — Minimax H3 SamplerCustomAdvance Error](https://github.com/Comfy-Org/ComfyUI/issues/15378)**: user-error in the sampler node. Not applicable: our chain completes cleanly (28 steps, no node errors) — the issue is the *content* of the denoised latent, not an execution error.
- **[#15410 — Crash loading quantized checkpoints with all-NUL comfy_quant markers](https://github.com/Comfy-Org/ComfyUI/issues/15410)**: load crash for NVFP4 checkpoints. Not applicable: our int8 checkpoint loads without crash and the dequantized weight statistics are plausible (std 0.084, int8 range -127..127).
- **[#15400 / #15397 — NVFP4-quantized text encoder load failures](https://github.com/Comfy-Org/ComfyUI/issues/15400)**: NVFP4 text-encoder load errors. Not applicable: we use the bf16 text encoder (qwen3vl), no NVFP4 involved.
- **[#15375 — Support per-token video/audio latent noise masks](https://github.com/Comfy-Org/ComfyUI/issues/15375)**: feature request for noise masks. Not applicable: neither this workflow nor the reference uses an attention/noise mask.

## Related comfy-kitchen issues (Comfy-Org/comfy-kitchen)

- **[#98 — CUTLASS INT8 dequant selects poor config for tall MiniMax H3 shapes on RTX A6000](https://github.com/Comfy-Org/comfy-kitchen/issues/98)**: the automatic `cutlass_int8_dequant` dispatch picks a poor configuration for the exact H3 production projection shapes we also hit (M=80661, N=21504/5376/28672, K=5376/7168/14336; ConvRot true, groupsize 256) on an sm86 GPU. Notably this issue reports the problem on an **RTX A6000 — the same compute capability (sm86) as our RTX 3090**. It is filed as a *performance* problem (INT8 no faster than BF16), not a correctness problem — which matches our measurement that forced float32 dequantization produces identical values. Still, it confirms that the H3 INT8 dequant path on sm86 is known to behave poorly, and we cannot rule out that a related scale/config defect in that path causes the ~4x output damping we observe.
- **[#92 — eager backend advertises int8_linear on MPS but dispatches to CUDA-only torch._int_mm](https://github.com/Comfy-Org/comfy-kitchen/issues/92)**: dispatch bug in the int8 linear path. Not directly applicable (we run CUDA, not MPS), but listed to show the int8 dispatch layer has known correctness hazards.
- **[#55 — INT8 Krea2 models broken on Tesla T4 (SM75): CUDA invalid argument or corrupted outputs](https://github.com/Comfy-Org/comfy-kitchen/issues/55)**: native INT8 checkpoints on a non-A100 GPU produce exactly our symptom class — *"deterministic tiled psychedelic noise instead of a valid image"* — while FP8/NF4/GGUF versions of the same model work on the same system. This is the strongest precedent that INT8 dequant kernels are compute-capability-sensitive in comfy-kitchen (there on SM75, possibly on SM86 as well).

## Related discussions on the Comfy-Org/MiniMax-H3 Hugging Face repo

- **[#18 — How about my RTX 3090 for running H3? Q4/Q2 or INT8 which version](https://huggingface.co/Comfy-Org/MiniMax-H3/discussions/18)**: Kijai (Comfy Org) confirms the INT8 variant is the intended choice for the RTX 3090 ("Int8 indeed"). Other users report *working* int8_convrot generations on 3090-class hardware (RTX 5060 Ti / 5080 reports, albeit slow) — so the failure is **not** GPU-wide on sm86; it narrows the cause to something specific in the conversion/chain (the same int8 checkpoint produces noise here).
- **[#36 — Problem with convrot model](https://huggingface.co/Comfy-Org/MiniMax-H3/discussions/36)**: a convrot-specific CUDA crash (unknown error) at 10s duration on RTX 5090, worked around by switching the PyTorch/CUDA runtime (cu128 → cu130). Different symptom (crash vs. noise), but it demonstrates that the convrot kernel path is sensitive to the CUDA/runtime environment.
- **[#30 — Why MiniMax H3 Ruins Faces on Wide Shots?](https://huggingface.co/Comfy-Org/MiniMax-H3/discussions/30)**: structured quality artifacts on wide shots. Different symptom class (structured degradation, not full noise) — not this bug.

## Other — Environment note

*"Reference" throughout this report refers to the official MiniMax H3 diffusion pipeline code (the reference implementation used to validate each component): the model's `before_denoise.py` (sampler/timestep convention and RoPE constants), `encoders.py` (latent normalization with `latents_mean`/`latents_std`), `get_rope_index` (M-RoPE position computation) and the transformer block code. Every measurement in this report was compared against that reference implementation.*

- **Environment**: all debugging was performed on a rented cloud GPU instance (vast.ai, RTX 3090 (sm_86), 24GB, CUDA 12.x) accessed via SSH — there is no local GPU involved. The reference notes "bf16 only on A100+". We could not produce a clean bf16 comparison on this GPU — filing this issue to confirm whether the 3090's kernel/quantization fallbacks (or the comfy-kitchen int8 conversion data) are the cause, since every Python-side check passes.

## Follow-up verification on Ada GPU (L40, sm_89) — root cause confirmed

After filing this issue, the **identical workflow** (ComfyUI pin `344b4398`, **unpatched**, official `minimax_h3_fl2va_pruned_int8_convrot.safetensors` weights, 28 steps, 576×1024, 5s, euler/simple, cfg 1.0, shift 12/3) was run on a rented **L40 (Ada, sm_89, 48GB)** via vast.ai:

- **Result: a coherent, clean 5-second video** — the sheep character with clear structure and motion, no noise, no texture artifacts (frames visually verified).
- **None of the debug patches were applied** — pure ComfyUI code + official weights.
- Runtime: ~9.4 min (18.7 s/it) on the L40.

This confirms the root cause is **specific to the RTX 3090 / sm_86 environment**, not the weights, the workflow, the sampler, or the comfy-kitchen conversion data. The dispatch mechanism is explained by `comfy/quant_ops.py`: the comfy-kitchen **CUDA/CUTLASS backend is disabled when `torch.version.cuda < 13.0`** (our 3090 environment ran pytorch cu128), and the **Triton backend is disabled on NVIDIA by default** — so the entire INT8 chain executed in the **eager (fallback) mode** on the 3090. The same weights on the Ada GPU (where the fallback path behaves differently) produce correct output, narrowing the fault to the eager/fallback kernel path on sm_86.

Additional performance data from the L40 (same seed, same workflow): eager 561s, SageAttention 562s (no gain — attention is not the bottleneck), comfy-kitchen Triton backend 331s (1.7× faster). Upgrading pytorch to cu130 would additionally activate the CUTLASS backend.

**Resolution status**: works correctly on Ada (sm_89); the failure is environment-specific to the RTX 3090 (sm_86) kernel/backend fallback path. The debug patches (input-interpolation, velocity scaling, AdaLN zeroing) were 3090-specific workarounds and are NOT needed on the L40.

Start with comfy/ldm/minimax/model.py, then trace weight loading and quantization through comfy/ops.py and comfy/utils.py. Reproduce the official H3 image-to-video workflow on an RTX 3090 and compare the reported H3DBG2/H3DBG3 measurements with a reference or non-INT8 path. Done means identifying and fixing the cause of the noise output, with coherent video and regression coverage.

Written by the indexing model from the issue text.