Comfy-Org / Comfy-Org/comfy-kitchen
H3 (MiniMax) on RTX 5060 Ti (sm_120): text conditioning ignored + numerical blowup in DiT blocks
- Dominant language
- Python
- Stars
- 220
- Forks
- 91
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 12
Description
# H3 (MiniMax) on RTX 5060 Ti (sm_120): text conditioning ignored + numerical blowup in DiT blocks
## Environment
- GPU: RTX 5060 Ti 16GB (sm_120 Blackwell)
- OS: Windows 11 + WSL2
- ComfyUI 0.34.2 (Windows native)
- torch 2.12.0+cu130 (also reproduced on 2.9.1+cu130)
- comfy-kitchen 0.2.31
## Models
- `minimax_h3_fl2va_pruned_w4a8_mixed.safetensors` (12.5GB, W4A8)
- `minimax_h3_fl2va_pruned_int8_convrot.safetensors` (20.9GB, INT8)
## Setup
krea2 CLIP (`qwen3vl_4b_fp8_scaled`) + `mmh3-4b-ClipProj-celeb-mlp` projection, BasicGuider + BasicScheduler(simple, 4 steps) + SamplerCustomAdvanced + MiniMaxH3SigmaShift(12, 3), turbo 4-step LoRA, 768x432x124 frames.
## Symptoms
1. Output is a fixed woven texture pattern, not the prompted scene. Both "mountain lake" and "sports car" prompts yield **byte-identical frames** (MD5 match at frames 0/30/60/90).
2. Different prompts → identical output, **even though the context tensors differ** (ctxsum -5617 vs -3591). Conditioning injection path verified working.
3. W4A8 and INT8 checkpoints → **identical output frames** (same MD5), suggesting the model is not actually driven by its weights.
4. **Numerical blowup**: after the 84 DiT blocks, text rows reach std ≈ 32256 (input std ≈ 2.26), video rows std ≈ 6944. Weights load correctly (block0 qkv_proj std ≈ 0.084, condition_proj 5376x5120).
## Debug trail (all verified with instrumentation)
- On torch 2.9.1 the model degraded to an **identity mapping** (DiT block outputs ≈ 0, latent never updated — sampling "ran" but output was the empty-latent VAE decode). Upgrading to torch 2.12.0 fixed this: the model now produces real dynamic output, but still ignores text.
- Text conditioning verified: layout segments `[(0,20,'text'), (20,434,'audio'), (434,12866,'video')]`, text_states (20,5376) injected at h[0:20] correctly, context content differs per prompt.
- Standalone kernel tests pass: `rms_rope_split_half_` (eager backend), torch SDPA, int8_linear (rel diff 0.18% vs eager).
- Zeroing the context produces byte-identical output → text has **zero** influence on generation.
- SDXL generation works fine on the same machine (does not use comfy-kitchen quantized ops).
## Related
- #120 (same GPU family, GPU LOST under comfy-kitchen)
- #55 (INT8 corrupted outputs on SM75)
Any hints? Is this a known incompatibility between Blackwell (sm_120) and comfy-kitchen quantized ops inside the H3 DiT? Happy to run any additional experiments or provide logs.
Contributor guide
Research direction
Start by reproducing the reported H3 generation on an RTX 5060 Ti with the listed W4A8 and INT8 checkpoints, then compare outputs with context zeroed and with each checkpoint. The report identifies comfy-kitchen quantized operations and notes standalone tests for RMS RoPE, SDPA, and int8_linear; inspect those paths and the DiT block behavior. Done means text conditioning influences generation and the numerical blowup is resolved, with results distinguishing the two checkpoints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 43/100