NVFP4 checkpoints without `_quantization_metadata` load silently as unquantized and crash in F.linear (LTX-2.5 nvfp4)
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
Loading an NVFP4-quantized diffusion model via UNETLoader should either
1. detect the quantization from the weight_scale / weight_scale_2 / input_scale tensors present in the file and load it through MixedPrecisionOps, or
2. fail loudly with a message that names the actual problem.
### Actual Behavior
Neither happens. `Lightricks/LTX-2.5`'s official `diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` loads "successfully", all 3528 quantization scale tensors are silently thrown away, and sampling then dies on the first attention projection of the first transformer block:
```
RuntimeError: mat1 and mat2 shapes cannot be multiplied (3520x4096 and 2048x4096)
```
`2048` is exactly half of the model's hidden dim `4096` — it is the packed FP4 weight `[out, in/2]` (uint8, two values per byte) being handed to `torch.nn.functional.linear` as if it were a real weight matrix.
The only hint that anything went wrong is a `[WARNING] unet unexpected: [...]` line listing 3528 keys, which is one single ~198 KB line in the log and easy to miss. Note the asymmetry in the log — the text encoder gets a quantization path, the diffusion model does not:
```
[INFO] Found quantization metadata version 1 <- text encoder only
[INFO] Using MixedPrecisionOps for text encoder
...
[INFO] model weight dtype torch.bfloat16, manual cast: None <- UNET: no quant path
[INFO] model_type FLUX
[WARNING] unet unexpected: ['transformer_blocks.0.attn1.to_q.input_scale',
'transformer_blocks.0.attn1.to_q.weight_scale_2',
'transformer_blocks.0.attn1.to_q.weight_scale', ... ] <- 3528 keys dropped
```
There is no `unet missing:` line — the packed weights themselves were accepted. Only the scales were dropped.
### Steps to Reproduce
1. ComfyUI 0.32.0, Blackwell GPU (NVFP4 compute supported).
2. Download `diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` from https://huggingface.co/Lightricks/LTX-2.5 (18.7 GB).
3. Open the stock `video_ltx2_5_t2v` template that ships with ComfyUI 0.32.0.
4. In the subgraph's `UNETLoader`, change `unet_name` from `ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensors` to `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors`. Change nothing else.
5. Queue. It fails at `SamplerCustomAdvanced` at step 0.
Reproduces regardless of `weight_dtype` on the loader (`default` / `fp8_e4m3fn` / `fp8_e5m2`), sampler, sigmas, CFG or resolution — none of those are related.
The state dict of that file:
| dtype | count | role |
|---|---|---|
| `U8` | 1176 | packed 4-bit weights, `[out, in/2]` |
| `F8_E4M3` | 1176 | `weight_scale` (block scales, group size 16) |
| `F32` | 1176 + 1176 + 290 | `weight_scale_2`, `input_scale`, misc |
| `BF16` | 2882 | unquantized remainder |
1176 = 42 blocks × 28 linear modules. Layer names carry the `model.diffusion_model.` prefix. Its `__metadata__` contains only `['config', 'gemma_source_checkpoint', 'license', 'model_version']`.
### Debug Logs
```powershell
[INFO] got prompt
[INFO] Model LTXAV prepared for dynamic VRAM loading. 24592MB Staged. 0 patches attached. Force pre-loaded 608 weights: 3303 KB.
0%| | 0/8 [00:00
Contributor guide
Research direction
Reproduce with the LTX-2.5 NVFP4 checkpoint through UNETLoader, then trace the diffusion-model loading path and compare it with the text encoder's MixedPrecisionOps path. Inspect the loading and model-call entry points shown in comfy/model_base.py and comfy/ldm/lightricks/av_model.py. Done means the scale tensors are handled for the diffusion model or loading fails with a specific diagnostic instead of reaching F.linear with packed weights.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100