Comfy-Org / Comfy-Org/ComfyUI

NVFP4 checkpoints without `_quantization_metadata` load silently as unquantized and crash in F.linear (LTX-2.5 nvfp4)

Open
#15,511 2 comments 2 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

Loading an NVFP4-quantized diffusion model via UNETLoader should either
1. detect the quantization from the weight_scale / weight_scale_2 / input_scale tensors present in the file and load it through MixedPrecisionOps, or
2. fail loudly with a message that names the actual problem.

### Actual Behavior

Neither happens. `Lightricks/LTX-2.5`'s official `diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` loads "successfully", all 3528 quantization scale tensors are silently thrown away, and sampling then dies on the first attention projection of the first transformer block:

```
RuntimeError: mat1 and mat2 shapes cannot be multiplied (3520x4096 and 2048x4096)
```

`2048` is exactly half of the model's hidden dim `4096` — it is the packed FP4 weight `[out, in/2]` (uint8, two values per byte) being handed to `torch.nn.functional.linear` as if it were a real weight matrix.

The only hint that anything went wrong is a `[WARNING] unet unexpected: [...]` line listing 3528 keys, which is one single ~198 KB line in the log and easy to miss. Note the asymmetry in the log — the text encoder gets a quantization path, the diffusion model does not:

```
[INFO] Found quantization metadata version 1 <- text encoder only
[INFO] Using MixedPrecisionOps for text encoder
...
[INFO] model weight dtype torch.bfloat16, manual cast: None <- UNET: no quant path
[INFO] model_type FLUX
[WARNING] unet unexpected: ['transformer_blocks.0.attn1.to_q.input_scale',
'transformer_blocks.0.attn1.to_q.weight_scale_2',
'transformer_blocks.0.attn1.to_q.weight_scale', ... ] <- 3528 keys dropped
```

There is no `unet missing:` line — the packed weights themselves were accepted. Only the scales were dropped.

### Steps to Reproduce

1. ComfyUI 0.32.0, Blackwell GPU (NVFP4 compute supported).
2. Download `diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` from https://huggingface.co/Lightricks/LTX-2.5 (18.7 GB).
3. Open the stock `video_ltx2_5_t2v` template that ships with ComfyUI 0.32.0.
4. In the subgraph's `UNETLoader`, change `unet_name` from `ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensors` to `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors`. Change nothing else.
5. Queue. It fails at `SamplerCustomAdvanced` at step 0.
Reproduces regardless of `weight_dtype` on the loader (`default` / `fp8_e4m3fn` / `fp8_e5m2`), sampler, sigmas, CFG or resolution — none of those are related.

The state dict of that file:

| dtype | count | role |
|---|---|---|
| `U8` | 1176 | packed 4-bit weights, `[out, in/2]` |
| `F8_E4M3` | 1176 | `weight_scale` (block scales, group size 16) |
| `F32` | 1176 + 1176 + 290 | `weight_scale_2`, `input_scale`, misc |
| `BF16` | 2882 | unquantized remainder |

1176 = 42 blocks × 28 linear modules. Layer names carry the `model.diffusion_model.` prefix. Its `__metadata__` contains only `['config', 'gemma_source_checkpoint', 'license', 'model_version']`.

### Debug Logs

```powershell
[INFO] got prompt
[INFO] Model LTXAV prepared for dynamic VRAM loading. 24592MB Staged. 0 patches attached. Force pre-loaded 608 weights: 3303 KB.
0%| | 0/8 [00:00

Contributor guide

Open the contributing guide

Research direction

Reproduce with the LTX-2.5 NVFP4 checkpoint through UNETLoader, then trace the diffusion-model loading path and compare it with the text encoder's MixedPrecisionOps path. Inspect the loading and model-call entry points shown in comfy/model_base.py and comfy/ldm/lightricks/av_model.py. Done means the scale tensors are handled for the diffusion model or loading fails with a specific diagnostic instead of reaching F.linear with packed weights.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.