intel / intel/auto-round

Track Wan2.2 dual-transformer quantization memory and tuning cost

Open
#2,235 0 comments 0 reactions 1 assignee Claimed by @changwangss View on GitHub
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

## Description

Quantizing Wan2.2-T2V-A14B-Diffusers with NVFP4 successfully completes the primary transformer, but the run has high resource cost before switching to `transformer_2`:

- Peak RAM: 564.85 GB
- Peak VRAM: 72.64 GB
- Primary-transformer tuning time: 47,823 seconds

PR #2230 fixes the independent stale `self.quantizer` access that crashes when starting `transformer_2`. This issue tracks reducing memory usage and tuning time separately.

## Reproduction

```bash
CUDA_VISIBLE_DEVICES=1 auto-round \
--model /data3/changwa1/Wan2.2-T2V-A14B-Diffusers \
--scheme NVFP4 \
--format llm_compressor \
--device_map 0 \
--batch_size 1 \
--nsamples 64 \
--seqlen 2048 \
--iters 200 \
--num_inference_steps 25 \
--low_gpu_mem_usage \
--model_dtype bf16
```

## Expected behavior

Dual-transformer diffusion quantization should avoid retaining unnecessary model/calibration state between transformer passes and provide a practical tuning cost for Wan2.2.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.