intel / intel/auto-round

[Bug]: crash + resume (AR_RESUME_DIR, immediate saving) silently duplicates already-quantized modules as stale fp weights in the final artifact

Open
#2,350 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

### Problem Description

## Evidence (opt-125m e2e)

crash after 2 blocks -> resume -> artifact contains 12 duplicated tensors = exactly the 6 quantized Linears of the 2 skipped blocks (each with both `qweight` and `.weight`). No warning is logged.

Qwen3.8-27B W4A16 artifact came out 47.4 GB instead of ~17.7 GB with 318/407 modules duplicated this way — silently.

## Root cause

`ShardWriter.finalize()`'s remaining-weights pass writes every tensor still present in the model tree whose exact NAME was not saved yet. Resumed-over blocks present plain `nn.Linear` modules (no wrapper), so their packed names (`qweight` etc., saved before the crash) never match the plain names — the name-exact dedup misses and the stale fp weight is written on top of the packed one.

### Reproduction Steps

1. run any quantization with immediate saving + `AR_RESUME_DIR=`
2. kill the process after >= 1 block completes (manifest marks it done)
3. rerun the same command with the same `AR_RESUME_DIR` -> run resumes, skipped blocks are not re-quantized
4. inspect the final artifact: every module of a skipped block exists TWICE — the packed tensors saved before the crash AND a plain fp `.weight` written at save time

### Environment Information

- auto-round: reproduced on a build based on upstream `main` @ `32369a69`; `ShardWriter.finalize()` is unchanged through current `main` @ `9e2650f4`. transformers 5.16.1, torch 2.11.0+cu128, immediate-saving path

### Error Logs

```shell

```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at ShardWriter.finalize() and trace its remaining-weights pass, then reproduce with immediate saving and AR_RESUME_DIR by stopping after at least one block and resuming. Inspect the final artifact for skipped-block tensors; done means resumed quantized modules are not accompanied by stale plain fp weights or duplicated tensor names.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.