intel / intel/auto-round

[Bug]: MTP logic is unaware of Transformers' layer renaming and quantizes the entire model again

Open
#2,324 2 comments 0 reactions 1 assignee Assigned to @xin3he View on GitHub
bug
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

### Problem Description

auto-round ./ --eval --eval_bs 16 --tasks lambada_openai

## 1 unexpected keys
]
[transformers] NemotronHForCausalLM LOAD REPORT from: ./
Key | Status | Details
-----------------------------------------+------------+--------
model.layers.{1...51}.mixer.gate.qzeros | UNEXPECTED |
model.layers.{1...51}.mixer.gate.qweight | UNEXPECTED |
model.layers.{1...51}.mixer.gate.scales | UNEXPECTED |

## 2 gate_proj issue

File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/auto_round/modeling/fused_moe/moe_experts_interface.py", line 302, in linear_loop_experts_forward
final_hidden_states = _run_experts_with_routes(top_k_index, top_k_weights)
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/auto_round/modeling/fused_moe/moe_experts_interface.py", line 279, in _run_experts_with_routes
gate_out = expert.gate_proj(expert_input) # (num_samples, intermediate_dim)
^^^^^^^^^^^^^^^^
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1972, in __getattr__
raise AttributeError(
f"'{type(self).__name__}' object has no attribute '{name}'"
)
AttributeError: '_ExpertContainer' object has no attribute 'gate_proj'

### Reproduction Steps

~

### Environment Information

_No response_

### Error Logs

```shell

```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.