[Bug]: MTP logic is unaware of Transformers' layer renaming and quantizes the entire model again
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 175
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 99
Description
### Problem Description
auto-round ./ --eval --eval_bs 16 --tasks lambada_openai
## 1 unexpected keys
]
[transformers] NemotronHForCausalLM LOAD REPORT from: ./
Key | Status | Details
-----------------------------------------+------------+--------
model.layers.{1...51}.mixer.gate.qzeros | UNEXPECTED |
model.layers.{1...51}.mixer.gate.qweight | UNEXPECTED |
model.layers.{1...51}.mixer.gate.scales | UNEXPECTED |
## 2 gate_proj issue
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/auto_round/modeling/fused_moe/moe_experts_interface.py", line 302, in linear_loop_experts_forward
final_hidden_states = _run_experts_with_routes(top_k_index, top_k_weights)
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/auto_round/modeling/fused_moe/moe_experts_interface.py", line 279, in _run_experts_with_routes
gate_out = expert.gate_proj(expert_input) # (num_samples, intermediate_dim)
^^^^^^^^^^^^^^^^
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1972, in __getattr__
raise AttributeError(
f"'{type(self).__name__}' object has no attribute '{name}'"
)
AttributeError: '_ExpertContainer' object has no attribute 'gate_proj'
### Reproduction Steps
~
### Environment Information
_No response_
### Error Logs
```shell
```
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.