InternLM / InternLM/lmdeploy

[Bug] The quantization process of Qwen/Qwen2-VL-7B-Instruct is getting killed without throwing error.

Open
#2,770 2 comments 0 reactions 1 assignee Claimed by @AllentDan View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

I am trying to AWQ 4-bit quantize Qwen/Qwen2-VL-7B-Instruct using LM DEPLOY awq quantization algorithm.
![image](https://github.com/user-attachments/assets/83a4d058-8ca6-4e75-aa28-b6aa9278afea)

### Reproduction

export HF_MODEL=Qwen/Qwen2-VL-7B-Instruct

export WORK_DIR = /Qwen/Qwen2-VL-7B-Instruct-4BIT-MODEL

lmdeploy lite auto_awq $HF_MODEL --calib-dataset 'ptb' --calib-samples 4 --calib-seqlen 64 --w-bits 4 --w-group-size 128 --batch-size 1 --work-dir $WORK_DIR

### Environment

```Shell
conda create -n lmdeploy python=3.8 -y
conda activate lmdeploy
pip install lmdeploy
```

### Error traceback

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.