33B AWQ量化+vLLM部署问题

vLLM 部署33B的模型，用autoAWQ量化时会报：
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████| 7/7 [00:45<00:00, 6.54s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Token indices sequence length is longer than the specified maximum sequence length for this model (98937 > 16384). Running this sequence through the model will result in indexing errors
Killed

看了下autoawq会自动加载一个mit-han-lab/pile-val-backup数据集，但是似乎没有正确切分，于是我手动切分之后再运行量化，代码如下：
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
from datasets import load_dataset

model_path = '/hy-tmp/deepseek-coder-33b-instruct'
quant_path = '/hy-tmp/deepseek-coder-33b-instruct-awq'
quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }

def load_pileval():
data = load_dataset("json",data_files="/usr/local/miniconda3/lib/python3.8/site-packages/awq/utils/pile-val-backup/val.jsonl",split="train")
return [text for text in data["text"] if text.strip() != '' and len(text.split(' ')) > 20 and len(text)<512]

model = AutoAWQForCausalLM.from_pretrained(model_path, trust_remote_code=True, device_map="auto",**{"low_cpu_mem_usage": True})
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

model.quantize(tokenizer, quant_config=quant_config, calib_data=load_pileval())

model.save_quantized(quant_path)
tokenizer.save_pretrained(quant_path)

再运行之后，直接就把进程kill了，用的是A100 40G，过程中监控了一下似乎也没有OOM，请问下各位有解决方案吗？

Start with the reported quantization script and the awq/utils/pile-val-backup/val.jsonl calibration data path, then inspect the model.quantize entry point and its logs. Compare the default calibration loading with the manually filtered calib_data and determine why the process is killed without an observed GPU OOM. Done means a reproducible cause and a documented working resolution for the 33B AWQ deployment.

Written by the indexing model from the issue text.