InternLM / InternLM/Tutorial

lmdeploy 部署量化出现错误

未关闭
#1,893 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
2k
派生
1.5k
PR 合并指标
30 天内没有已合并 PR

描述

lmdeploy lite auto_awq \
/root/models/internlm2_5-1_8b-chat \
--calib-dataset 'ptb' \
--calib-samples 128 \
--calib-seqlen 2048 \
--w-bits 4 \
--w-group-size 128 \
--batch-size 1 \
--search-scale False \
--work-dir /root/models/internlm2_5-1_8b-chat-w4a16-4bit
输入此命令后
报错显示

```
Traceback (most recent call last):
File "/root/.conda/envs/lmdeploy/bin/lmdeploy", line 8, in
sys.exit(run())
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 36, in run
args.run(args)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/cli/lite.py", line 139, in auto_awq
auto_awq(**kwargs)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/apis/auto_awq.py", line 80, in auto_awq
vl_model, model, tokenizer, work_dir = calibrate(model,
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/apis/calibrate.py", line 216, in calibrate
calib_loader, _ = get_calib_loaders(calib_dataset,
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 302, in get_calib_loaders
return get_ptb(tokenizer, nsamples, seed, seqlen)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 58, in get_ptb
traindata = load_dataset('ptb_text_only', 'penn_treebank', split='train')
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/datasets/load.py", line 2074, in load_dataset
builder_instance = load_dataset_builder(
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/datasets/load.py", line 1832, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
TypeError: 'NoneType' object is not callable
```
如何解决?

贡献指南

这个仓库没有索引到贡献指南

调研方向

Start with the lmdeploy/lite/utils/calib_dataloader.py get_ptb entry point and the datasets.load_dataset call shown in the traceback. Reproduce the command in a clean environment, then inspect the installed datasets package and its compatibility with lmdeploy. Done means identifying the cause of the NoneType error and documenting a verified resolution.

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
machine-learning
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。