InternLM / InternLM/Tutorial

lmdeploy 部署量化出现错误

Aperta
#1,893 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
2k
Fork
1.5k
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

lmdeploy lite auto_awq \
/root/models/internlm2_5-1_8b-chat \
--calib-dataset 'ptb' \
--calib-samples 128 \
--calib-seqlen 2048 \
--w-bits 4 \
--w-group-size 128 \
--batch-size 1 \
--search-scale False \
--work-dir /root/models/internlm2_5-1_8b-chat-w4a16-4bit
输入此命令后
报错显示

```
Traceback (most recent call last):
File "/root/.conda/envs/lmdeploy/bin/lmdeploy", line 8, in
sys.exit(run())
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 36, in run
args.run(args)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/cli/lite.py", line 139, in auto_awq
auto_awq(**kwargs)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/apis/auto_awq.py", line 80, in auto_awq
vl_model, model, tokenizer, work_dir = calibrate(model,
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/apis/calibrate.py", line 216, in calibrate
calib_loader, _ = get_calib_loaders(calib_dataset,
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 302, in get_calib_loaders
return get_ptb(tokenizer, nsamples, seed, seqlen)
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 58, in get_ptb
traindata = load_dataset('ptb_text_only', 'penn_treebank', split='train')
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/datasets/load.py", line 2074, in load_dataset
builder_instance = load_dataset_builder(
File "/root/.conda/envs/lmdeploy/lib/python3.10/site-packages/datasets/load.py", line 1832, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
TypeError: 'NoneType' object is not callable
```
如何解决?

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.