Encountered issues while using OpenGVLab/InternVL3-1B during quantization
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
using the latest lmdeploy with below codes:
lmdeploy lite auto_awq \
$HF_MODEL \
--calib-dataset 'ptb' \
--calib-samples 128 \
--calib-seqlen 2048 \
--w-bits 4 \
--w-group-size 128 \
--work-dir $WORK_DIR
meets below ERROR:
Loading calibrate dataset ...
`trust_remote_code` is not supported anymore.
Please check that the Hugging Face dataset 'ptb_text_only' isn't based on a loading script and remove `trust_remote_code`.
If the dataset is based on a loading script, please ask the dataset author to remove it and convert it to a standard format like Parquet.
Traceback (most recent call last):
File "/root/miniconda3/envs/vla/bin/lmdeploy", line 8, in
sys.exit(run())
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 39, in run
args.run(args)
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/cli/lite.py", line 111, in auto_awq
auto_awq(**kwargs)
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/lite/apis/auto_awq.py", line 86, in auto_awq
vl_model, model, tokenizer, work_dir = calibrate(model,
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/lite/apis/calibrate.py", line 295, in calibrate
calib_loader, _ = get_calib_loaders(calib_dataset, tokenizer, nsamples=calib_samples, seqlen=calib_seqlen)
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 296, in get_calib_loaders
return get_ptb(tokenizer, nsamples, seed, seqlen)
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 58, in get_ptb
traindata = load_dataset('ptb_text_only', 'penn_treebank', split='train', trust_remote_code=True)
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/datasets/load.py", line 1392, in load_dataset
builder_instance = load_dataset_builder(
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/datasets/load.py", line 1132, in load_dataset_builder
dataset_module = dataset_module_factory(
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/datasets/load.py", line 1031, in dataset_module_factory
raise e1 from None
File "/root/miniconda3/envs/vla/lib/python3.10/site-packages/datasets/load.py", line 953, in dataset_module_factory
raise ConnectionError(f"Couldn't reach '{path}' on the Hub ({e.__class__.__name__})") from e
ConnectionError: Couldn't reach 'ptb_text_only' on the Hub (LocalEntryNotFoundError)
how to solve this problem, or should I change the parameters to quantize InternVL3-1B
Contributor guide
Assessment
This issue has not been assessed yet.