InternLM / InternLM/lmdeploy

[Feature] Low priority: Allow specifying HuggingFace model/repo name in `lmdeploy convert`

Open
#1,749 2 comments 0 reactions 1 assignee Claimed by @lvhan028 View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

This is **not** an important feature, but I figured I'd mention it because it was a small point of friction that I think could be improved in the future. Currently my script does this:
```
pip install 'huggingface_hub[cli,hf_transfer]==0.23.2'
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli download lmdeploy/llama2-chat-70b-4bit --local-dir /root/llama2-chat-70b-4bit
lmdeploy convert llama2 /root/llama2-chat-70b-4bit --model-format awq --group-size 128 --tp $(nvidia-smi -L | wc -l) --dst-path /root/turbomind-model-files
```
Ideally I could just write this:
```
lmdeploy convert llama2 lmdeploy/llama2-chat-70b-4bit --model-format awq --group-size 128 --tp $(nvidia-smi -L | wc -l) --dst-path /root/turbomind-model-files
```
And it as a bonus nice-to-have it would be cool if I only needed enough disk space to fit one version of the model - e.g. for 70B 4bit model, if my disk space was 40GB, ideally that would be enough. But currently ~80GB is required because both AWQ and turbomind formats must be stored on disk at the same time until turbomind conversion is complete. But this feature is not very important because disk space is cheap.

### Related resources

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.