[Feature] Support deploy model on specific device id
Open
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
I notice that if I want to deploy a LLM on a GPU device with specific device id, the only way I can do is `export CUDA_VISIBLE_DEVICES=xxx`.
Is there any other way to set the specific device id of lmdeploy instance? that the tensor from another device id can be addressed on this device, rather than be limited by the id listed in CUDA_VISIBLE_DEVICES?
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.