vllm-project / vllm-project/production-stack
BitsandBytes not installed in 'lmcache/vllm-openai' image.
Open
Nobody has claimed this yet.
feature request
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
Getting error for bitsandbytes library not being installed, when i run the model in quantized mode.
Why do you need this feature?
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Inspect the image definition for lmcache/vllm-openai and reproduce the reported failure when starting a quantized model. Identify how Python dependencies are installed in that image, then verify that importing bitsandbytes succeeds and the quantized model starts successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- devops, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100