Version compatibility problem
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
A new version of vllm is available, which supports inference using bnb quantization. The properties of the vllm SamplingParams class have changed in the latest vllm version, but the fastchat code does not appear to have been updated to accommodate the new version of vllm, resulting in a package version compatibility issue. Here is the error message for accessing the model: TypeError: Unexpected keyword argument 'use beam search'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating FastChat's use of the vllm SamplingParams class and compare its arguments with the installed vllm version that raises the error. Reproduce the model-access failure, then verify that the compatibility issue is resolved without breaking supported vllm usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100