microsoft / microsoft/MInference
[Question]: What is the current highest supported version of vLLM?
@iofu728 is already working on this.
Since Aug 4, 2025.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 82
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
Describe the issue
I use vllm 0.6.6,but meet the following errror:
[rank0]: Traceback (most recent call last): [rank0]: File "/home/zyq/MInference/experiments/benchmarks/benchmark_e2e_vllm.py", line 74, in <module> [rank0]: run_target_length(args.context_window, llm, sampling_params, args.attn_type) [rank0]: File "/home/zyq/MInference/experiments/benchmarks/benchmark_e2e_vllm.py", line 30, in run_target_length [rank0]: outputs = llm.generate([prompt], sampling_params) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/utils.py", line 1021, in inner [rank0]: return fn(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/entrypoints/llm.py", line 462, in generate [rank0]: outputs = self._run_engine(use_tqdm=use_tqdm) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/entrypoints/llm.py", line 1242, in _run_engine [rank0]: step_outputs = self.llm_engine.step() [rank0]: ^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 1390, in step [rank0]: outputs = self.model_executor.execute_model( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/executor/gpu_executor.py", line 88, in execute_model [rank0]: output = self.driver_worker.execute_model(execute_model_req) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 343, in execute_model [rank0]: output = self.model_runner.execute_model( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/worker/model_runner_base.py", line 152, in _wrapper [rank0]: raise type(err)( [rank0]: TypeError: Error in model execution (input dumped to /tmp/err_execute_model_input_20250709-040337.pkl): minference_patch_vllm_executor.<locals>.vllm_attn_forward() got an unexpected keyword argument 'attn_type'
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.