microsoft / microsoft/MInference

[Question]: What is the current highest supported version of vLLM?

Open
#168 1 comment 0 reactions 1 assignee View on GitHub

@iofu728 is already working on this.

Since Aug 4, 2025.

question
Dominant language
Python
Stars
1.2k
Forks
82
Avg merge
1d 18h
Merged PRs (30d)
1

Description

Describe the issue

I use vllm 0.6.6,but meet the following errror:

[rank0]: Traceback (most recent call last): [rank0]: File "/home/zyq/MInference/experiments/benchmarks/benchmark_e2e_vllm.py", line 74, in <module> [rank0]: run_target_length(args.context_window, llm, sampling_params, args.attn_type) [rank0]: File "/home/zyq/MInference/experiments/benchmarks/benchmark_e2e_vllm.py", line 30, in run_target_length [rank0]: outputs = llm.generate([prompt], sampling_params) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/utils.py", line 1021, in inner [rank0]: return fn(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/entrypoints/llm.py", line 462, in generate [rank0]: outputs = self._run_engine(use_tqdm=use_tqdm) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/entrypoints/llm.py", line 1242, in _run_engine [rank0]: step_outputs = self.llm_engine.step() [rank0]: ^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 1390, in step [rank0]: outputs = self.model_executor.execute_model( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/executor/gpu_executor.py", line 88, in execute_model [rank0]: output = self.driver_worker.execute_model(execute_model_req) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 343, in execute_model [rank0]: output = self.model_runner.execute_model( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "/data/miniconda3/envs/lmcache/lib/python3.11/site-packages/vllm/worker/model_runner_base.py", line 152, in _wrapper [rank0]: raise type(err)( [rank0]: TypeError: Error in model execution (input dumped to /tmp/err_execute_model_input_20250709-040337.pkl): minference_patch_vllm_executor.<locals>.vllm_attn_forward() got an unexpected keyword argument 'attn_type'

No response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.