vllm serve fails with ModuleNotFoundError: No module named 'vllm._C'
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 603
- PR merge metrics
- No merged PRs in 30d
Description
I am following installation guide on Windows 11 x64.
vllm serve fails
(.venv) PS D:\Experiments\fara> vllm serve "microsoft/Fara-7B" --port 5000 --dtype auto
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "D:\Experiments\fara\.venv\Scripts\vllm.exe\__main__.py", line 2, in <module>
from vllm.entrypoints.cli.main import main
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\entrypoints\cli\__init__.py", line 3, in <module>
from vllm.entrypoints.cli.benchmark.latency import BenchmarkLatencySubcommand
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\entrypoints\cli\benchmark\latency.py", line 5, in <module>
from vllm.benchmarks.latency import add_cli_args, main
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\benchmarks\latency.py", line 17, in <module>
from vllm.engine.arg_utils import EngineArgs
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\engine\arg_utils.py", line 35, in <module>
from vllm.attention.backends.registry import AttentionBackendEnum
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\attention\__init__.py", line 4, in <module>
from vllm.attention.backends.abstract import (
...<3 lines>...
)
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\attention\backends\abstract.py", line 9, in <module>
from vllm.model_executor.layers.linear import ColumnParallelLinear
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\model_executor\__init__.py", line 4, in <module>
from vllm.model_executor.parameter import BasevLLMParameter, PackedvLLMParameter
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\model_executor\parameter.py", line 11, in <module>
from vllm.distributed import (
...<2 lines>...
)
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\distributed\__init__.py", line 4, in <module>
from .communication_op import *
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\distributed\communication_op.py", line 9, in <module>
from .parallel_state import get_tp_group
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\distributed\parallel_state.py", line 250, in <module>
direct_register_custom_op(
~~~~~~~~~~~~~~~~~~~~~~~~~^
op_name="all_reduce",
^^^^^^^^^^^^^^^^^^^^^
op_func=all_reduce,
^^^^^^^^^^^^^^^^^^^
fake_impl=all_reduce_fake,
^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\utils\torch_utils.py", line 640, in direct_register_custom_op
from vllm.platforms import current_platform
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\platforms\__init__.py", line 257, in __getattr__
_current_platform = resolve_obj_by_qualname(platform_cls_qualname)()
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\utils\import_utils.py", line 89, in resolve_obj_by_qualname
module = importlib.import_module(module_name)
File "C:\Users\rail\AppData\Local\Programs\Python\Python313\Lib\importlib\__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Experiments\fara\.venv\Lib\site-packages\vllm\platforms\cuda.py", line 16, in <module>
import vllm._C # noqa
^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'vllm._C'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the installation guide and the reported vllm serve command on Windows 11 x64. Trace the import path through the listed vllm.platforms.cuda and vllm._C entry points, then verify the documented platform and installation expectations. Done means the guide or issue outcome clearly explains this invocation and its supported result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100