microsoft / microsoft/MInference
[Question]: ModuleNotFoundError: No module named 'minference.cuda'
@iofu728 is already working on this.
Since Jul 17, 2024.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 82
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
Describe the issue
I encountered some issues when using minference in Python.
import minference
The problem is Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/workspace/MInference/minference/__init__.py", line 8, in <module>
from .models_patch import MInference
File "/workspace/MInference/minference/models_patch.py", line 7, in <module>
from .patch import minference_patch, minference_patch_vllm, patch_hf
File "/workspace/MInference/minference/patch.py", line 12, in <module>
from .modules.minference_forward import (
File "/workspace/MInference/minference/modules/minference_forward.py", line 20, in <module>
from ..ops.pit_sparse_flash_attention_v2 import vertical_slash_sparse_attention
File "/workspace/MInference/minference/ops/pit_sparse_flash_attention_v2.py", line 10, in <module>
from ..cuda import convert_vertical_slash_indexes
ModuleNotFoundError: No module named 'minference.cuda'
enviornment :
Python 3.10.14
minference 0.1.4.post3
triton 2.1.0
torch 2.3.0
CUDA 11.8
vllm 0.4.2+cu118
flash-attn 2.5.8
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.