deepseek-ai / deepseek-ai/DeepGEMM
Can not install DeepGEMM with commit of 95e81b3
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
I install deep_gemm for sglang, however, meet somg bug can't conquer:
command:
```bash
cd DeepGEMM
git submodule update --init --recursive
git checkout 95e81b3
pip install .
```
console print:
```
Traceback (most recent call last):
File "/usr/local/bin/mineru-sglang-server", line 5, in
from mineru.cli.vlm_sglang_server import main
File "/usr/local/lib/python3.10/dist-packages/mineru/cli/vlm_sglang_server.py", line 1, in
from ..model.vlm_sglang_model.server import main
File "/usr/local/lib/python3.10/dist-packages/mineru/model/vlm_sglang_model/__init__.py", line 1, in
from sglang.srt.configs.model_config import multimodal_model_archs
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/model_config.py", line 26, in
from sglang.srt.layers.quantization import QUANTIZATION_METHODS
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/__init__.py", line 56, in
from sglang.srt.layers.quantization.blockwise_int8 import BlockInt8Config
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/blockwise_int8.py", line 21, in
from sglang.srt.layers.quantization.utils import is_layer_skipped
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/utils.py", line 8, in
from sglang.srt.layers.quantization.fp8_kernel import scaled_fp8_quant
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/fp8_kernel.py", line 26, in
from sglang.srt.layers.quantization.deep_gemm import _ENABLE_JIT_DEEPGEMM
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/deep_gemm.py", line 16, in
import deep_gemm
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/__init__.py", line 13, in
from .utils import bench, bench_kineto, calc_diff
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/utils/__init__.py", line 1, in
from . import layout, math
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/utils/layout.py", line 6, in
from ..jit.compiler import get_device_arch
ImportError: cannot import name 'get_device_arch' from 'deep_gemm.jit.compiler' (/usr/local/lib/python3.10/dist-packages/deep_gemm/jit/compiler.py)
```
I can not find get_device_arch in any files!! 😭
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.