deepseek-ai / deepseek-ai/DeepGEMM

Can not install DeepGEMM with commit of 95e81b3

Open
#127 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
7.8k
Forks
1.3k
Avg merge
3d 7h
Merged PRs (30d)
3

Description

I install deep_gemm for sglang, however, meet somg bug can't conquer:

command:
```bash
cd DeepGEMM
git submodule update --init --recursive
git checkout 95e81b3
pip install .
```

console print:
```
Traceback (most recent call last):
File "/usr/local/bin/mineru-sglang-server", line 5, in
from mineru.cli.vlm_sglang_server import main
File "/usr/local/lib/python3.10/dist-packages/mineru/cli/vlm_sglang_server.py", line 1, in
from ..model.vlm_sglang_model.server import main
File "/usr/local/lib/python3.10/dist-packages/mineru/model/vlm_sglang_model/__init__.py", line 1, in
from sglang.srt.configs.model_config import multimodal_model_archs
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/model_config.py", line 26, in
from sglang.srt.layers.quantization import QUANTIZATION_METHODS
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/__init__.py", line 56, in
from sglang.srt.layers.quantization.blockwise_int8 import BlockInt8Config
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/blockwise_int8.py", line 21, in
from sglang.srt.layers.quantization.utils import is_layer_skipped
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/utils.py", line 8, in
from sglang.srt.layers.quantization.fp8_kernel import scaled_fp8_quant
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/fp8_kernel.py", line 26, in
from sglang.srt.layers.quantization.deep_gemm import _ENABLE_JIT_DEEPGEMM
File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/quantization/deep_gemm.py", line 16, in
import deep_gemm
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/__init__.py", line 13, in
from .utils import bench, bench_kineto, calc_diff
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/utils/__init__.py", line 1, in
from . import layout, math
File "/usr/local/lib/python3.10/dist-packages/deep_gemm/utils/layout.py", line 6, in
from ..jit.compiler import get_device_arch
ImportError: cannot import name 'get_device_arch' from 'deep_gemm.jit.compiler' (/usr/local/lib/python3.10/dist-packages/deep_gemm/jit/compiler.py)
```
I can not find get_device_arch in any files!! 😭

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.