[Bug] run glm4-32b-0414 error
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
### Describe the bug
`2025-05-23 14:13:19,161 - lmdeploy - WARNING - archs.py:55 - Try to run with pytorch engine because `./GLM-4-32B-0414/` is not explicitly supported by lmdeploy.
2025-05-23 14:13:23,915 - lmdeploy - ERROR - base.py:53 - RuntimeError: Triton Error [CUDA]: device kernel image is invalid
2025-05-23 14:13:23,916 - lmdeploy - ERROR - base.py:54 - check failed!
This Error might caused by mismatching between NVIDIA Driver and nvcc compiler.
Try solution https://github.com/triton-lang/triton/issues/1955#issuecomment-1929908209 or reinstall the driver.`
### Reproduction
lmdeploy serve api_server ./GLM-4-32B-0414/ --server-name 0.0.0.0 --server-port 11401 --tp 1 --cache-max-entry-count 0.8 --session-len 32000 --chat-template glm4
### Environment
```Shell
GPU 0,1,2,3,4,5,6,7: Tesla V100-PCIE-32GB
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.7, V11.7.64
GCC: gcc (GCC) 7.3.0
PyTorch: 2.5.1+cu118
PyTorch compiling details: PyTorch built with:
- GCC 9.3
- C++ Version: 201703
- Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v3.5.3 (Git Hash 66f0cb9eb66affd2da3bf5f8d897376f04aae6af)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX512
- CUDA Runtime 11.8
- NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90
- CuDNN 90.1
- Magma 2.6.1
TorchVision: 0.20.1+cu118
LMDeploy: 0.7.3+
transformers: 4.51.3
gradio: 3.40.0
fastapi: 0.115.6
pydantic: 2.10.5
triton: 3.1.0
```
### Error traceback
```Shell
```
Contributor guide
Assessment
This issue has not been assessed yet.