InternLM / InternLM/lmdeploy

[Bug] triton.runtime.autotuner.OutOfResources: out of resource: shared memory, Required: 108672, Hardware limit: 101376. Reducing block sizes or `num_stages` may help.

Open
#2,451 7 comments 0 reactions 1 assignee Claimed by @grimoire View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

你好,我尝试通过lmdeploy部署DeepSeek-Coder-V2-Lite-Instruct,报错如下:triton.runtime.autotuner.OutOfResources: out of resource: shared memory, Required: 108672, Hardware limit: 101376. Reducing block sizes or `num_stages` may help.
我的电脑是A6000 48G,我想部署16B的模型应该是可以的。
代码如下:
```
backend_config = PytorchEngineConfig(tp=1, block_size=32)
LLM = pipeline(self.MODEL_PATH,backend_config=backend_config)
```
我尝试减少了block_size,也尝试更换成了 turbomind backend,也尝试修改了cache_max_entry_count但这并没有帮助到我。

### Reproduction

1

### Environment

```Shell
1
```

### Error traceback

```Shell
1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.