[Bug] unified_attention split kv for prefill with more workspace coredump
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
When using prefix cache, if the input length is very long and the matched prefix is also very long, the efficiency of calculating the prefill in this situation is relatively low. According to the comment 'but only have 1 workspace disable split kv for prefill for now', allocating a separate workspace for prefill still results in a coredump.
### Reproduction
Start the lmdeploy server with any 7B model, output long data, and ensure the prefix easily matches a long sequence. Modify unified_attention to allocate a separate workspace for prefill, then perform a stress test.
### Environment
```Shell
latest version
```
### Error traceback
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.