[Bug] 运行batchsize异常

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

使用llama2-7b A100-80GB 单卡测试，256个并发打请求，实际触发运行的active_size只有200左右，在刚开始能快速出现250左右的active_size，后续会掉下来，200，160，190，200左右

### Reproduction

我尝试用256个请求不间断发送（每次结束立刻发送新的数据送往服务端），在实际运行中 在此处打印https://github.com/InternLM/lmdeploy/blob/main/src/turbomind/models/llama/LlamaBatch.cc#L1602 观察active_size实际运行batchsize的输出，发现只有200左右，请问这符合预期吗？我理解应该是250+左右吧

### Environment

```Shell
GPU A100卡 80GB
```

### Error traceback

_No response_