[Bug] LMDeploy server running internVL2-8B shut down without error report when user uploads certain images

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

I followed the documentation on [hugging face](https://huggingface.co/OpenGVLab/InternVL2-8B) and the documentation for [lmdeploy](https://lmdeploy.readthedocs.io/zh-cn/latest/serving/api_server_vl.html) and deployed the LMDeploy service running internVL2-8B. When I use the openAI API and upload certain images, the LMDeploy service simply shuts down without reporting any errors.

This only happens when uploading a specific portion of images, the service works fine when using other images (such as the [tiger image](https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg) from the official documentation). Here is a test image that causes the service to shut down.


It only happens when I using API. The model can works fine in offline inference.

The reference code and server running log(info level) is as follows.

### Reproduction

server deploy command:

```
lmdeploy serve api_server OpenGVLab/InternVL2-8B --model-name InternVL2-8B --backend turbomind --server-port 23333 --chat-template chat_template.json --api-keys ********* --log-level INFO
```

client to request server:

```
from openai import OpenAI

client = OpenAI(api_key="*********", base_url='http://*********:23333/v1')

model_name = client.models.list().data[0].id
response = client.chat.completions.create(
model="InternVL2-8B",
messages=[{
'role':
'user',
'content': [{
'type': 'text',
'text': '请你描述一下这张图片',
}, {
'type': 'image_url',
'image_url': {
'url':
'http://*********/thumb-1920-987888.jpg',
},
}],
}],
temperature=0.8,
top_p=0.8)
print(response)
```

offline inference to test model:

```
from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig, GenerationConfig
from lmdeploy.vl import load_image

model = 'OpenGVLab/InternVL2-8B'
system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态大语言模型。'
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,
backend_config=TurbomindEngineConfig(session_len=8192))

image = load_image('http://*********/thumb-1920-987888.jpg')
gen_config = GenerationConfig(top_k=40, top_p=0.8, temperature=0.8)

sess = pipe.chat(('请描述这张图片', image), gen_config=gen_config)
print(sess.response.text)
```

chat_template.json:

```
{
"model_name":"internvl-internlm2",
"meta_instruction":"我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态大语言模型。",
"stop_words":["<|im_start|>", "<|im_end|>"]
}
```

### Environment

```Shell
sys.platform: linux
Python: 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0: NVIDIA GeForce RTX 4090
CUDA_HOME: /usr/local/cuda-12.5
NVCC: Cuda compilation tools, release 12.5, V12.5.82
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.3) 9.4.0
PyTorch: 2.2.2+cu121
```

### Error traceback

Here is the info log report in last rows before server shut down.

```Shell
2024-07-22 14:21:17,526 - lmdeploy - INFO - session_id=3, history_tokens=0, input_tokens=2357, max_new_tokens=None, seq_start=True, seq_end=True, step=0, prep=True
2024-07-22 14:21:17,526 - lmdeploy - INFO - Register stream callback for 3
[TM][INFO] [forward] Enqueue requests
[TM][INFO] [forward] Wait for requests to complete ...
[TM][INFO] [ProcessInferRequests] Request for 3 received.
[TM][WARNING] [ProcessInferRequests] [3] total sequence length (2357 + 30419) exceeds `session_len` (32776), `request_output_len` is truncated to 30418
[TM][INFO] [Forward] [0, 1), dc_bsz = 0, pf_bsz = 1, n_tok = 2357, max_q = 2357, max_k = 2357
已放弃 (核心已转储)
```