[Bug] PyTorch Engine poor performance compared to vllm

### Checklist

- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.

### Describe the bug

I tried to benchmark the PyTorch Engine performance and find it very poor...

`PyTorch Engine`
concurrency: 4
input token Throughput: 101.53 tokens/s
output token Throughput: 93.32 tokens/s
total token Throughput: 194.85 tokens/s

`vllm`
concurrency: 4
input token Throughput: 184.18 tokens/s
output token Throughput: 169.28 tokens/s
total token Throughput: 353.46 tokens/s

Is it normal? Do I miss something when use PyTorch Engine?

### Reproduction

model: Qwen14B
GPU: A100
LMDeploy: 0.3.0
Dataset: ShareGPT_V3_unfiltered_cleaned_split.json
profile_restful_api.py

### Environment

```Shell
LMDeploy: 0.3.0
```

### Error traceback

_No response_