[Bug] The GPU memory doesn't change after changing batch_size

### Checklist

- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.

### Describe the bug

The GPU memory doesn't change after changing batch_size。These two results are as follows：
```
node_id=0 node_num=1
[TM][WARNING] [LlamaTritonModel] `cache_block_seq_len` is not set, default to 128.
[TM][INFO] Barrier(1)
world_size=1 tensor_para_size=1 pipeline_para_size=1
Model:
head_num: 32
kv_head_num: 32
size_per_head: 128
inter_size: 11008
num_layer: 32
vocab_size: 103168
attn_bias: 1
max_batch_size: 8
max_context_token_num: 8224
session_len: 2056
step_length: 1
cache_max_entry_count: 48
cache_block_seq_len: 128
cache_chunk_size: 1
use_context_fmha: 1
start_id: 1
tensor_para_size: 1
pipeline_para_size: 1
enable_custom_all_reduce: 0
model_name: internlm-chat-7b
model_dir: ./lmdeploy/workspace2/triton_models/weights
quant_policy: 0
group_size: 0
[TM][INFO] Barrier(1)
[INFO] rank = 0
[WARNING] gemm_config.in is not found; using default GEMM algo
[TM][INFO] NCCL group_id = 0
[TM][INFO] [BlockManager] block_size = 64 MB
[TM][INFO] [BlockManager] max_block_count = 48
[TM][INFO] [BlockManager] chunk_size = 1
[TM][ERROR] LlamaBatch::Start()
model instance 0 is created
[TM][INFO] [InternalThreadEntry] 0
after allocation : free: 24.62 GB, total: 39.45 GB, used: 14.83 GB
request_batch_size=8
max_input_len=1
request_batch_size=8 max_input_len=1
[INFO] request is created
[TM][INFO] [forward] Enqueue requests
[TM][INFO] [forward] Wait for requests to complete ...
[TM][WARNING] [ProcessInferRequests] Request for 0 received.
[TM][WARNING] [ProcessInferRequests] Request for 1 received.
[TM][WARNING] [ProcessInferRequests] Request for 2 received.
[TM][WARNING] [ProcessInferRequests] Request for 3 received.
[TM][WARNING] [ProcessInferRequests] Request for 4 received.
[TM][WARNING] [ProcessInferRequests] Request for 5 received.
[TM][WARNING] [ProcessInferRequests] Request for 6 received.
[TM][WARNING] [ProcessInferRequests] Request for 7 received.
[TM][INFO] [initGen] batch_size = 8
[TM][INFO] [initGen] max_context_len = 1
[TM][INFO] [initGen] slot sequence_id context_len seq_limit_len finished
[TM][INFO] [initGen] 0 0 1 2049 0
[TM][INFO] [initGen] 1 1 1 2049 0
[TM][INFO] [initGen] 2 2 1 2049 0
[TM][INFO] [initGen] 3 3 1 2049 0
[TM][INFO] [initGen] 4 4 1 2049 0
[TM][INFO] [initGen] 5 5 1 2049 0
[TM][INFO] [initGen] 6 6 1 2049 0
[TM][INFO] [initGen] 7 7 1 2049 0
[TM][INFO] ------------------------- step = 0 -------------------------
[TM][INFO] [CompleteRequest] slot = 0, id = 0
[TM][INFO] [CompleteRequest] slot = 1, id = 1
[TM][INFO] [CompleteRequest] slot = 2, id = 2
[TM][INFO] [CompleteRequest] slot = 3, id = 3
[TM][INFO] [CompleteRequest] slot = 4, id = 4
[TM][INFO] [CompleteRequest] slot = 5, id = 5
[TM][INFO] [CompleteRequest] slot = 6, id = 6
[TM][INFO] [CompleteRequest] slot = 7, id = 7
[TM][INFO] [forward] Request complete for 0, ec = 0
[TM][INFO] [forward] Request complete for 1, ec = 0
[TM][INFO] [forward] Request complete for 2, ec = 0
[TM][INFO] [forward] Request complete for 3, ec = 0
[TM][INFO] [forward] Request complete for 4, ec = 0
[TM][INFO] [forward] Request complete for 5, ec = 0
[TM][INFO] [forward] Request complete for 6, ec = 0
[TM][INFO] [forward] Request complete for 7, ec = 0
[INFO] forward is completed.
sequence length: 6, 6, 6, 6, 6, 6, 6, 6
[TM][ERROR] ~LlamaBatch()
[TM][INFO] [InternalThreadEntry] stop requested.
[TM][INFO] [OutputThreadEntry] stop requested.
```
```
node_id=0 node_num=1
[TM][WARNING] [LlamaTritonModel] `cache_block_seq_len` is not set, default to 128.
[TM][INFO] Barrier(1)
world_size=1 tensor_para_size=1 pipeline_para_size=1
Model:
head_num: 32
kv_head_num: 32
size_per_head: 128
inter_size: 11008
num_layer: 32
vocab_size: 103168
attn_bias: 1
max_batch_size: 16
max_context_token_num: 8224
session_len: 2056
step_length: 1
cache_max_entry_count: 48
cache_block_seq_len: 128
cache_chunk_size: 1
use_context_fmha: 1
start_id: 1
tensor_para_size: 1
pipeline_para_size: 1
enable_custom_all_reduce: 0
model_name: internlm-chat-7b
model_dir: ./lmdeploy/workspace2/triton_models/weights
quant_policy: 0
group_size: 0
[TM][INFO] Barrier(1)
[INFO] rank = 0
[WARNING] gemm_config.in is not found; using default GEMM algo
[TM][INFO] NCCL group_id = 0
[TM][INFO] [BlockManager] block_size = 64 MB
[TM][INFO] [BlockManager] max_block_count = 48
[TM][INFO] [BlockManager] chunk_size = 1
[TM][ERROR] LlamaBatch::Start()
model instance 0 is created
[TM][INFO] [InternalThreadEntry] 0
after allocation : free: 24.62 GB, total: 39.45 GB, used: 14.83 GB
request_batch_size=16
max_input_len=1
request_batch_size=16 max_input_len=1
[INFO] request is created
[TM][INFO] [forward] Enqueue requests
[TM][INFO] [forward] Wait for requests to complete ...
[TM][WARNING] [ProcessInferRequests] Request for 0 received.
[TM][WARNING] [ProcessInferRequests] Request for 1 received.
[TM][WARNING] [ProcessInferRequests] Request for 2 received.
[TM][WARNING] [ProcessInferRequests] Request for 3 received.
[TM][WARNING] [ProcessInferRequests] Request for 4 received.
[TM][WARNING] [ProcessInferRequests] Request for 5 received.
[TM][WARNING] [ProcessInferRequests] Request for 6 received.
[TM][WARNING] [ProcessInferRequests] Request for 7 received.
[TM][WARNING] [ProcessInferRequests] Request for 8 received.
[TM][WARNING] [ProcessInferRequests] Request for 9 received.
[TM][WARNING] [ProcessInferRequests] Request for 10 received.
[TM][WARNING] [ProcessInferRequests] Request for 11 received.
[TM][WARNING] [ProcessInferRequests] Request for 12 received.
[TM][WARNING] [ProcessInferRequests] Request for 13 received.
[TM][WARNING] [ProcessInferRequests] Request for 14 received.
[TM][WARNING] [ProcessInferRequests] Request for 15 received.
[TM][INFO] [initGen] batch_size = 16
[TM][INFO] [initGen] max_context_len = 1
[TM][INFO] [initGen] slot sequence_id context_len seq_limit_len finished
[TM][INFO] [initGen] 0 0 1 2049 0
[TM][INFO] [initGen] 1 1 1 2049 0
[TM][INFO] [initGen] 2 2 1 2049 0
[TM][INFO] [initGen] 3 3 1 2049 0
[TM][INFO] [initGen] 4 4 1 2049 0
[TM][INFO] [initGen] 5 5 1 2049 0
[TM][INFO] [initGen] 6 6 1 2049 0
[TM][INFO] [initGen] 7 7 1 2049 0
[TM][INFO] [initGen] 8 8 1 2049 0
[TM][INFO] [initGen] 9 9 1 2049 0
[TM][INFO] [initGen] 10 10 1 2049 0
[TM][INFO] [initGen] 11 11 1 2049 0
[TM][INFO] [initGen] 12 12 1 2049 0
[TM][INFO] [initGen] 13 13 1 2049 0
[TM][INFO] [initGen] 14 14 1 2049 0
[TM][INFO] [initGen] 15 15 1 2049 0
[TM][INFO] ------------------------- step = 0 -------------------------
[TM][INFO] [CompleteRequest] slot = 0, id = 0
[TM][INFO] [CompleteRequest] slot = 1, id = 1
[TM][INFO] [CompleteRequest] slot = 2, id = 2
[TM][INFO] [CompleteRequest] slot = 3, id = 3
[TM][INFO] [CompleteRequest] slot = 4, id = 4
[TM][INFO] [CompleteRequest] slot = 5, id = 5
[TM][INFO] [CompleteRequest] slot = 6, id = 6
[TM][INFO] [CompleteRequest] slot = 7, id = 7
[TM][INFO] [CompleteRequest] slot = 8, id = 8
[TM][INFO] [CompleteRequest] slot = 9, id = 9
[TM][INFO] [CompleteRequest] slot = 10, id = 10
[TM][INFO] [CompleteRequest] slot = 11, id = 11
[TM][INFO] [CompleteRequest] slot = 12, id = 12
[TM][INFO] [CompleteRequest] slot = 13, id = 13
[TM][INFO] [CompleteRequest] slot = 14, id = 14
[TM][INFO] [CompleteRequest] slot = 15, id = 15
[TM][INFO] [forward] Request complete for 0, ec = 0
[TM][INFO] [forward] Request complete for 1, ec = 0
[TM][INFO] [forward] Request complete for 2, ec = 0
[TM][INFO] [forward] Request complete for 3, ec = 0
[TM][INFO] [forward] Request complete for 4, ec = 0
[TM][INFO] [forward] Request complete for 5, ec = 0
[TM][INFO] [forward] Request complete for 6, ec = 0
[TM][INFO] [forward] Request complete for 7, ec = 0
[TM][INFO] [forward] Request complete for 8, ec = 0
[TM][INFO] [forward] Request complete for 9, ec = 0
[TM][INFO] [forward] Request complete for 10, ec = 0
[TM][INFO] [forward] Request complete for 11, ec = 0
[TM][INFO] [forward] Request complete for 12, ec = 0
[TM][INFO] [forward] Request complete for 13, ec = 0
[TM][INFO] [forward] Request complete for 14, ec = 0
[TM][INFO] [forward] Request complete for 15, ec = 0
[INFO] forward is completed.
sequence length: 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6
[TM][ERROR] ~LlamaBatch()
[TM][INFO] [InternalThreadEntry] stop requested.
[TM][INFO] [OutputThreadEntry] stop requested.
```
I find that
```
[TM][INFO] [InternalThreadEntry] 0
after allocation : free: 24.62 GB, total: 39.45 GB, used: 14.83 GB
```
is the same after I changed both max_batch_size and request_batch_size=16 from 8 to 16.

### Reproduction

CUDA_VISIBLE_DEVICES=1 ./bin/llama_triton_example

### Environment

```Shell
sys.platform: linux
Python: 3.9.18 (main, Sep 11 2023, 13:41:44) [GCC 11.2.0]
CUDA available: True
numpy_random_seed: 2147483648
GPU 0,1,2,3,4,5,6,7: NVIDIA A100-SXM4-40GB
CUDA_HOME: /tools/cluster-software/cuda-cudnn/cuda-11.7.1-8.5.0
NVCC: Cuda compilation tools, release 11.7, V11.7.99
GCC: gcc (GCC) 11.4.0
PyTorch: 1.13.0+cu117
PyTorch compiling details: PyTorch built with:
- GCC 9.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.7
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.5
- Magma 2.6.1
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.14.0+cu117
LMDeploy: 0.0.14+b7c88ca
transformers: 4.35.0
gradio: 3.50.2
fastapi: 0.104.1
pydantic: 2.4.2
```

### Error traceback

_No response_