smooth 量化后推理性能没有提升

i use the quantized model by smooth quant ， Why hasn't the inference speed increased?

`lmdeploy lite smooth_quant /model/llama2-7b-hf/ --work-dir /model/lmdeploy/llama2-7b-w8/`
使用 smooth 量化后，模型文件由 12.56GB 减小到 6.55GB，推理时显存占用只减少了 1GB，推理速度提升也不明显。
`python profile_throughput.py /dataset/ShareGPT_V3_unfiltered_cleaned_split.json /model/llama2-7b-hf/ --backend pytorch` result is 2850.681 token/s
but `python profile_throughput.py /dataset/ShareGPT_V3_unfiltered_cleaned_split.json /model/lmdeploy/llama2-7b-w8/ --backend pytorch` result is 2896.486 token/s

_Originally posted by @zxy1119 in https://github.com/InternLM/lmdeploy/issues/2295#issuecomment-2286196586_