How to cancel tensor parallel in vllm of the master-worker strategy.

Nobody has claimed this yet.

Hi, I am making multi replica of the vllm instance with the fast-chat. But I found the `master-worker` is unstable which sometimes increase its output while sometimes not. After many experiments, I found that this is caused by the `Tensor Parallel`, the vllm's request in a period of time will be responsed all in one, which causes the sharpe decrease of the GPU utility. This may cause the low utility of the GPU groups. So I want to know how to cancel this feature to process the request one by one.

Start by reviewing the FastChat master-worker serving path and its vLLM tensor-parallel configuration, then reproduce the multi-replica setup described in the issue. Compare GPU utilization and response batching with tensor parallel enabled; done requires a confirmed way to disable or control it, or a clearly documented limitation.

Written by the indexing model from the issue text.