Add new vllm serving options

Nobody has claimed this yet.

I would like to propose to add a new system to to serve vllm workers so that all its capabilities are exposed. Currently there is no support for greedy search and beamsearch from vllm workers.

I have done modifying the chat completion endpoint. However, it is specifically written for vllm workers, not the transformers model_workers.

The preliminary code can be found at
https://github.com/tjtanaa/fastchat-serve/tree/vllm-api-expose-tj

Custom OpenAI API server with best_of, use_beam_search
https://github.com/tjtanaa/fastchat-serve/blob/vllm-api-expose-tj/src/fastchat_serve/openai_api_server_vllm.py

Modified vllm worker to support greedy search and beam search.
https://github.com/tjtanaa/fastchat-serve/blob/vllm-api-expose-tj/src/fastchat_serve/vllm_worker.py

Would this be helpful?

Start by reviewing the referenced fastchat_serve/openai_api_server_vllm.py and fastchat_serve/vllm_worker.py changes, then compare them with the current vLLM serving and chat completion entry points. Done means vLLM workers expose the requested greedy and beam-search capabilities, including the proposed options, with behavior covered by the relevant tests.

Written by the indexing model from the issue text.