Regarding issue with output format for Vicuna (lmsys/vicuna-7b-v1.5) using fastchat/serve/huggingface_api.py

Nobody has claimed this yet.

Hi!

I was using the [huggingface_api](https://github.com/lm-sys/FastChat/blob/main/fastchat/serve/huggingface_api.py) for inference on `lmsys/vicuna-7b-v1.5` using the following command:

`python3 -m fastchat.serve.huggingface_api --model lmsys/vicuna-7b-v1.5`

The **ASSISTANT's** output looks like (with the special characters "▁" and additional spaces):

> USER: Hello! Who are you?
> **ASSISTANT**: ▁I ' m ▁a ▁language ▁model ▁called ▁Vic una , ▁and ▁I ▁was ▁trained ▁by ▁Lar ge ▁Model ▁Systems ▁Organ ization ▁( L MS YS ) ▁research ers .

However, I was expecting the output to be clean:

> USER: Hello! Who are you?
> **ASSISTANT**: I'm a language model called Vicuna , and I was trained by Large Model Systems Organization (LMSYS) researchers.

I need to have clean output because I am performing multi-turn generation (i.e. pass the first response of the assistant back to the assistant as context for generating next response).

Sorry if I am missing something fundamental here but any help would be much appreciated!

Start with fastchat/serve/huggingface_api.py and reproduce the issue using `python3 -m fastchat.serve.huggingface_api --model lmsys/vicuna-7b-v1.5`. Trace how the generated response is decoded and displayed, paying attention to the special characters and extra spaces. Done means the assistant output is clean enough to pass back as context for multi-turn generation.

Written by the indexing model from the issue text.