unable to communicate with mistral models deployed in EC2 by using gradio_webserver API

Nobody has claimed this yet.

I have deployed mistral7b and 8x&B models in two AWS EC2 server using vllm and fstchat , and now I want to communicate through a web interface(gradio_webserver) and i am using api_endpoints.json file to register models.

**api_endpoints.json**

{
"Mistral": {
"model_name": "mistral7b",
"api_type": "openai",
"api_base": "http://10.0.0.0/v1",
"api_key": "sk-******",
"anony_only": false,
"recommended_config": {
"temperature": 0.7,
"top_p": 1.0
},
"text-arena": true,
"vision-arena": false,
}
}

**error from interface:**

**command used to launch webUI**

python3 -m fastchat.serve.gradio_web_server --host 0.0.0.0 --port 8080 --controller-url "" --share --register api_endpoints.json

Start with api_endpoints.json and the gradio_web_server launch command, then verify how the registered OpenAI endpoint connects to the Mistral deployments on EC2. Reproduce the reported interface failure and use the displayed error to determine whether the endpoint configuration or server communication is incorrect; done means the web interface communicates with both deployed models.

Written by the indexing model from the issue text.