Could not call v1/chat/completion successfully in new langchain endpoint in openai-compatible server

Nobody has claimed this yet.

I tried to built a new endpoint in openai-compatible server which calls langchain in it and got an error.

Here's how I start the OpenAI compatible server:
```
python -m fastchat.serve.controller
```
```
python -m fastchat.serve.model_worker --model-names "gpt-3.5-turbo,text-davinci-003,text-embedding-ada-002" --model-path lmsys/vicuna-13b-v1.3 --load-8bit
```
```
python -m fastchat.serve.openai_api_server --host 0.0.0.0 --port 8080
set OPENAI_API_BASE=http://localhost:8080/v1
set OPENAI_API_KEY=EMPTY
```
And here is how I wrote the endpoint:
```
import necessary packages
@app.post("/v1/chat/langchain/basic", dependencies=[Depends(check_api_key)])
async def create_chat_completion_langchain(request: ChatCompletionRequest):
print("request: ", request)
test_langchain_routes.test_chat()
return "foo"

def test_chat():
chat = ChatOpenAI(temperature=0)
messages = [
SystemMessage(
content="You are a helpful assistant that translates English to French."
),
HumanMessage(
content="Translate this sentence from English to French. I love programming."
),
]
print(chat(messages))
return "foo"
```
When I call v1/langchain/basic endpoint, my server got stuck, and here is the error message:
```
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=600).

Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=600).

Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=600).

Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=600).

Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=600).

INFO: 127.0.0.1:64465 - "POST /v1/chat/langchain/basic HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
```

Two observations:
1. When I call test_chat() with python, it works fine and gives the correct response.
2. When I replace OPENAI_API_BASE with actual openai website and with api key but keep everything else same, and it also works well.
```
set OPENAI_API_BASE=https://api.openai.com/v1

set OPENAI_API_KEY=${openai api key}
```
Does anyone have ideas on how to fix the bug?

Reproduce the timeout using the listed controller, model_worker, and openai_api_server entry points, then inspect the request path for the custom /v1/chat/langchain/basic endpoint and its LangChain call. Done means the endpoint completes successfully and returns a response instead of retrying until a 500 error.

Written by the indexing model from the issue text.