[embedding model & connect with Langchain] "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error

Nobody has claimed this yet.

**current fastchat version:** 0.2.36

I served the embedding model in openai style with the code below and I want to connet with LangChain, but errors occured! how to deal with this problem?

-- serve embedding model comand
```shell
CUDA_VISIBLE_DEVICES=2 ~/.conda/envs/fastchat/bin/python \
-m fastchat.serve.model_worker \
--model-path /public/llm/models/bge-large-zh-v1.5 \
--model-names "alias-embeddings,gpt-3.5-turbo,text-davinci-003,text-embedding-ada-002,bge-large-zh-v1.5"\
--port 31002 \
--worker http://localhost:31002 &
```

-- LangChain
```python
from langchain.prompts.example_selector import SemanticSimilarityExampleSelector
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings

a = OpenAIEmbeddings(
model='text-embedding-ada-002'
)
a.embed_documents(['hi there', '你好'])
```
-- Error
```text
2024-03-29 16:27:01 | INFO | stdout | INFO: 127.0.0.1:58774 - "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error
2024-03-29 16:27:01 | ERROR | stderr | ERROR: Exception in ASGI application
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
2024-03-29 16:27:01 | ERROR | stderr | sock = connection.create_connection(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
2024-03-29 16:27:01 | ERROR | stderr | for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/socket.py", line 962, in getaddrinfo
2024-03-29 16:27:01 | ERROR | stderr | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | socket.gaierror: [Errno -2] Name or service not known
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
2024-03-29 16:27:01 | ERROR | stderr | response = self._make_request(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
2024-03-29 16:27:01 | ERROR | stderr | raise new_e
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
2024-03-29 16:27:01 | ERROR | stderr | self._validate_conn(conn)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
2024-03-29 16:27:01 | ERROR | stderr | conn.connect()
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 611, in connect
2024-03-29 16:27:01 | ERROR | stderr | self.sock = sock = self._new_conn()
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 210, in _new_conn
2024-03-29 16:27:01 | ERROR | stderr | raise NameResolutionError(self.host, self, e) from e
2024-03-29 16:27:01 | ERROR | stderr | urllib3.exceptions.NameResolutionError: : Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
2024-03-29 16:27:01 | ERROR | stderr | resp = conn.urlopen(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
2024-03-29 16:27:01 | ERROR | stderr | retries = retries.increment(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
2024-03-29 16:27:01 | ERROR | stderr | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NameResolutionError(": Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)"))
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | During handling of the above exception, another exception occurred:
2024-03-29 16:27:01 | ERROR | stderr |
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
2024-03-29 16:27:01 | ERROR | stderr | result = await app( # type: ignore[func-returns-value]
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2024-03-29 16:27:01 | ERROR | stderr | return await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await super().__call__(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await self.middleware_stack(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
2024-03-29 16:27:01 | ERROR | stderr | raise exc
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await self.app(scope, receive, _send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr | raise exc
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr | await app(scope, receive, sender)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
2024-03-29 16:27:01 | ERROR | stderr | await self.middleware_stack(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
2024-03-29 16:27:01 | ERROR | stderr | await route.handle(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
2024-03-29 16:27:01 | ERROR | stderr | await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
2024-03-29 16:27:01 | ERROR | stderr | await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr | raise exc
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr | await app(scope, receive, sender)
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
2024-03-29 16:27:01 | ERROR | stderr | response = await func(request)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
2024-03-29 16:27:01 | ERROR | stderr | raw_response = await run_endpoint_function(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
2024-03-29 16:27:01 | ERROR | stderr | return await dependant.call(**values)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/fastchat/serve/openai_api_server.py", line 717, in create_embeddings
2024-03-29 16:27:01 | ERROR | stderr | request.input = process_input(request.model, request.input)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/fastchat/serve/openai_api_server.py", line 245, in process_input
2024-03-29 16:27:01 | ERROR | stderr | decoding = tiktoken.model.encoding_for_model(model_name)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/model.py", line 101, in encoding_for_model
2024-03-29 16:27:01 | ERROR | stderr | return get_encoding(encoding_name_for_model(model_name))
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/registry.py", line 73, in get_encoding
2024-03-29 16:27:01 | ERROR | stderr | enc = Encoding(**constructor())
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken_ext/openai_public.py", line 72, in cl100k_base
2024-03-29 16:27:01 | ERROR | stderr | mergeable_ranks = load_tiktoken_bpe(
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 147, in load_tiktoken_bpe
2024-03-29 16:27:01 | ERROR | stderr | contents = read_file_cached(tiktoken_bpe_file, expected_hash)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 64, in read_file_cached
2024-03-29 16:27:01 | ERROR | stderr | contents = read_file(blobpath)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 25, in read_file
2024-03-29 16:27:01 | ERROR | stderr | resp = requests.get(blobpath)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/api.py", line 73, in get
2024-03-29 16:27:01 | ERROR | stderr | return request("get", url, params=params, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/api.py", line 59, in request
2024-03-29 16:27:01 | ERROR | stderr | return session.request(method=method, url=url, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
2024-03-29 16:27:01 | ERROR | stderr | resp = self.send(prep, **send_kwargs)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
2024-03-29 16:27:01 | ERROR | stderr | r = adapter.send(request, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
2024-03-29 16:27:01 | ERROR | stderr | raise ConnectionError(e, request=request)
2024-03-29 16:27:01 | ERROR | stderr | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NameResolutionError(": Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)"))

```

Start at fastchat/serve/openai_api_server.py:717 and process_input at :245, then reproduce the POST /v1/embeddings request using the listed model and inputs. Done means the failure is characterized and the issue identifies a repository change or documentation need, with a regression check if applicable.

Written by the indexing model from the issue text.