lm-sys / lm-sys/FastChat

ERROR: Exception in ASGI application

Open
#2,336 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
No merged PRs in 30d

Description

2023-08-30 09:03:27 | ERROR | stderr | ERROR:    Exception in ASGI application
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/asyncio/selector_events.py", line 848, in _read_ready__data_received
2023-08-30 09:03:27 | ERROR | stderr |     data = self._sock.recv(self.max_size)
2023-08-30 09:03:27 | ERROR | stderr | ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     yield
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 34, in read
2023-08-30 09:03:27 | ERROR | stderr |     return await self._stream.receive(max_bytes=max_bytes)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 1212, in receive
2023-08-30 09:03:27 | ERROR | stderr |     raise self._protocol.exception
2023-08-30 09:03:27 | ERROR | stderr | anyio.BrokenResourceError
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     yield
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     resp = await self._pool.handle_async_request(req)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     response = await connection.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection.py", line 96, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     return await self._connection.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 121, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 99, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     ) = await self._receive_response_headers(**kwargs)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 164, in _receive_response_headers
2023-08-30 09:03:27 | ERROR | stderr |     event = await self._receive_event(timeout=timeout)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 200, in _receive_event
2023-08-30 09:03:27 | ERROR | stderr |     data = await self._network_stream.read(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 36, in read
2023-08-30 09:03:27 | ERROR | stderr |     return b""
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/contextlib.py", line 131, in __exit__
2023-08-30 09:03:27 | ERROR | stderr |     self.gen.throw(type, value, traceback)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     raise to_exc(exc) from exc
2023-08-30 09:03:27 | ERROR | stderr | httpcore.ReadError
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
2023-08-30 09:03:27 | ERROR | stderr |     result = await app(  # type: ignore[func-returns-value]
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     return await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/applications.py", line 289, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await super().__call__(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/applications.py", line 122, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.middleware_stack(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, _send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/cors.py", line 83, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, sender)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise e
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 718, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await route.handle(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 66, in app
2023-08-30 09:03:27 | ERROR | stderr |     response = await func(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/routing.py", line 273, in app
2023-08-30 09:03:27 | ERROR | stderr |     raw_response = await run_endpoint_function(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
2023-08-30 09:03:27 | ERROR | stderr |     return await dependant.call(**values)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastchat/serve/openai_api_server.py", line 483, in create_completion
2023-08-30 09:03:27 | ERROR | stderr |     error_check_ret = await check_length(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastchat/serve/openai_api_server.py", line 129, in check_length
2023-08-30 09:03:27 | ERROR | stderr |     response = await client.post(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1848, in post
2023-08-30 09:03:27 | ERROR | stderr |     return await self.request(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1530, in request
2023-08-30 09:03:27 | ERROR | stderr |     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1617, in send
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_handling_auth(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_handling_redirects(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_single_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1719, in _send_single_request
2023-08-30 09:03:27 | ERROR | stderr |     response = await transport.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     resp = await self._pool.handle_async_request(req)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/contextlib.py", line 131, in __exit__
2023-08-30 09:03:27 | ERROR | stderr |     self.gen.throw(type, value, traceback)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     raise mapped_exc(message) from exc
2023-08-30 09:03:27 | ERROR | stderr | httpx.ReadError
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21001/list_models "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21001/get_worker_address "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | openai_api_server | model_name: 4pd/qwen-7b-chat@main, worker_addr: http://10.233.69.164:21002
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21002/model_details "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21002/count_token "HTTP/1.1 200 OK"

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in fastchat/serve/openai_api_server.py at check_length and create_completion, then trace the HTTP request to the worker shown in the log. Reproduce the connection-reset scenario and inspect the surrounding httpx, httpcore, and anyio behavior. Done should mean the failure is handled or its cause and expected resolution are established, but the issue does not specify the desired behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.