json.decoder.JSONDecodeError: Invalid \uXXXX escape: line 1 column 1442 (char 1441)

The worker and controller are located on different servers within the intranet. During the model generation process, the controller reported an error of 'JSON. decoder. JSONDecodeError: Invalid uXXXX escape: line 1 column 1440 (char 1439)'
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 428, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
return await self.app(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/fastapi/applications.py", line 270, in __call__
await super().__call__(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
await self.middleware_stack(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
await self.app(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
raise exc
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/routing.py", line 706, in __call__
await route.handle(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/routing.py", line 69, in app
await response(scope, receive, send)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/responses.py", line 266, in __call__
async with anyio.create_task_group() as task_group:
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
raise exceptions[0]
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/responses.py", line 269, in wrap
await func()
File "/root/anaconda3/envs/aigpu310/lib/python3.10/site-packages/starlette/responses.py", line 258, in stream_response
async for chunk in self.body_iterator:
File "/data/model/FastChat/fastchat/serve/openai_api_server.py", line 424, in chat_completion_stream_generator
async for content in generate_completion_stream(gen_params):
File "/data/model/FastChat/fastchat/serve/openai_api_server.py", line 583, in generate_completion_stream
data = json.loads(chunk.decode())
File "/root/anaconda3/envs/aigpu310/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/root/anaconda3/envs/aigpu310/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/root/anaconda3/envs/aigpu310/lib/python3.10/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid \uXXXX escape: line 1 column 1442 (char 1441)
2023-08-08 21:23:04 | INFO | controller | Receive heart beat. http://192.168.1.32:31004

The latest version has been upgraded and the environment has been reinstalled

Start in fastchat/serve/openai_api_server.py at generate_completion_stream, where the traceback shows json.loads(chunk.decode()) failing. Reproduce the controller/worker setup described in the report and trace the streamed chunk that contains the invalid escape. Done means the reported distributed setup no longer raises this JSONDecodeError and valid streaming responses continue to work.

Written by the indexing model from the issue text.