[Question] httpx.ReadError when R3 enabled

Your Question

When training glm-4.5-air, the RolloutManager crashes with httpx.ReadError during rollout. It works fine for the first several rollout steps, then fails intermittently. The error originates from router.py:142 where the proxy forwards requests to the upstream inference server.

(RolloutManager pid=2075311) ERROR: Exception in ASGI application 14:00:31 [91/1920]
(RolloutManager pid=2075311) Traceback (most recent call last):
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
(RolloutManager pid=2075311) yield
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_transports/default.py", line 394, in handle_async_request
(RolloutManager pid=2075311) resp = await self._pool.handle_async_request(req)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
(RolloutManager pid=2075311) raise exc from None
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
(RolloutManager pid=2075311) response = await connection.handle_async_request(
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/connection.py", line 103, in handle_async_request
(RolloutManager pid=2075311) return await self._connection.handle_async_request(request)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/http11.py", line 136, in handle_async_request
(RolloutManager pid=2075311) raise exc
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/http11.py", line 106, in handle_async_request
(RolloutManager pid=2075311) ) = await self._receive_response_headers(**kwargs)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers
(RolloutManager pid=2075311) event = await self._receive_event(timeout=timeout)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_async/http11.py", line 217, in _receive_event
(RolloutManager pid=2075311) data = await self._network_stream.read(
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_backends/anyio.py", line 32, in read
(RolloutManager pid=2075311) with map_exceptions(exc_map):
(RolloutManager pid=2075311) File "/usr/lib/python3.12/contextlib.py", line 158, in exit
(RolloutManager pid=2075311) self.gen.throw(value)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpcore/_exceptions.py", line 14, in map_exceptions
(RolloutManager pid=2075311) raise to_exc(exc) from exc
(RolloutManager pid=2075311) httpcore.ReadError
(RolloutManager pid=2075311)
(RolloutManager pid=2075311) The above exception was the direct cause of the following exception:
(RolloutManager pid=2075311)
(RolloutManager pid=2075311) Traceback (most recent call last):
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/uvicorn/protocols/http/h11_impl.py", line 410, in run_asgi
(RolloutManager pid=2075311) result = await app( # type: ignore[func-returns-value]
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
(RolloutManager pid=2075311) return await self.app(scope, receive, send)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/applications.py", line 1134, in call
(RolloutManager pid=2075311) await super().call(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/applications.py", line 107, in call
(RolloutManager pid=2075311) await self.middleware_stack(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 186, in call
(RolloutManager pid=2075311) raise exc
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 164, in call
(RolloutManager pid=2075311) await self.app(scope, receive, _send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/exceptions.py", line 63, in call
(RolloutManager pid=2075311) await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
(RolloutManager pid=2075311) raise exc
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
(RolloutManager pid=2075311) await app(scope, receive, sender)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 716, in call
(RolloutManager pid=2075311) await self.middleware_stack(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 736, in app
(RolloutManager pid=2075311) await route.handle(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 290, in handle
(RolloutManager pid=2075311) await self.app(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 119, in app
(RolloutManager pid=2075311) await wrap_app_handling_exceptions(app, request)(scope, receive, send)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
(RolloutManager pid=2075311) raise exc
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
(RolloutManager pid=2075311) await app(scope, receive, sender)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 105, in app
(RolloutManager pid=2075311) response = await f(request)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 425, in app
(RolloutManager pid=2075311) raw_response = await run_endpoint_function(
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 313, in run_endpoint_function
(RolloutManager pid=2075311) return await dependant.call(**values)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/slime/slime/router/router.py", line 142, in proxy
(RolloutManager pid=2075311) response = await self.client.request(request.method, url, content=body, headers=headers)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_client.py", line 1540, in request
(RolloutManager pid=2075311) return await self.send(request, auth=auth, follow_redirects=follow_redirects)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_client.py", line 1629, in send
(RolloutManager pid=2075311) response = await self._send_handling_auth(
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_client.py", line 1657, in _send_handling_auth
(RolloutManager pid=2075311) response = await self._send_handling_redirects(
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_client.py", line 1694, in _send_handling_redirects
(RolloutManager pid=2075311) response = await self._send_single_request(request)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_client.py", line 1730, in _send_single_request
(RolloutManager pid=2075311) response = await transport.handle_async_request(request)
(RolloutManager pid=2075311) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_transports/default.py", line 393, in handle_async_request
(RolloutManager pid=2075311) with map_httpcore_exceptions():
(RolloutManager pid=2075311) File "/usr/lib/python3.12/contextlib.py", line 158, in exit
(RolloutManager pid=2075311) self.gen.throw(value)
(RolloutManager pid=2075311) File "/usr/local/lib/python3.12/dist-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
(RolloutManager pid=2075311) raise mapped_exc(message) from exc
(RolloutManager pid=2075311) httpx.ReadError

What I've Tried

Searched existing issues

Environment (if relevant)

• slime version: commit a0ac45a
• docker version: latest

Additional Context

SGLANG_ARGS=(
--rollout-num-gpus-per-engine 8
--sglang-mem-fraction-static 0.6
--sglang-enable-dp-attention
--sglang-dp-size 8
--sglang-ep-size 8
--sglang-enable-dp-lm-head
--sglang-moe-dense-tp-size 1

mtp

--sglang-speculative-algorithm EAGLE
--sglang-speculative-num-steps 2
--sglang-speculative-eagle-topk 1
--sglang-speculative-num-draft-tokens 3

)

CUSTOM_ARGS=(
--use-rollout-routing-replay
--use-opsm
--use-slime-router
)

Pre-submission Checklist

• I have read the CONTRIBUTING.md and understand the collaboration scope.
• I have read the documentation and FAQ and my question is not answered there.
• I have searched for existing issues and my question has not been asked before.

Start with slime/router/router.py at the proxy entry point on line 142 and reproduce training with the listed SGLANG_ARGS and CUSTOM_ARGS. Trace the httpx/ASGI request path and upstream inference-server behavior around the intermittent ReadError; done means the cause is identified and the rollout no longer crashes.