ds4-server does not support --batched-session > 1
- Ngôn ngữ chính
- C
- Star
- 22.3k
- Fork
- 2.1k
- Merge trung bình
- 1 ngày 3 giờ
- Pull request đã merge (30 ngày)
- 4
Mô tả
Hi,
I was trying to get the "ds4-server" to work on a distributed cluster to server multiple users (3+).
When I try to set "--batched-session" to ANY value greater than 1, I get this error (for example, when setting `--batched-session 2`):
```
ds4: failed to create distributed coordinator session: enable to listen on 10.0.0.1:1234: Address already in use
0812 00:44:42 ds4-server: failed to create metal session 2/2
```
-
This is what DeepSeek v4 Flash 0731 said of the error:
The coordinator's worker-listener is opened once per resident session, not once per process. In distributed coordinator mode each session you create via --batched-session N independently tries to bind the same listen address.
Walk through the failure:
1. ds4-server with --batched-session N sets slot_count = N and creates N resident sessions (ds4_server.c:13368).
2. Each ds4_session_create() in coordinator mode calls ds4_dist_session_create() (ds4.c:58610 / ds4.c:58769).
3. ds4_dist_session_create() unconditionally calls dist_open_listener(opt->listen_host, opt->listen_port, ...) (ds4_distributed.c:5409), which bind()s 10.0.0.1:1234 (ds4_distributed.c:1220).
4. Session 1 binds 10.0.0.1:1234 successfully. Session 2 tries the same bind() → EADDRINUSE → ds4_dist_session_create returns 1 → "failed to create distributed coordinator session: ... Address already in
use" → the server bails on 2/2.
So --batched-session N with N > 1 is incompatible with --role coordinator as written. There's no guard rejecting the combination — the server happily enables batched mode and the coordinator role, then dies
on the second session. (Note --batched-session is a server-level field while the distributed role lives in engine.distributed, so ds4_dist_prepare_engine_options can't see it and can't reject it.)
Immediate workaround
Run the coordinator without --batched-session (or with --batched-session 1, the default). The single-session coordinator binds the listener once and works — but it serves one request at a time, so it doesn't
give you the multi-user concurrency you're after.
What would actually fix it
True multi-user concurrency in distributed mode needs a code change: the coordinator's worker listener (and its accept loop) should be owned once per coordinator process and shared by all N resident
ds4_dist_sessions, instead of each session opening its own bind() on the same host:port. The N sessions still get their own route plan / request streams; they just share the single socket where workers
connect. That means touching ds4_dist_session_create() (ds4_distributed.c:5402) and the session-ownership model so the first session opens the listener and the rest reuse it.
This is a real feature/bug fix, not a flag change.
Hope this helps, but from this and the "tensor parallelism" not working with ds4-server, the ds4-server is really starting to fall behind the cli version it seems.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.