antirez / antirez/ds4

Distributed pipeline crash / layer routing limitation: coordinator crashes without output layer, but --layers cannot express non-contiguous spans

Aperta
#610 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
22.3k
Fork
2.1k
Merge medio
1g 3h
PR unite (30g)
4

Descrizione

Environment

ds4 commit: efdadd41e20134af4f3381e1ed90e96fe4faef6f (and also tested after fetch showing newer 0a7ad77, please confirm if fix is expected there)
Platform: Linux
GPU: NVIDIA GB10 (sm_121)
2 nodes (DGX1 + DGX2), testing distributed pipeline
Backend: CUDA
What I’m trying to do Split model across two machines:

coordinator on DGX1 for early layers
worker on DGX2 for later layers
Example intent:

coordinator: 0:20
worker: 21:output
Observed behavior

Coordinator with early-layer span crashes at startup:
bash
/home/administrator/ds4/bin/ds4-server \
--role coordinator \
--layers 0:20 \
--listen 0.0.0.0 9000 \
--host 0.0.0.0 \
--port 8000 \
--ctx 8192
Logs end with:

model map restricted to 0:20
memory planning completes
Segmentation fault (core dumped)
If trying to include output with non-contiguous syntax, parser rejects it:
bash
--layers 0:20,output
Error:

Text
invalid --layers 0:20,output: invalid end layer in 0:20,output
Worker rejects coordinator-only prefill flags (expected, just noting):
Text
--dist-prefill-chunk requires --role coordinator
When testing API during broken routing, sometimes get:
JSON
{"error":{"message":"coordinator route does not start at layer 0","type":"invalid_request_error"}}
Why this seems problematic If coordinator runtime effectively requires owning output/final stage in this configuration, but --layers only accepts contiguous spans and cannot express 0:K + output, then a valid split topology may be impossible to express via CLI.

Expected behavior At least one of:

Coordinator should work with contiguous 0:K while worker handles K+1:output (no segfault), or
CLI should support disjoint layer assignment (e.g. 0:20,output) if coordinator needs output stage, or
A clear documented constraint and recommended valid topology for multi-node split on CUDA/GB10.
Request

Please confirm whether this is a bug or intended limitation.
If bug: any patch/workaround?
If limitation: can disjoint --layers support be added, or can coordinator ownership rules be relaxed?
Additional notes

Standalone single-node tensor-parallel mode works on both nodes.
Problem appears specific to distributed coordinator path.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.