antirez / antirez/ds4

Distributed pipeline crash / layer routing limitation: coordinator crashes without output layer, but --layers cannot express non-contiguous spans

Offen
#610 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
22.3k
Forks
2.1k
Ø Merge
1 T. 3 Std.
Gemergte PRs (30 T.)
4

Beschreibung

Environment

ds4 commit: efdadd41e20134af4f3381e1ed90e96fe4faef6f (and also tested after fetch showing newer 0a7ad77, please confirm if fix is expected there)
Platform: Linux
GPU: NVIDIA GB10 (sm_121)
2 nodes (DGX1 + DGX2), testing distributed pipeline
Backend: CUDA
What I’m trying to do Split model across two machines:

coordinator on DGX1 for early layers
worker on DGX2 for later layers
Example intent:

coordinator: 0:20
worker: 21:output
Observed behavior

Coordinator with early-layer span crashes at startup:
bash
/home/administrator/ds4/bin/ds4-server \
--role coordinator \
--layers 0:20 \
--listen 0.0.0.0 9000 \
--host 0.0.0.0 \
--port 8000 \
--ctx 8192
Logs end with:

model map restricted to 0:20
memory planning completes
Segmentation fault (core dumped)
If trying to include output with non-contiguous syntax, parser rejects it:
bash
--layers 0:20,output
Error:

Text
invalid --layers 0:20,output: invalid end layer in 0:20,output
Worker rejects coordinator-only prefill flags (expected, just noting):
Text
--dist-prefill-chunk requires --role coordinator
When testing API during broken routing, sometimes get:
JSON
{"error":{"message":"coordinator route does not start at layer 0","type":"invalid_request_error"}}
Why this seems problematic If coordinator runtime effectively requires owning output/final stage in this configuration, but --layers only accepts contiguous spans and cannot express 0:K + output, then a valid split topology may be impossible to express via CLI.

Expected behavior At least one of:

Coordinator should work with contiguous 0:K while worker handles K+1:output (no segfault), or
CLI should support disjoint layer assignment (e.g. 0:20,output) if coordinator needs output stage, or
A clear documented constraint and recommended valid topology for multi-node split on CUDA/GB10.
Request

Please confirm whether this is a bug or intended limitation.
If bug: any patch/workaround?
If limitation: can disjoint --layers support be added, or can coordinator ownership rules be relaxed?
Additional notes

Standalone single-node tensor-parallel mode works on both nodes.
Problem appears specific to distributed coordinator path.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.