antirez / antirez/ds4

Segfault (exit 139) when --imatrix-out is combined with --role coordinator

Offen Anfängerfreundlich
#697 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
22.4k
Forks
2.1k
Ø Merge
2 T. 13 Std.
Gemergte PRs (30 T.)
5

Beschreibung

Collecting a routed-MoE imatrix over the distributed path crashes with SIGSEGV.

```sh
# worker (giro)
./ds4 --metal -m --role worker --layers 21:output --ctx 32768 \
--coordinator 172.16.0.11 1234

# coordinator (llama)
./ds4 --metal -m --role coordinator --layers 0:20 \
--listen 172.16.0.11 1234 --ctx 32768 \
--imatrix-dataset gguf-tools/imatrix/dataset/rendered_prompts.txt \
--imatrix-out /tmp/imx_probe.dat --imatrix-max-tokens 20000
```

Coordinator output ends at:

```
ds4: restricting metal model map to layers 0:20+output (46 spans, 71.70 GiB tensor span)
ds4: memory: KV 0.61 GiB + buffers 0.25 GiB + resident model 71.70 GiB = 72.56 GiB planned
ds4: collecting routed-MoE imatrix from gguf-tools/imatrix/dataset/rendered_prompts.txt (model=DeepSeek V4 Flash, layers=43, experts=256, ctx=32768, chunk=4096)
```

then exits **139** (SIGSEGV). No imatrix file is written. Both halves load fine and the worker registers normally; the crash happens once collection starts.

Environment: 2× M4 Max 128 GB over Thunderbolt, Metal, `6747e77`, Flash 0731 MXFP4 GGUF, split `0:20` / `21:output`.

`ds4_distributed.c` has no `imatrix` references, so I assume collection was simply never wired for the distributed path — that is entirely reasonable. The report is only that it **crashes instead of refusing**, which is easy to mistake for a hardware or OOM problem. A check like the existing `--dist-activation-bits requires --role coordinator` guard would make it obvious.

Workaround for anyone hitting this: collect solo. On a 128 GB machine the 145 GB MXFP4 quant needs `--ssd-streaming`, which runs at ~137 t/s, so 1.5M calibration tokens takes roughly 3 hours versus ~50 minutes had the split worked. Not urgent, but imatrix collection is a prefill-only workload and therefore close to the ideal case for pipeline parallelism — the prefill speedup measured on this pair is 1.6-1.7x.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start with ds4_distributed.c and locate the existing --dist-activation-bits/--role coordinator validation, then trace the imatrix collection entry point and argument handling. Reproduce the supplied coordinator/worker commands; done means the unsupported --imatrix-out combination is rejected clearly instead of reaching SIGSEGV, while solo collection remains unaffected.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
c
Bereich
distributed-systems
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
72/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.