antirez / antirez/ds4

Segfault (exit 139) when --imatrix-out is combined with --role coordinator

Ouverte Adaptée aux débutants
#697 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
C
Étoiles
22.4k
Forks
2.1k
Merge moyen
2 j 13 h
PR mergées (30 j)
5

Description

Collecting a routed-MoE imatrix over the distributed path crashes with SIGSEGV.

```sh
# worker (giro)
./ds4 --metal -m --role worker --layers 21:output --ctx 32768 \
--coordinator 172.16.0.11 1234

# coordinator (llama)
./ds4 --metal -m --role coordinator --layers 0:20 \
--listen 172.16.0.11 1234 --ctx 32768 \
--imatrix-dataset gguf-tools/imatrix/dataset/rendered_prompts.txt \
--imatrix-out /tmp/imx_probe.dat --imatrix-max-tokens 20000
```

Coordinator output ends at:

```
ds4: restricting metal model map to layers 0:20+output (46 spans, 71.70 GiB tensor span)
ds4: memory: KV 0.61 GiB + buffers 0.25 GiB + resident model 71.70 GiB = 72.56 GiB planned
ds4: collecting routed-MoE imatrix from gguf-tools/imatrix/dataset/rendered_prompts.txt (model=DeepSeek V4 Flash, layers=43, experts=256, ctx=32768, chunk=4096)
```

then exits **139** (SIGSEGV). No imatrix file is written. Both halves load fine and the worker registers normally; the crash happens once collection starts.

Environment: 2× M4 Max 128 GB over Thunderbolt, Metal, `6747e77`, Flash 0731 MXFP4 GGUF, split `0:20` / `21:output`.

`ds4_distributed.c` has no `imatrix` references, so I assume collection was simply never wired for the distributed path — that is entirely reasonable. The report is only that it **crashes instead of refusing**, which is easy to mistake for a hardware or OOM problem. A check like the existing `--dist-activation-bits requires --role coordinator` guard would make it obvious.

Workaround for anyone hitting this: collect solo. On a 128 GB machine the 145 GB MXFP4 quant needs `--ssd-streaming`, which runs at ~137 t/s, so 1.5M calibration tokens takes roughly 3 hours versus ~50 minutes had the split worked. Not urgent, but imatrix collection is a prefill-only workload and therefore close to the ideal case for pipeline parallelism — the prefill speedup measured on this pair is 1.6-1.7x.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Start with ds4_distributed.c and locate the existing --dist-activation-bits/--role coordinator validation, then trace the imatrix collection entry point and argument handling. Reproduce the supplied coordinator/worker commands; done means the unsupported --imatrix-out combination is rejected clearly instead of reaching SIGSEGV, while solo collection remains unaffected.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
c
Domaine
distributed-systems
Type d'issue
Bug
Difficulté
2/5
Temps estimé
1-3 heures
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
72/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.