antirez / antirez/ds4

Segfault (exit 139) when --imatrix-out is combined with --role coordinator

未关闭 适合新手
#697 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.4k
派生
2.1k
平均合并
2 天 13 小时
30 天内合并 PR
5

描述

Collecting a routed-MoE imatrix over the distributed path crashes with SIGSEGV.

```sh
# worker (giro)
./ds4 --metal -m --role worker --layers 21:output --ctx 32768 \
--coordinator 172.16.0.11 1234

# coordinator (llama)
./ds4 --metal -m --role coordinator --layers 0:20 \
--listen 172.16.0.11 1234 --ctx 32768 \
--imatrix-dataset gguf-tools/imatrix/dataset/rendered_prompts.txt \
--imatrix-out /tmp/imx_probe.dat --imatrix-max-tokens 20000
```

Coordinator output ends at:

```
ds4: restricting metal model map to layers 0:20+output (46 spans, 71.70 GiB tensor span)
ds4: memory: KV 0.61 GiB + buffers 0.25 GiB + resident model 71.70 GiB = 72.56 GiB planned
ds4: collecting routed-MoE imatrix from gguf-tools/imatrix/dataset/rendered_prompts.txt (model=DeepSeek V4 Flash, layers=43, experts=256, ctx=32768, chunk=4096)
```

then exits **139** (SIGSEGV). No imatrix file is written. Both halves load fine and the worker registers normally; the crash happens once collection starts.

Environment: 2× M4 Max 128 GB over Thunderbolt, Metal, `6747e77`, Flash 0731 MXFP4 GGUF, split `0:20` / `21:output`.

`ds4_distributed.c` has no `imatrix` references, so I assume collection was simply never wired for the distributed path — that is entirely reasonable. The report is only that it **crashes instead of refusing**, which is easy to mistake for a hardware or OOM problem. A check like the existing `--dist-activation-bits requires --role coordinator` guard would make it obvious.

Workaround for anyone hitting this: collect solo. On a 128 GB machine the 145 GB MXFP4 quant needs `--ssd-streaming`, which runs at ~137 t/s, so 1.5M calibration tokens takes roughly 3 hours versus ~50 minutes had the split worked. Not urgent, but imatrix collection is a prefill-only workload and therefore close to the ideal case for pipeline parallelism — the prefill speedup measured on this pair is 1.6-1.7x.

贡献指南

打开贡献指南

调研方向

Start with ds4_distributed.c and locate the existing --dist-activation-bits/--role coordinator validation, then trace the imatrix collection entry point and argument handling. Reproduce the supplied coordinator/worker commands; done means the unsupported --imatrix-out combination is rejected clearly instead of reaching SIGSEGV, while solo collection remains unaffected.

由索引模型根据 Issue 内容生成。

评估

技术栈
c
领域
distributed-systems
Issue 类型
缺陷
难度
2/5
预计耗时
1-3 小时
活跃度
冷清
描述清晰度
基本清楚
新手友好度
72/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。