DSpark preserved-MXFP4 planner support and mixed IQ2/MXFP4 Metal tests
- 主要言語
- C
- スター
- 22.4k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
> **Disclosure:** This issue was prepared and filed by **Codex (OpenAI)** on behalf of `@apetersson` while attempting to build a 128 GB-optimized, selectively native-MXFP4 DeepSeek V4 Flash quant on the experimental `ds4f-mxfp4` branch. The audited revision is `4893e0c40fba03dbc85555faeb035799aa04e0b6`. Please independently review the analysis and proposed patch.
## Summary
The branch can plan and generate preserved native MXFP4 routed experts for the main GGUF, but the equivalent DSpark support plan rejects `--routed-w2 mxfp4` before generation:
```text
loaded imatrix ...: 138 entries
error: unsupported DSpark planned tensor type
```
This blocks a mixed DSpark support model with IQ2_XXS gate/up and losslessly preserved native MXFP4 down experts. The generator and current Metal runtime appear to have the necessary primitives, but the intended mixed 1–5-token DSpark path also lacks targeted coverage.
This is related to, but narrower than, #641: that issue describes type-39 runtime support on older branches. This report is about the new branch's DSpark converter/planner and mixed-path tests.
## Reproduction
Using a native DeepSeek V4 Flash checkpoint whose routed experts are packed I8 weights plus F8_E8M0 scales:
```sh
./gguf-tools/deepseek4-quantize \
--hf \
--template \
--dspark-support \
--experts iq2_xxs \
--routed-w2 mxfp4 \
--imatrix \
--imatrix-strict \
--dspark-block-size 5 \
--dspark-target-layers 40,41,42 \
--out /tmp/dspark-mxfp4.gguf \
--dry-run
```
Result:
```text
error: unsupported DSpark planned tensor type
```
## Apparent cause
The main planner has a deliberately narrow preserved-expert exception:
```c
const bool preserved_mxfp4 =
type == DS4Q_TYPE_MXFP4 && parse_expert_tensor(src->name).is_expert;
```
`dspark_plan_set_size()` instead rejects any type that is not I32 or an ordinary quantizable target:
```c
if (tp->meta.type != DS4Q_TYPE_I32 && !is_quantizable_target(tp->meta.type)) {
die("unsupported DSpark planned tensor type");
}
```
MXFP4 is intentionally not an ordinary requantization target, so making it generically quantizable would be the wrong fix. The exception should remain constrained to compatible packed routed experts.
The DSpark generator already recognizes `mtp.*` expert tensors and reaches the common native MXFP4 repacker after planning.
## Proposed converter change
Mirror the main planner's preserved-MXFP4 exception in the DSpark planner, with all of the following constraints:
- The tensor is a DSpark routed expert tensor.
- The source weight is packed I8 and the paired scale is F8_E8M0.
- Shape and scale shape match the native repacker contract.
- The logical first dimension is MXFP4-block aligned.
- MXFP4 remains rejected for ordinary/non-expert tensors.
## Missing Metal test coverage
The planned support layout is:
- Gate: IQ2_XXS
- Up: IQ2_XXS
- Down: MXFP4
- DSpark proposal block: up to 5 tokens
Please add a CPU/reference versus Metal test covering token counts `1, 2, 3, 4, 5`, including:
- The fused tiny gate/up path.
- MXFP4 down plus fused expert sum for 1–4 rows.
- The generic MXFP4 MV plus expert-summation path used by the 5-row block.
- A real support-binding/verifier smoke test if a small fixture can cover it.
The current all-MXFP4 Metal test is useful but does not exercise this mixed DSpark layout.
## Why this matters for the 128 GB profile
For the three DSpark down aggregates:
- Q4_K sidecar: approximately `6.9840 GiB`
- Native-MXFP4-down sidecar: approximately `6.7965 GiB`
- Saving: exactly `0.1875 GiB`
The larger benefit is fidelity: the three down tensors retain the checkpoint's exact FP4 codes and scale bytes instead of being requantized. The main model remains authoritative, so this primarily affects proposal fidelity, acceptance and speed rather than final target-model correctness.
## Acceptance criteria
- Strict dry-run succeeds with `--dspark-support --routed-w2 mxfp4`.
- Exactly the three intended DSpark down aggregate tensors are MXFP4.
- Non-expert MXFP4 requests still fail.
- Native codes and scale bytes are checked during generation.
- Mixed IQ2/IQ2/MXFP4 Metal tests pass for 1–5 tokens on Apple Silicon.
- A real DSpark verifier smoke test completes without binding, bounds or correctness errors.
コントリビューションガイド
調査の方向性
Start at dspark_plan_set_size() and the existing main-planner preserved_mxfp4 exception; compare the DSpark generator's mtp.* handling and the current all-MXFP4 Metal test. Reproduce the strict dry-run, then exercise CPU/reference versus Metal for 1–5 tokens, checking the three down aggregates' native codes and scale bytes and the non-expert rejection. Finish with the DSpark verifier smoke test if the fixture supports it.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- c
- 領域
- machine-learning, testing-qa
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 静か
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 45/100