M3 Ultra 512 Testing results with DSv4F 0731

# Context scaling: ds4-bench 2K–64K, DeepSeek-V4-Flash Q4K on M3 Ultra

Results from a `ds4-bench` context sweep, 2048 → 65536 in 2048-token steps, 128 generated tokens per frontier.

## Environment

| | |
|---|---|
| Device | Apple M3 Ultra, 512 GiB RAM |
| Backend | Metal (Metal 4 tensor API disabled — pre-M5/pre-A19) |
| Model | `DeepSeek-V4-Flash-Q4KExperts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-imatrix-0731.gguf` |
| Prompt | `speed-bench/promessi_sposi.txt` |
| Drift-patch flags | `hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=off` |

## Command

```
./ds4-bench \
-m ds4flash.gguf \
--prompt-file speed-bench/promessi_sposi.txt \
--ctx-start 2048 \
--ctx-max 65536 \
--step-incr 2048 \
--gen-tokens 128
```

Startup log:

```
ds4: Metal device Apple M3 Ultra, 512.00 GiB RAM
ds4: Metal 4 tensor API disabled for pre-M5/pre-A19 devices
ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=off
ds4: requesting Metal residency (may take tens of seconds)... done
ds4: warming Metal model views... done
ds4: Metal model views created in 4.213 ms, residency requested in 987.535 ms, warmup 7.258 ms (mapped 157001.67 MiB from offset 5.09 MiB)
ds4: Metal mapped mmaped model as 1 overlapping shared buffers
ds4: metal backend initialized for graph diagnostics
ds4: memory: KV 0.86 GiB (raw 0.36 + compressed 0.50) + buffers 0.50 GiB + resident model 153.32 GiB = 154.68 GiB planned
ds4: memory detail: ctx=65665 prefill_cap=4096 raw_kv_rows=4352 compressed_kv_rows=16418 backend=metal
ds4-bench: context buffers 1395.67 MiB (ctx=65665, backend=metal, prefill_chunk=4096, raw_kv_rows=4352, compressed_kv_rows=16418)
```

`--ctx-alloc` was not passed, so it defaulted to `ctx_max + gen_tokens + 1` = 65665, which is the allocated context in the log lines above.

## Summary

| Metric | @ 2048 | @ 65536 | Change |
|---|---|---|---|
| `prefill_tps` | 592.76 tok/s | 399.78 tok/s | −32.6% |
| `gen_steady_tps` | 36.12 tok/s | 29.57 tok/s | −18.1% |
| `gen_first_ms` | 32.815 ms | 40.460 ms | +23.3% |

Reading notes, per the benchmark source:

- `prefill_tokens` is `frontier - previous`, so every row after the first measures only the newest 2048-token interval. The declining `prefill_tps` is therefore per-chunk cost against a growing cache, not an amortized whole-context average. The first row is the exception: `previous` is 0, so its 2048 tokens are the entire context.
- `gen_tps` uses `gen_t0`/`gen_t1`, which wrap the full decode loop including the `ds4_session_argmax_excluding` calls; `gen_steady_tps` sums only the `ds4_session_eval` intervals and excludes the first token. `gen_tps` is consequently below `gen_steady_tps` in every row.
- Snapshot save/restore sits outside both timing windows, so the prefill and generation numbers are unaffected by which restore path a row took.
## Full results

| ctx_tokens | prefill_tokens | prefill_tps | gen_tokens | gen_tps | gen_first_ms | gen_steady_tokens | gen_steady_tps | kvcache_bytes |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 2048 | 2048 | 592.76 | 128 | 35.48 | 32.815 | 127 | 36.12 | 52184460 |
| 4096 | 2048 | 540.18 | 128 | 32.86 | 30.628 | 127 | 33.37 | 80373132 |
| 6144 | 2048 | 534.71 | 128 | 32.66 | 32.823 | 127 | 33.18 | 108561804 |
| 8192 | 2048 | 527.46 | 128 | 32.51 | 33.048 | 127 | 33.02 | 136750476 |
| 10240 | 2048 | 517.65 | 128 | 32.25 | 33.063 | 127 | 32.75 | 164939148 |
| 12288 | 2048 | 514.92 | 128 | 32.21 | 32.368 | 127 | 32.70 | 193127820 |
| 14336 | 2048 | 510.16 | 128 | 31.98 | 34.284 | 127 | 32.49 | 221316492 |
| 16384 | 2048 | 505.19 | 128 | 31.71 | 33.500 | 127 | 32.20 | 249505164 |
| 18432 | 2048 | 497.99 | 128 | 31.56 | 33.773 | 127 | 32.04 | 277693836 |
| 20480 | 2048 | 493.71 | 128 | 31.47 | 33.882 | 127 | 31.96 | 305882508 |
| 22528 | 2048 | 488.93 | 128 | 31.43 | 33.997 | 127 | 31.92 | 334071180 |
| 24576 | 2048 | 483.65 | 128 | 31.33 | 34.405 | 127 | 31.84 | 362259852 |
| 26624 | 2048 | 475.78 | 128 | 31.26 | 34.158 | 127 | 31.76 | 390448524 |
| 28672 | 2048 | 471.54 | 128 | 31.14 | 34.955 | 127 | 31.65 | 418637196 |
| 30720 | 2048 | 467.02 | 128 | 31.02 | 34.452 | 127 | 31.51 | 446825868 |
| 32768 | 2048 | 463.94 | 128 | 30.76 | 35.035 | 127 | 31.25 | 475014540 |
| 34816 | 2048 | 456.82 | 128 | 30.56 | 36.350 | 127 | 31.06 | 503203212 |
| 36864 | 2048 | 454.30 | 128 | 30.55 | 34.937 | 127 | 31.03 | 531391884 |
| 38912 | 2048 | 450.15 | 128 | 30.42 | 35.080 | 127 | 30.89 | 559580556 |
| 40960 | 2048 | 445.75 | 128 | 30.39 | 35.634 | 127 | 30.88 | 587769228 |
| 43008 | 2048 | 439.64 | 128 | 30.24 | 35.581 | 127 | 30.72 | 615957900 |
| 45056 | 2048 | 435.39 | 128 | 30.22 | 35.651 | 127 | 30.69 | 644146572 |
| 47104 | 2048 | 432.49 | 128 | 30.04 | 36.135 | 127 | 30.51 | 672335244 |
| 49152 | 2048 | 429.00 | 128 | 30.05 | 36.174 | 127 | 30.52 | 700523916 |
| 51200 | 2048 | 423.73 | 128 | 29.97 | 35.557 | 127 | 30.44 | 728712588 |
| 53248 | 2048 | 420.93 | 128 | 29.90 | 36.075 | 127 | 30.37 | 756901260 |
| 55296 | 2048 | 417.28 | 128 | 29.75 | 36.359 | 127 | 30.21 | 785089932 |
| 57344 | 2048 | 413.90 | 128 | 29.78 | 36.285 | 127 | 30.24 | 813278604 |
| 59392 | 2048 | 408.39 | 128 | 29.59 | 36.667 | 127 | 30.05 | 841467276 |
| 61440 | 2048 | 406.03 | 128 | 29.53 | 36.772 | 127 | 29.99 | 869655948 |
| 63488 | 2048 | 402.92 | 128 | 29.44 | 36.904 | 127 | 29.89 | 897844620 |
| 65536 | 2048 | 399.78 | 128 | 29.11 | 40.460 | 127 | 29.57 | 0 |

Raw CSV:

```csv
ctx_tokens,prefill_tokens,prefill_tps,gen_tokens,gen_tps,gen_first_ms,gen_steady_tokens,gen_steady_tps,kvcache_bytes
2048,2048,592.76,128,35.48,32.815,127,36.12,52184460
4096,2048,540.18,128,32.86,30.628,127,33.37,80373132
6144,2048,534.71,128,32.66,32.823,127,33.18,108561804
8192,2048,527.46,128,32.51,33.048,127,33.02,136750476
10240,2048,517.65,128,32.25,33.063,127,32.75,164939148
12288,2048,514.92,128,32.21,32.368,127,32.70,193127820
14336,2048,510.16,128,31.98,34.284,127,32.49,221316492
16384,2048,505.19,128,31.71,33.500,127,32.20,249505164
18432,2048,497.99,128,31.56,33.773,127,32.04,277693836
20480,2048,493.71,128,31.47,33.882,127,31.96,305882508
22528,2048,488.93,128,31.43,33.997,127,31.92,334071180
24576,2048,483.65,128,31.33,34.405,127,31.84,362259852
26624,2048,475.78,128,31.26,34.158,127,31.76,390448524
28672,2048,471.54,128,31.14,34.955,127,31.65,418637196
30720,2048,467.02,128,31.02,34.452,127,31.51,446825868
32768,2048,463.94,128,30.76,35.035,127,31.25,475014540
34816,2048,456.82,128,30.56,36.350,127,31.06,503203212
36864,2048,454.30,128,30.55,34.937,127,31.03,531391884
38912,2048,450.15,128,30.42,35.080,127,30.89,559580556
40960,2048,445.75,128,30.39,35.634,127,30.88,587769228
43008,2048,439.64,128,30.24,35.581,127,30.72,615957900
45056,2048,435.39,128,30.22,35.651,127,30.69,644146572
47104,2048,432.49,128,30.04,36.135,127,30.51,672335244
49152,2048,429.00,128,30.05,36.174,127,30.52,700523916
51200,2048,423.73,128,29.97,35.557,127,30.44,728712588
53248,2048,420.93,128,29.90,36.075,127,30.37,756901260
55296,2048,417.28,128,29.75,36.359,127,30.21,785089932
57344,2048,413.90,128,29.78,36.285,127,30.24,813278604
59392,2048,408.39,128,29.59,36.667,127,30.05,841467276
61440,2048,406.03,128,29.53,36.772,127,29.99,869655948
63488,2048,402.92,128,29.44,36.904,127,29.89,897844620
65536,2048,399.78,128,29.11,40.460,127,29.57,0
```