SSD streaming: 34% faster at identical settings on DeepSeek V4 Flash (M4 Max 36GB), output byte identical

I run DeepSeek V4 Flash on a MacBook Pro M4 Max with 36 GB RAM using --ssd-streaming, and I spent the last two weeks making it faster without touching quality. At identical settings (same cache size, same flags, code changes only) I get 4.29 t/s where stock gets 3.20, so +34% from the code alone. With better cache settings on top I am at 4.62 t/s, coming from 2.2 to 2.5 with stock defaults. The output is byte identical to stock. I checked the md5 of the generated text on every deterministic benchmark run, around 40 of them. The changes only affect when and how bytes are read from disk, quality is untouched.

The changes, all behind env switches:

1. Router lookahead prefetch. The SSD sits idle about 30% of decode because reads for layer L+1 cannot start before its router runs. A small thread runs layer L+1's router on the hidden state that layer L's router just consumed. The top 4 guesses cover 46% of the next layer's misses, with 0.4 wasted reads per useful one. Prefetch only reads when the disk has free time, demand reads always come first, and guesses expire once decode passes their layer. Average expert load time dropped 5.3% and the miss counters stay bit identical, staging only changes where the bytes come from.

2. Certain prefetch for the hash layers. Layers 0 to 2 route on token id, so their experts are known the moment a token is sampled. Layers 1 and 2 get queued at sampling time. Layer 0 is skipped, its demand read starts about 1 ms later anyway and queueing it just got in the way.

3. Eviction protect. The same forecast stamps the experts expected next, and the eviction scan refuses stamped entries while the stamp is fresh. Zero extra I/O. Misses went from 19,395 to 19,320 and speed from 4.54 to 4.62 t/s on my benchmark.

4. A settable wire reserve (DS4_METAL_WIRE_RESERVE_GIB). Stock reserves 12 GiB for everything that is not expert cache. On my machine 10 is enough, and the freed 2 GiB hold 160 more cache slots. Misses dropped 6.8%.

5. A packed expert file. One expert is three tensors about a gigabyte apart in the GGUF, so one miss costs three scattered reads. A python tool (misc/pack_experts.py) copies each expert's slices into a second file where they sit next to each other, and the engine loads an expert with one contiguous read. Expert load time went from 5.68 to 5.01 ms. The engine finds the file automatically when it sits next to the model. So nobody has to store 137 GB twice, the tool by default frees the expert space inside the GGUF, but only after verifying every byte landed in the packed file, and it is reversible with --restore. People who use the same GGUF with other software can pass --keep-gguf-experts and keep both complete. If the engine meets a freed GGUF without its packed file it stops with a clear error instead of computing on zeroed weights.

Everything is on my fork with the full numbers, the measurement method and the things that did not work: https://github.com/lukeckprobierts/ds4 (SPEEDUP.md).

Would you take some of this upstream? If yes I would split it into small PRs, first the prefetch plus eviction protect, then the wire reserve setting, then the packed expert file with its tool. If you want it cut differently just say so.