CUDA --ssd-streaming: illegal memory access at first 2048-token prefill in ds4-bench (RTX 3090, layer-major FFN encode)
- Lingua principale
- C
- Stelle
- 22.3k
- Fork
- 2.1k
- Merge medio
- 1g 3h
- PR unite (30g)
- 4
Descrizione
## Summary
`ds4-bench` with CUDA `--ssd-streaming` crashes with an **illegal memory access at the very first prefill** (2048 tokens), on a plain RTX 3090. The failing stage is the layer-major streaming prefill, FFN encode of layer 2. Reproduced 3/3 runs; while a `ds4-server` on the same build, same box, same model serves short-prompt chat completions for hours without issue (we ran one idle for 4 h during the #836 measurement).
Likely the same family as the mid-prefill poisoning on GB10 (#759 / #847), but this signature is different: consumer sm_86, first prefill of a fresh process, deterministic, and reachable with the project's own bench tool — so it may be the easiest handle on this class of bug so far.
## Environment (per CONTRIBUTING.md)
- RunPod pod, 1× RTX 3090 24 GB (sm_86), driver 580.65.06, 251 GB RAM
- `make cuda CUDA_ARCH=sm_86`, CUDA 12.4 toolkit, commit `c1d4597`
- Model quant: `ds4f-q2` (DeepSeek-V4-Flash-IQ2XXS-...-0731, 81 GB)
## Repro (deterministic, 3/3)
```sh
./ds4-bench -m ./ds4flash.gguf --cuda --ssd-streaming \
--prompt-file speed-bench/promessi_sposi.txt \
--ctx-start 2048 --ctx-max 8192 --step-incr 2048 \
--gen-tokens 64 --csv /tmp/speed.csv
```
Output:
```
ds4: CUDA model range alloc failed for hc_scale (0.00 MiB): an illegal memory access was encountered
ds4: CUDA model range copy failed for hc_base at 0.00 MiB: an illegal memory access was encountered
ds4: CUDA model range alloc failed for hc_base (0.00 MiB): an illegal memory access was encountered
ds4: gpu layer 2 ffn batch encode failed
ds4: gpu layer-major prefill layer 2 encode failed
ds4: CUDA synchronize failed: an illegal memory access was encountered
ds4-bench: prefill to 2048 failed: cuda prefill failed
```
(The `model range alloc failed` lines are the poisoned-context cascade; the first IMA happens earlier in the layer-major prefill.)
## Notes
- Identical behavior on unpatched `c1d4597` and with PR #872 applied — that patch is host-side shape validation in the attention prefill fallback and is unrelated; mentioning it only because we found this crash while trying to run the before/after bench for that PR.
- Without `--ssd-streaming` the model cannot fit a 24 GB card (arena alloc OOM), so on this class of hardware the bench currently has no working path.
- Happy to run diagnostics on a rented pod if a compute-sanitizer trace or an env-var A/B (`DS4_CUDA_NO_TOPK_STREAM`, etc.) would help — CUDA 13.0 compute-sanitizer works on these drivers, the 12.4 one does not start.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.