antirez / antirez/ds4

cuda-generic: --mtp startup fails with cascading "illegal memory access" (RTX 4090 / WSL2)

未关闭
#522 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.3k
派生
2.1k
平均合并
1 天 3 小时
30 天内合并 PR
4

描述

**Summary:** On a generic CUDA build, enabling `--mtp` makes startup fail with a cascade of
`illegal memory access` errors and the agent exits with `ds4-agent: session backend is required`.
The same build, model, and machine work fine without `--mtp`.

## Environment

- GPU: NVIDIA GeForce RTX 4090 (sm_89, 24 GB), Windows driver 610.62
- Host: Windows 11 Pro, 128 GB DDR4; running under **WSL2** (Ubuntu 24.04, kernel `6.6.87.2-microsoft-standard-WSL2`, 98 GB RAM visible)
- CUDA toolkit: 12.6 (`Build cuda_12.6.r12.6/compiler.35059454_0`)
- ds4: commit `80ebbc3` (current main), built with `make cuda-generic`
- Main model: `DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf` (q2-imatrix, via `download_model.sh q2-imatrix`)
- MTP model: `DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf` (3,807,602,400 bytes, via `download_model.sh mtp`)

## Command

```sh
./ds4-agent --mtp gguf/DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf --mtp-draft 2
```

## What happens

Both model mappings register, then the q8 fp16 cache memory query starts failing immediately and
every subsequent CUDA call fails with the same (sticky) error until the agent gives up:

```
ds4: Linux cuda backend set oom_score_adj=1000
ds4: MTP support model loaded: gguf/DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf (draft=2)
ds4: CUDA backend initialized on NVIDIA GeForce RTX 4090 (sm_89)
ds4: CUDA registered 80.76 GiB model mapping for device access
ds4: CUDA registered 3.55 GiB model mapping for device access
ds4: CUDA preparing model tensor mappings: 80.24 GiB
ds4: CUDA q8 fp16 cache memory query failed: an illegal memory access was encountered; using q8 kernels
[... repeated ~230x ...]
ds4: CUDA startup model preparation covered 80.76 GiB of tensor spans in 0.332s
ds4: CUDA preparing model tensor mappings: 2.38 GiB
ds4: CUDA q8 fp16 cache memory query failed: an illegal memory access was encountered; using q8 kernels
[... repeated 8x ...]
ds4: CUDA startup model preparation covered 3.55 GiB of tensor spans in 0.000s
ds4: cuda backend initialized for graph diagnostics
ds4-agent: context buffers 2461.24 MiB (ctx=100000, backend=cuda, prefill_chunk=4096, raw_kv_rows=4352, compressed_kv_rows=25002)
ds4: CUDA tensor alloc failed: an illegal memory access was encountered
[... repeated ~400x ...]
ds4-agent: session backend is required
```

The first illegal access appears during the q8 fp16 cache memory query, right after the MTP
mapping (3.55 GiB) is registered — before that point the same query sequence works in the
no-MTP case.

## Baseline (works)

Identical build/model/machine without `--mtp` starts and runs normally:

```
ds4: CUDA q8 fp16 cache limit reached; using q8 kernels (request=64.00 MiB cached=7.94 GiB limit=8.00 GiB)
ds4: CUDA startup model preparation covered 80.76 GiB of tensor spans in 3.030s
```

Generation ~1.5–1.7 t/s (expected for dual-channel DDR4 host memory), agent tools and sessions
all working.

## Notes

- Possibly related to the startup-residency work: #325 (cuda-spark `--mtp` failure) was fixed
before this commit, and "Fix ROCm MTP startup residency" (June 16) touched the same area for
ROCm. This one is `cuda-generic` on a discrete GPU.
- WSL2 GPU paravirtualization may be relevant for host-mapped memory paths; I can test
driver/toolkit variations or patches if useful.
- I understand MTP is documented as experimental; reporting since the project notes ask for
outside CUDA testing.

Happy to run diagnostics, traces, or patched builds on this machine.

---
*Report drafted with AI assistance (Claude); the crash, logs, and hardware details are from a real session on my machine and I've reviewed the report before posting.*

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。