mudler / mudler/vllm.cpp

SIGSEGV in InputBatch::add_request serving a text completion on a synthetic DeepSeek-V4 fixture, ~50% of runs

Open
#3,027 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: -

A TEXT completion served through VllmServerMain against a synthetic DeepSeek-V4 GGUF fixture crashes on roughly half of runs, at one token as readily as at 260.

Two signatures were observed, both on the same harness:

  • SIGSEGV in InputBatch::add_request
  • otherwise SIGSEGV in gather_block_table

Observed 3 crashes in 6 runs while building the W5 serving gate for #2411. The IMAGE request path on the same harness reached ModelRegistry::Forward 8 times out of 8, so whatever this is, it is not the multimodal path.

It is reported rather than gated. The wave that found it removed the text case from its suite instead of asserting a behaviour it could not make deterministic — tests/vllm/entrypoints/openai/test_serve_deepseek_v4_mm.cpp therefore runs 2 cases with 1 skipped, and passes 8 of 8 for me at that shape. Asserting an unstable case would have made the suite a coin flip; leaving it unrecorded would have hidden a crash.

Two fixture conditions had to be pinned to get the load to happen at all, and they may be related or may be separate defects:

  1. A GGUF without deepseek4.context_length resolves max_model_len = 0 and then SIGSEGVs, rather than refusing by name.
  2. Prefix caching must be disabled: BlockPool::cache_full_blocks refuses this KV topology's block/hash-block pair.

What is NOT established: whether this reproduces on a real DeepSeek-V4 checkpoint, or only on the synthetic fixture; and whether it predates #2411 entirely. Nothing in W5 touches InputBatch or the block table, which is why this is filed separately from that row rather than as a regression against it. Establishing which of those two it is, is the first step, not the fix.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running tests/vllm/entrypoints/openai/test_serve_deepseek_v4_mm.cpp with the synthetic DeepSeek-V4 GGUF fixture and its pinned max_model_len and prefix-caching conditions. Trace the text path from VllmServerMain through InputBatch::add_request and gather_block_table. Done means establishing whether the crash reproduces on a real checkpoint or only the fixture, with a deterministic regression case or a documented cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend-api-design, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.