mudler / mudler/vllm.cpp

fix(KV-PREFIX-MATCH-UNIT): the mamba back-off predicate compares block sizes instead of reading mamba_cache_mode

Open Beginner friendly
#2,663 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: KV-PREFIX-MATCH-UNIT

Found by wave PORTQ-2 re-deriving PORT-NOW entry 73 of 5559679229..e126687a9a
(#2646). Upstream:
23c0f337b7 vllm#51180. Previously read by #2524's sample
(.agents/sync/2026-09-01-cdefd9d.md §13 item 2); re-verified at e24ec8bfd,
both anchors still exact.

A one-line predicate swap

src/vllm/v1/core/kv_cache_utils.cpp:679-682 is the pre-fix form:

  for (const KVCacheGroupSpec& g : groups) {
    const MambaSpec* m = dynamic_cast<const MambaSpec*>(g.kv_cache_spec.get());
    if (m != nullptr && m->block_size != cache_block_size) {
      return {scheduler_block_size, scheduler_block_size};
    }
  }

Upstream replaces g.kv_cache_spec.block_size != cache_config.block_size with
g.kv_cache_spec.mamba_cache_mode != "align". The field already exists here —
std::string mamba_cache_mode; at include/vllm/v1/kv_cache_interface.h:481
and is already read elsewhere
(src/vllm/v1/core/single_type_kv_cache_manager.cpp:752,788,818,880;
src/vllm/v1/core/kv_cache_coordinator.cpp:104).

The red-first hook is already in the tree

tests/vllm/v1/test_prefix_match_unit.cpp:174-180 builds its "align" arm as
MakeMamba(16) against cache_block_size = 16, and MakeMamba (:43-47)
leaves mamba_cache_mode at its "none" default
(include/vllm/v1/kv_cache_interface.h:469). Under the corrected predicate that
arm backs off too, so CHECK(hash_a != hash_n) at :180 goes RED. Whoever takes
this gets the failing-for-the-right-reason test for free, and then gives the
align arm an explicit "align" mode.

Size

Roughly 6 lines over two files (src/vllm/v1/core/kv_cache_utils.cpp and
tests/vllm/v1/test_prefix_match_unit.cpp), plus the comment at
include/vllm/v1/core/kv_cache_utils.h:429, which states the old rule and would
otherwise be left saying something the code no longer does.

Nothing was executed for this finding: no build, no test, no GPU.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vllm/v1/core/kv_cache_utils.cpp:679-682 and read the related comment in include/vllm/v1/core/kv_cache_utils.h:429. Update the predicate to use the existing mamba_cache_mode value, set the align arm in tests/vllm/v1/test_prefix_match_unit.cpp to explicit align mode, and run that test to confirm the corrected back-off behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance, testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.