fix(KV-MANAGER-ALLOC): a short external cache hit requests a negative block count, inflating the free-block counter before it throws
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: KV-MANAGER-ALLOC (.agents/engine-matrix.md:98, "Slot allocation, watermark, admission, release"), whose recorded upstream test anchor is tests/v1/core/test_single_type_kv_cache_manager.py — the very file this commit extends. Resolved by line number before filing.
Found by wave PORTQ-7 (#2717) re-deriving PORT-NOW entry [266], upstream 53bf990502 vllm#52707. Nothing was executed.
The defect, and why it is worse here than upstream
SingleTypeKVCacheManager::allocate_external_computed_blocks (src/vllm/v1/core/single_type_kv_cache_manager.cpp:132-158) carries upstream's unclamped expression verbatim:
std::vector<KVCacheBlock*> allocated_blocks = block_pool.get_new_blocks(
cdiv(num_total_computed_tokens, block_size) -
static_cast<int>(req_blocks.size())); // :148-150
When an external or connector hit is shorter than the blocks the request already holds, the count goes negative. Upstream's Python then merely returned []. This tree does something worse, and each link was read:
BlockPool::get_new_blocks(src/vllm/v1/core/block_pool.cpp:340-345) testsnum_blocks > get_num_free_blocks(), which a negative count passes, and forwards topopleft_n.FreeKVCacheBlockQueue::popleft_n(src/vllm/v1/core/kv_cache_utils.cpp:781-791) skips itsif (n == 0) return {};guard, passesassert(num_free_blocks >= n)trivially — and that assert is compiled out underNDEBUGin any case — then executesnum_free_blocks -= n, inflating the free-block counter. That is precisely the pool corruption upstream's new regression test asserts against.- It then calls
ret.reserve(static_cast<size_t>(n))with a negativeintcast tosize_t, which throwsstd::length_error. The pop loop never runs, so the counter stays inflated behind the throw.
There is no override to soften it: allocate_external_computed_blocks appears exactly twice in this tree — the declaration at include/vllm/v1/core/single_type_kv_cache_manager.h:123 and this definition. No subclass overrides it.
Reachability
src/vllm/v1/core/kv_cache_coordinator.cpp:213-216 -> src/vllm/v1/core/kv_cache_manager.cpp:268-273 -> the connector's num_external_computed_tokens at src/vllm/v1/core/sched/scheduler.cpp:894, live whenever --kv-transfer-config selects a connector.
Honest limit on that claim. No configuration in this tree was found that actually drives the count negative through KVCacheManager::allocate_slots, because here allocate_new_computed_blocks runs before allocate_new_blocks (kv_cache_manager.cpp:268-280) and early-returns for any already-tracked request (kv_cache_coordinator.cpp:191-205), so the block table is empty on entry. Upstream's own regression test likewise drives the manager directly rather than through allocate_slots. What would settle it: enumerating the get_num_skipped_tokens overrides — SlidingWindowManager (:536), ChunkedLocalAttentionManager (:633), MambaManager (:887, which returns n-1) — against a multi-group config with a live connector.
The clamp is worth porting regardless: it is one line guarding a counter corruption plus a throw, and it costs nothing if the path is unreachable.
The commit is the whole distance
git show 5559679229:vllm/v1/core/single_type_kv_cache_manager.py:282-319 is exactly the ported shape. Not a pre-pin hole. No later in-range commit touches the clamp.
Size
2-3 product lines — the std::max(0, ...) clamp before get_new_blocks — plus one ported regression test. Upstream's test_external_computed_blocks_do_not_corrupt_free_pool is ~40 lines and asserts that get_num_free_blocks() is unchanged and the block table stays at 3; port it with its parameters and its assertions rather than re-deriving.
Consider also clamping or refusing inside popleft_n. A negative n reaching that function is a caller bug in every case, and it currently corrupts state before it throws. That is a second, independent hardening this issue does not require but a reviewer should weigh.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with allocate_external_computed_blocks in src/vllm/v1/core/single_type_kv_cache_manager.cpp and compare its behavior with tests/v1/core/test_single_type_kv_cache_manager.py. Port the upstream regression test for a short external cache hit and verify the free-block count is unchanged and the block table remains at three blocks; the clamp belongs immediately before get_new_blocks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100