NVIDIA / NVIDIA/TensorRT-LLM

BlockRadixTree.clear() does not terminate when the tree holds a childless RootBlock

Open
#17,908 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

KV-Cache Management
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

System Info
  • TensorRT-LLM: 1.3.0rc23 (built from f20ea652); behaviour re-checked against main @ fbda11f5
  • GPU: NVIDIA GeForce RTX 5090 (sm_120), driver 595.71.05
  • PyTorch 2.11.0+cu130, CUDA 13.0, Python 3.12.13
  • Ubuntu 25.10, kernel 6.17.0-41-generic
  • Backend: PyTorch; single GPU, no TP/PP
  • Model where observed: Qwen3.6-35B-A3B-NVFP4 with FP8 KV cache (where model-specific)
Who can help?

No response

Reproduction

BlockRadixTree.clear() in tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py shrinks
self.next only as a side effect of its inner loop: detach_next() prunes a RootBlock when its
last child is detached. A RootBlock that is already childless has no last child to detach, so the
outer loop re-selects it indefinitely and clear() never returns. The trailing
assert not self.next is unreachable in that state.

A childless root is not exotic. RootBlock.__init__ ends with prev.next[self.key] = self, so
add_or_get_existing() publishes an empty root by construction, before its caller attaches a
block. Any interruption between those two steps leaves the tree permanently unclearable.

tree = BlockRadixTree(object(), 32)                        # life_cycles unused by clear()
tree.add_or_get_existing(ReuseScope(lora_id=0, salt=None))
tree.clear()                                               # never returns

clear() touches neither the life-cycle registry nor pages, so this needs no device and no engine.
(salt must be None: ReuseScope.to_bytes() asserts non-int fields are unset.)

Expected behavior

clear() removes every root and returns, leaving the tree empty.

actual behavior

clear() never returns when a childless RootBlock is present: it pins one core at 100% with no GPU activity, and KVCacheManager.shutdown() hangs behind it.

additional notes

Make each iteration of clear() take an explicit progress step — detach a childless root, or remove
one subtree — so termination does not depend on a callee's side effect, and centralise root-edge
bookkeeping so eager pruning and explicit teardown cannot diverge.

We have a patch with eight CPU-only regression tests (6 fail before, all pass after) and are opening
this Issue Request first per CONTRIBUTING.md.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py and trace BlockRadixTree.clear(), RootBlock.init(), and detach_next(). Reproduce with the CPU-only snippet from the issue, then run the eight CPU-only regression tests mentioned in the report. Done means clear() returns with every root removed and the tree empty, without requiring a GPU or engine.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.