BlockRadixTree.clear() does not terminate when the tree holds a childless RootBlock
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- TensorRT-LLM: 1.3.0rc23 (built from
f20ea652); behaviour re-checked againstmain@fbda11f5 - GPU: NVIDIA GeForce RTX 5090 (sm_120), driver 595.71.05
- PyTorch 2.11.0+cu130, CUDA 13.0, Python 3.12.13
- Ubuntu 25.10, kernel 6.17.0-41-generic
- Backend: PyTorch; single GPU, no TP/PP
- Model where observed: Qwen3.6-35B-A3B-NVFP4 with FP8 KV cache (where model-specific)
Who can help?
No response
Reproduction
BlockRadixTree.clear() in tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py shrinks
self.next only as a side effect of its inner loop: detach_next() prunes a RootBlock when its
last child is detached. A RootBlock that is already childless has no last child to detach, so the
outer loop re-selects it indefinitely and clear() never returns. The trailing
assert not self.next is unreachable in that state.
A childless root is not exotic. RootBlock.__init__ ends with prev.next[self.key] = self, so
add_or_get_existing() publishes an empty root by construction, before its caller attaches a
block. Any interruption between those two steps leaves the tree permanently unclearable.
tree = BlockRadixTree(object(), 32) # life_cycles unused by clear()
tree.add_or_get_existing(ReuseScope(lora_id=0, salt=None))
tree.clear() # never returns
clear() touches neither the life-cycle registry nor pages, so this needs no device and no engine.
(salt must be None: ReuseScope.to_bytes() asserts non-int fields are unset.)
Expected behavior
clear() removes every root and returns, leaving the tree empty.
actual behavior
clear() never returns when a childless RootBlock is present: it pins one core at 100% with no GPU activity, and KVCacheManager.shutdown() hangs behind it.
additional notes
Make each iteration of clear() take an explicit progress step — detach a childless root, or remove
one subtree — so termination does not depend on a callee's side effect, and centralise root-edge
bookkeeping so eager pruning and explicit teardown cannot diverge.
We have a patch with eight CPU-only regression tests (6 fail before, all pass after) and are opening
this Issue Request first per CONTRIBUTING.md.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py and trace BlockRadixTree.clear(), RootBlock.init(), and detach_next(). Reproduce with the CPU-only snippet from the issue, then run the eight CPU-only regression tests mentioned in the report. Done means clear() returns with every root removed and the tree empty, without requiring a GPU or engine.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100