NVIDIA / NVIDIA/TensorRT-LLM

[Bug]: HTTP cluster get_prefix exposes expired keys before cleanup sweep

Open
#17,763 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Disaggregated serving
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

System Info

Platform-independent HTTP cluster-storage behavior on current main.

Reproduction

HttpClusterStorageServer.get() checks expire_time before returning a value, but get_prefix() returns every matching entry without checking expiration. The background cleanup task runs periodically, so during the interval after a TTL expires and before the next sweep, get_prefix() can return stale data that get() already considers expired.

Expected behavior

Point reads and prefix reads should use the same expiration semantics. Expired entries must not be returned by get_prefix().

Actual behavior

Expired entries remain visible through prefix reads until the background cleanup loop removes them.

Proposed fix

Filter prefix results using the same expire_time rule as get(), with a deterministic CPU-only regression test that does not depend on sleeping for the cleanup interval.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at HttpClusterStorageServer.get_prefix() and compare its expiration handling with get(), focusing on the existing expire_time rule. Add a deterministic CPU-only regression test for prefix reads with an expired entry, without waiting for the cleanup interval. Done means get_prefix() no longer returns entries that get() considers expired.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.