SemanticCache.check() unconditionally refreshes TTL on hit (sliding-window with no opt-out)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 427
- Forks
- 101
- Avg merge
- 6d 3h
- Merged PRs (30d)
- 20
Description
Current behavior
SemanticCache.check (redisvl/extensions/cache/llm/semantic.py:432) refreshes the TTL of every matched entry on every hit:
# Refresh TTL on all found keys
for key in redis_keys:
self.expire(key)
This is a sliding-window TTL: a popular entry effectively lives as long as it keeps getting hit. There is no constructor flag, per-call argument, or per-entry attribute to disable this.
Why this is a problem
Sliding window is the wrong default in any domain where the popularity of an entry correlates with the cost of it being stale:
- a procedure superseded by a regulatory bulletin that the cache has not yet invalidated
- a balance-patch-affected gameplay answer in a live-ops surface
- a pricing or eligibility answer that changed at midnight
In these domains the most-asked question is the most likely to be cached and the most likely to fan out a stale answer. The standard mitigation today — store an absolute expires_at numeric and filter on it at read time — works but requires every customer to discover the pattern independently.
Proposed API
SemanticCache(
name="cache",
redis_url=...,
refresh_ttl_on_hit=True, # current behavior, default unchanged
)
cache.check(
prompt=...,
refresh_ttl=None, # per-call override
)
refresh_ttl_on_hit=False would make check() a pure read. Customers wanting absolute expiry would set this to False, store a numeric expires_at filterable field, and filter on it.
Backwards compatibility
Default remains True. Existing behavior unchanged. Opt-in behavioral change for customers who need it.
Notes
Surfaced while writing a scoped semantic caching architecture spec; came up repeatedly when stress-testing the design against regulated and fast-mutating domains where sliding-window TTL is actively hostile.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in redisvl/extensions/cache/llm/semantic.py at SemanticCache.check() around line 432, then read the surrounding SemanticCache constructor and current TTL-refresh path. Verify the proposed constructor flag and per-call override preserve the default behavior while allowing a pure read when disabled; done means both options are supported without changing existing callers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100