redis / redis/redis-vl-python

SemanticCache.check() unconditionally refreshes TTL on hit (sliding-window with no opt-out)

Open
#603 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cache enhancement
Dominant language
Python
Stars
427
Forks
101
Avg merge
6d 3h
Merged PRs (30d)
20

Description

Current behavior

SemanticCache.check (redisvl/extensions/cache/llm/semantic.py:432) refreshes the TTL of every matched entry on every hit:

# Refresh TTL on all found keys
for key in redis_keys:
    self.expire(key)

This is a sliding-window TTL: a popular entry effectively lives as long as it keeps getting hit. There is no constructor flag, per-call argument, or per-entry attribute to disable this.

Why this is a problem

Sliding window is the wrong default in any domain where the popularity of an entry correlates with the cost of it being stale:

  • a procedure superseded by a regulatory bulletin that the cache has not yet invalidated
  • a balance-patch-affected gameplay answer in a live-ops surface
  • a pricing or eligibility answer that changed at midnight

In these domains the most-asked question is the most likely to be cached and the most likely to fan out a stale answer. The standard mitigation today — store an absolute expires_at numeric and filter on it at read time — works but requires every customer to discover the pattern independently.

Proposed API

SemanticCache(
    name="cache",
    redis_url=...,
    refresh_ttl_on_hit=True,    # current behavior, default unchanged
)

cache.check(
    prompt=...,
    refresh_ttl=None,           # per-call override
)

refresh_ttl_on_hit=False would make check() a pure read. Customers wanting absolute expiry would set this to False, store a numeric expires_at filterable field, and filter on it.

Backwards compatibility

Default remains True. Existing behavior unchanged. Opt-in behavioral change for customers who need it.

Notes

Surfaced while writing a scoped semantic caching architecture spec; came up repeatedly when stress-testing the design against regulated and fast-mutating domains where sliding-window TTL is actively hostile.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in redisvl/extensions/cache/llm/semantic.py at SemanticCache.check() around line 432, then read the surrounding SemanticCache constructor and current TTL-refresh path. Verify the proposed constructor flag and per-call override preserve the default behavior while allowing a pure read when disabled; done means both options are supported without changing existing callers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.