api7 / api7/aisix

feat(cache): manual purge endpoint for a cache policy

Open
#89 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cross-repo enhancement P2
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Background

Today every cached response sits in the DP's in-memory cache until its ttl_seconds expires. There is no operator-facing way to invalidate a policy's entries on demand — the only options are:

  1. Wait for natural TTL expiry (worst case = full TTL window, default 1h).
  2. Restart the DP (blunt; affects every policy and all in-flight requests).

Both are unacceptable when the operator just deployed a new system-prompt template, fixed a bad prompt that's now poisoning responses, or rolled the upstream model to a new version.

Proposed fix

End-to-end "Clear cached entries" capability:

  • cp-api: new endpoint DELETE /api/orgs/:org/envs/:env/cache_policies/:id/entries.
  • dp-manager: forwards a purge command to the env's DPs over the existing mTLS control channel.
  • DP: handles the purge by calling cache.invalidate_all() (moka exposes this directly). Scope to a specific policy when the backend supports prefixed keys; for the in-memory backend, full invalidation is acceptable as the first cut and matches operator intent ("clear my cached responses").
  • Dashboard: cache-policy detail page gains a destructive-style "Clear cached entries" button + confirmation dialog.

Why this delivers value

  1. Incident response. Bad prompt template ships → operator can clear poisoned responses in seconds rather than waiting out the TTL or restarting the DP.
  2. Lets operators raise TTL. Many teams keep TTL artificially short (5–15 min) purely so they can "wait it out" in an emergency. With manual purge they can confidently raise it to 1–6h, recovering significant hit-rate.
  3. Prompt rollout hygiene. Pair purge with prompt-template changes for clean cutovers — no half-old / half-new responses for the next TTL window.
  4. Developer ergonomics. Editing a downstream prompt and wanting to see fresh model output is one button click instead of waiting / restarting.

Acceptance criteria

  • DP exposes a purge handler reachable via the existing dp-manager command channel.
  • cp-api: DELETE /api/orgs/:org/envs/:env/cache_policies/:id/entries with auth, audit-log entry, idempotent behavior.
  • Dashboard: confirmation-gated button on the cache-policy detail page.
  • e2e: write a known prompt → confirm cache hit → call purge → confirm next request is a miss (and cache repopulates).
  • No purge required across DPs in the same env to be transactional — best-effort fan-out is fine, but the response should report which DPs ack'd.

Out of scope

  • Per-key purge (granular invalidation) — full-policy purge covers the operational use cases; per-key can come later.
  • Distributed cache backends (Redis / pgvector) — they'll need their own purge implementations under the same trait method when those backends ship.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the existing dp-manager command channel, the DP cache implementation, the cp-api cache-policy routes, the dashboard cache-policy detail page, and the end-to-end test setup. Start by mapping how commands and cache policies currently flow across these components. Done means an authenticated, audited, idempotent purge reaches the DPs, the dashboard confirms the action, acknowledgements are reported, and the e2e flow proves the next request misses and repopulates the cache.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend, frontend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.