Potential cache collision in the semantic caching policy

Current Limitation

The semantic cache policy (gateway/policies/semantic-cache/v0.1.0) lacks user or tenant-level isolation in its cache key strategy. Cache entries are scoped only by api_id ({APIName}:{APIVersion}), meaning all users of the same API share the same cache namespace.

Vulnerability Details

Affected Code: semanticcache.go L355-358

cacheFilter := map[string]interface{}{
    "threshold": fmt.Sprintf("%.2f", p.threshold),
    "api_id":    apiID,
    "ctx":       context.Background(),
}

The cache lookup and storage use only api_id for isolation. No user identifier, session token, or tenant ID is included.

Impact

Cross-user data leakage: User A may receive cached responses intended for User B when their queries are semantically similar
Privacy violation: Personalized or user-specific LLM responses (account details, recommendations, private data) could be served to unrelated users

Reproduction Scenario

User A sends: "What is my account balance?"
LLM responds with User A's balance → cached
User B sends: "Show me my account balance"
Semantic similarity exceeds threshold → User B receives User A's cached balance

Suggested Improvement

Proposed Solution

Add user/tenant context to the cache filter:

cacheFilter := map[string]interface{}{
    "threshold": fmt.Sprintf("%.2f", p.threshold),
    "api_id":    apiID,
    "user_id":   ctx.UserID,    // or extract from JWT/headers
    "tenant_id": ctx.TenantID,  // I am not sure whether we need to support
    "ctx":       context.Background(),
}

Consider adding a policy parameter to configure the isolation level:

none — current behavior (shared cache)
user — per-user cache isolation
tenant — per-tenant cache isolation // if we support multi tenant might have to consider this as well.

Version

No response

Start at gateway/policies/semantic-cache/v0.1.0/semanticcache.go lines 355-358 and trace how the cache filter is used for lookup and storage. Determine which user or tenant context is available and how an isolation level should be configured, then verify that semantically similar requests from different users do not share private responses.