Kong / Kong/developer.konghq.com

AI Gateway: Conditional semantic caching in the AI Semantic Cache plugin

Open
#4,121 2 comments 0 reactions 0 assignees View on GitHub
internal priority: high product:ai-gateway release-docs
Dominant language
Ruby
Stars
28
Forks
121
Avg merge
1d 4h
Merged PRs (30d)
313

Description

## Jobs to be done

Users need to control when prompts are cached based on semantic conditions, such as excluding time-sensitive queries or preventing cache hits for follow-up requests like "tell me more." For this, we'll need to document the `only_cache_if` and `skip_cache_if` configuration parameters in the AI Semantic Cache plugin reference.

## Definition of done

- Include plugin examples showing how to cache general queries while excluding time-sensitive or contextual follow-ups.
- Prepare a how-to for using conditional caching using the AI Semantic Cache plugin. Best to mimic a simplified real-world scenario.

## Information

Person of contact: Wangchong
Jira ticket: https://konghq.atlassian.net/browse/AG-109

## Size
S to M

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the AI Semantic Cache plugin reference and existing how-to documentation in the developer.konghq.com repository. Document the only_cache_if and skip_cache_if parameters, add examples for general, time-sensitive, and contextual follow-up prompts, and finish with a simplified real-world conditional-caching guide.

Written by the indexing model from the issue text.

Assessment

Domain
ai, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.