cloudflare / cloudflare/agents

Append reminders to fake system prompt “updates" without cache invalidation

Open
#2,089 0 comments 1 reaction 1 assignee Claimed by @threepointone View on GitHub
enhancement
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

`Session.freezeSystemPrompt()` preserves a stable prefix for provider prompt caching, but it also freezes context that may change between turns. Calling `refreshSystemPrompt()` keeps that context current but invalidates the cached prefix.

It would be useful for selected context blocks to opt into dependency or revision tracking. When one changes, its updated value could be supplied as inference-only tail context while leaving the frozen prompt intact. An explicit `refreshSystemPrompt()` would then promote current values into the frozen prompt and reset the baseline.

Examples include the current date, workspace state, or project instructions.

`beforeTurn` can handle these cases manually today; this feature would connect that behavior to the Session context lifecycle and make cache invalidation an explicit choice.

Perhaps something like

```ts
// Illustrative API — exact names are not important.
configureSession(session: Session) {
return session
.withContext("environment", {
provider: {
get: async () =>
`Today's date is ${new Date().toISOString().slice(0, 10)}.`
},
cache: {
revision: () => new Date().toISOString().slice(0, 10),
whenChanged: "tail"
}
})
.withCachedPrompt();
}
```

The first rendered value would be included in the frozen system prompt. When `revision()` changes, the updated block would be supplied as inference-only tail context without rewriting that prompt. The application could later accept the cache bust explicitly:

```ts
await this.session.refreshSystemPrompt();
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.