agentscope-ai / agentscope-ai/agentscope-java
[Feature]: render session-scoped variables at the end of the system prompt
- 主要語言
- Java
- 星號
- 5.6k
- 分支
- 1.3k
- 平均合併
- 4 天 12 小時
- 30 天內合併 PR
- 77
描述
## Feature request: render session-scoped variables at the end of the system prompt
**Is your feature request related to a problem? Please describe.**
The harness renders session-scoped values (workspace path, AgentStateStore ID, skill ``) in the **middle** of the system prompt. Because these values change on every conversation, they cut the cross-session stable prefix at roughly 50% of the system prompt, which defeats provider-side prompt caching for the first model call of every new conversation.
Environment: `agentscope-harness 2.0.1`, `agentscope-extensions-model-dashscope 2.0.1`, DashScope OpenAI-compatible endpoint, model `qwen3.7-plus`.
In our deployment the system prompt is 12,919 characters and the conversation id appears 6 times, the earliest at offset 6,420:
| Offset | Content | Rendered by |
| --- | --- | --- |
| 6420 | `The workspace directory is: //` | harness |
| 6552 | `AgentStateStore ID: ` | `WorkspaceContextMiddleware` |
| 8075 | `Project (the user's source tree you're assisting with): ` | `WorkspaceContextMiddleware` |
| 8250 | `Workspace (your home base — memory, sessions, skills, runtime data): ` | `WorkspaceContextMiddleware` |
| 11065 | skill ``: `/.skills-cache/...` | `SkillPromptBuilder` |
| 11802 | same, for the second skill | `SkillPromptBuilder` |
As a result the byte-stable cross-session prefix is only about 6,716 characters, roughly 20% of the whole request body.
We measured the impact with a rewriting proxy. Same backend configuration, same query, a fresh conversation id per run, reading `usage.prompt_tokens_details.cached_tokens` of the **first** model call only:
| Handling | first-call `cached_tokens` |
| --- | --- |
| forwarded unchanged (control) | 0, 0, 0, 0 |
| the 6 occurrences replaced by a constant, real value appended at the **end** of the system message | 0, 0, **4224, 4224, 4224** |
Changing only the *position* of these variables turned a configuration that never hit the cache into one that hits it consistently. The two leading zeros are cache-entry creation.
This also matches the provider's own guidance (put repeated content at the beginning of the prompt and varying content at the end). The current rendering order is exactly the anti-pattern that guidance warns about.
**Describe the solution you'd like**
Group the session-scoped values into a single block and render it at the **end** of the system prompt. For example, keep a `## Session Context` section at the very end holding the workspace path, the AgentStateStore ID and the skill `files-root` values, and have the earlier prose refer to it instead of inlining the paths.
If changing the default layout is too disruptive, an opt-in would be enough for us, for example:
```java
HarnessAgent.builder()
.sessionContextPlacement(SessionContextPlacement.END) // default: INLINE (current behaviour)
```
A smaller, independent improvement: let `SkillPromptBuilder` emit `` as a path relative to the workspace, which removes two absolute-path occurrences on its own.
**Describe alternatives you've considered**
Integrators cannot fix this on their own. We only pass `sysPrompt`, `workspace(Path)` and `defaultSessionId`; the wording and the position of the lines above are decided entirely by the library. We evaluated three workarounds:
1. **Change the workspace layout to be per user** (no conversation id in the path). This only moves the stable prefix from offset 6,420 to 6,552, since `AgentStateStore ID` is the next breaking point. The gain is about 132 characters, effectively zero, and it would make all conversations of one user share a workspace directory, which breaks session-level isolation.
2. **Intercept at the model layer and rewrite the system message before sending.** This produces the measured gain, but the model then sees a workspace path that does not match the real directory, which risks breaking tools that build absolute paths (for example a shell tool). Trading tool correctness for a cache discount is not acceptable to us.
3. **Ask upstream** (this issue).
**Additional context**
One caveat we want to be explicit about, so the expected benefit is not overstated: on the same provider we also observed that **two configurations with structurally equivalent payloads can differ in whether a cache entry is ever created** (one hit 4/5 times, the other 0/5 across repeated runs), and we have not been able to determine the condition from the client side. We have raised that separately with the provider.
So the benefit of this request is not "the cache will always hit". It is "one confirmed cause of cache invalidation, purely due to rendering position, is removed". The 4,224-token result above is a before/after comparison within a single configuration, so it is not affected by that unknown factor.
Reproduction, without our codebase:
1. Build a harness agent with skills enabled and `workspace` pointing at `//`.
2. Send one fixed short user message, using a **fresh conversation id** each time, at least 5 runs.
3. Read `usage.prompt_tokens_details.cached_tokens` from the provider response, looking only at the **first** model call of each run. Later calls within the same run hit the cache for unrelated reasons and must be excluded.
4. Repeat with the session ids replaced by a constant and the real value appended at the end of the system message, and compare.
貢獻指南
評估
這個 Issue 還沒有評估資料。