anthropics / anthropics/claude-agent-sdk-typescript
Efficient System Prompt Updates Mid-Conversation
- Lingua principale
- Shell
- Stelle
- 1.8k
- Fork
- 226
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## Summary
There's no current way to have efficient system prompts updates during an ongoing conversation.
## Use Case
We're building an agent with dynamic memory that can change mid-conversation. We need to inject updated context into the system prompt as the conversation evolves.
## Current Behavior
The SDK only allows setting `systemPrompt` at query initialization:
```typescript
const q = query({
prompt: promptStream, // async generator for multi-turn
options: {
systemPrompt: { type: 'preset', preset: 'claude_code', append: contextHere },
},
});
```
With streaming prompt mode (async generator), there's no way to update `systemPrompt` between turns.
## Workaround: Resume with New System Prompt
We tested using `resume` to update the system prompt:
```typescript
// Turn N+1: Resume session with updated system prompt
const q = query({
prompt: nextMessage,
options: {
resume: sessionId,
systemPrompt: newSystemPromptWithFreshContext, // Updated!
},
});
```
The new system prompt is applied while preserving conversation history.
## Performance Analysis
However, this approach has overhead. We benchmarked 3-turn conversations:
| Mode | Avg Turn Time | Pattern |
|------|---------------|---------|
| **Streaming** (single query, prompt generator) | ~2,700ms | Turn 1: 4.8s, Turn 2: 2.8s, Turn 3: 2.0s |
| **Resume-per-turn** (new query each turn) | ~4,150ms | All turns: ~4s consistently |
**Overhead: ~1-1.5 seconds per turn** when using resume-per-turn.
## Root Cause Analysis
We tested whether the overhead was from prompt re-processing or lack of prompt caching by comparing short (~50 tokens) vs long (~5,400 tokens) system prompts:
| Mode | Short Prompt | Long Prompt | Difference |
|------|-------------|-------------|------------|
| **Streaming** | 2,706ms | 3,238ms | +531ms |
| **Resume** | 4,148ms | 4,144ms | **-4ms** (identical!) |
We see that the overhead is NOT from prompt processing. It seems it's likely from subprocess restart.
This is partially good news because large system prompts don't add penalty, but the fixed ~1.5s overhead per turn is significant for interactive applications.
## Feature Request
It would be valuable to have a way to update the system prompt within an active streaming query, avoiding subprocess restart overhead.
## Environment
- `@anthropic-ai/claude-agent-sdk`: latest
- Node.js: v22.x
- Platform: macOS
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.