anthropics / anthropics/claude-agent-sdk-typescript

Efficient System Prompt Updates Mid-Conversation

Aperta
#96 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Shell
Stelle
1.8k
Fork
226
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Summary

There's no current way to have efficient system prompts updates during an ongoing conversation.

## Use Case

We're building an agent with dynamic memory that can change mid-conversation. We need to inject updated context into the system prompt as the conversation evolves.

## Current Behavior

The SDK only allows setting `systemPrompt` at query initialization:

```typescript
const q = query({
prompt: promptStream, // async generator for multi-turn
options: {
systemPrompt: { type: 'preset', preset: 'claude_code', append: contextHere },
},
});
```

With streaming prompt mode (async generator), there's no way to update `systemPrompt` between turns.

## Workaround: Resume with New System Prompt

We tested using `resume` to update the system prompt:

```typescript
// Turn N+1: Resume session with updated system prompt
const q = query({
prompt: nextMessage,
options: {
resume: sessionId,
systemPrompt: newSystemPromptWithFreshContext, // Updated!
},
});
```

The new system prompt is applied while preserving conversation history.

## Performance Analysis

However, this approach has overhead. We benchmarked 3-turn conversations:

| Mode | Avg Turn Time | Pattern |
|------|---------------|---------|
| **Streaming** (single query, prompt generator) | ~2,700ms | Turn 1: 4.8s, Turn 2: 2.8s, Turn 3: 2.0s |
| **Resume-per-turn** (new query each turn) | ~4,150ms | All turns: ~4s consistently |

**Overhead: ~1-1.5 seconds per turn** when using resume-per-turn.

## Root Cause Analysis

We tested whether the overhead was from prompt re-processing or lack of prompt caching by comparing short (~50 tokens) vs long (~5,400 tokens) system prompts:

| Mode | Short Prompt | Long Prompt | Difference |
|------|-------------|-------------|------------|
| **Streaming** | 2,706ms | 3,238ms | +531ms |
| **Resume** | 4,148ms | 4,144ms | **-4ms** (identical!) |

We see that the overhead is NOT from prompt processing. It seems it's likely from subprocess restart.

This is partially good news because large system prompts don't add penalty, but the fixed ~1.5s overhead per turn is significant for interactive applications.

## Feature Request

It would be valuable to have a way to update the system prompt within an active streaming query, avoiding subprocess restart overhead.

## Environment

- `@anthropic-ai/claude-agent-sdk`: latest
- Node.js: v22.x
- Platform: macOS

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.