anthropics / anthropics/claude-agent-sdk-typescript

Efficient System Prompt Updates Mid-Conversation

Offen
#96 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Shell
Sterne
1.8k
Forks
226
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

## Summary

There's no current way to have efficient system prompts updates during an ongoing conversation.

## Use Case

We're building an agent with dynamic memory that can change mid-conversation. We need to inject updated context into the system prompt as the conversation evolves.

## Current Behavior

The SDK only allows setting `systemPrompt` at query initialization:

```typescript
const q = query({
prompt: promptStream, // async generator for multi-turn
options: {
systemPrompt: { type: 'preset', preset: 'claude_code', append: contextHere },
},
});
```

With streaming prompt mode (async generator), there's no way to update `systemPrompt` between turns.

## Workaround: Resume with New System Prompt

We tested using `resume` to update the system prompt:

```typescript
// Turn N+1: Resume session with updated system prompt
const q = query({
prompt: nextMessage,
options: {
resume: sessionId,
systemPrompt: newSystemPromptWithFreshContext, // Updated!
},
});
```

The new system prompt is applied while preserving conversation history.

## Performance Analysis

However, this approach has overhead. We benchmarked 3-turn conversations:

| Mode | Avg Turn Time | Pattern |
|------|---------------|---------|
| **Streaming** (single query, prompt generator) | ~2,700ms | Turn 1: 4.8s, Turn 2: 2.8s, Turn 3: 2.0s |
| **Resume-per-turn** (new query each turn) | ~4,150ms | All turns: ~4s consistently |

**Overhead: ~1-1.5 seconds per turn** when using resume-per-turn.

## Root Cause Analysis

We tested whether the overhead was from prompt re-processing or lack of prompt caching by comparing short (~50 tokens) vs long (~5,400 tokens) system prompts:

| Mode | Short Prompt | Long Prompt | Difference |
|------|-------------|-------------|------------|
| **Streaming** | 2,706ms | 3,238ms | +531ms |
| **Resume** | 4,148ms | 4,144ms | **-4ms** (identical!) |

We see that the overhead is NOT from prompt processing. It seems it's likely from subprocess restart.

This is partially good news because large system prompts don't add penalty, but the fixed ~1.5s overhead per turn is significant for interactive applications.

## Feature Request

It would be valuable to have a way to update the system prompt within an active streaming query, avoiding subprocess restart overhead.

## Environment

- `@anthropic-ai/claude-agent-sdk`: latest
- Node.js: v22.x
- Platform: macOS

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.