google / google/adk-web

adk web: 'System Instruction Performance Analysis' false-positives in multi-agent sessions

Open
#462 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1k
Forks
267
Avg merge
1d 12h
Merged PRs (30d)
9

Description

## Summary

The dev UI's **System Instruction Performance Analysis** warning ("System instructions were modified between consecutive turns in this session. This change breaks context cache alignment, resulting in a cache miss...") fires whenever a session contains more than one agent, even though no system instruction was actually modified.

`updateSystemInstructionFlags()` compares system instructions across **all** LLM spans in a session, sorted by time, **without partitioning by agent**. In any multi-agent / sub-agent flow, crossing an agent boundary makes the system instruction legitimately differ, which trips the warning. The diff dialog then shows two *different agents'* prompts, which is confusing.

## Where

`src/app/components/chat/chat.component.ts` → `updateSystemInstructionFlags()`:

```ts
const llmSpans = flatSpans
.filter(s => {
const isGCSpan = s.attrOperationName === OPERATION_GENERATE_CONTENT;
const isLegacySpan = s.name === 'call_llm';
return (isGCSpan || isLegacySpan) && s.io?.inputs !== undefined;
})
.sort((a, b) => (a.start_time || 0) - (b.start_time || 0));

// Compare consecutive LLM turns
for (let i = 1; i < llmSpans.length; i++) {
const currentSpan = llmSpans[i];
const precedingSpan = llmSpans[i - 1];

const currentSys = extractSystemInstruction(currentSpan.io?.inputs);
const precedingSys = extractSystemInstruction(precedingSpan.io?.inputs);

if (currentSys && precedingSys && currentSys !== precedingSys) {
// ...flag event.systemInstructionChanged = true
}
}
```

`llmSpans` spans the entire session across every agent, so consecutive spans from *different* agents get compared directly. There is no grouping by agent. The agent is the correct boundary: each agent has its own system instruction, and therefore its own cache context. "System instruction modified between turns" is only meaningful *within a single agent's own sequence of calls*. Two distinct agents never shared a cache to align in the first place, so comparing them is a category error — independent of whether they happen to use the same model or provider.

## Steps to reproduce

1. Run a root agent that delegates to a sub-agent (e.g. via `transfer_to_agent` / AgentTool), each with its own static `instruction`.
2. In `adk web`, send one message that causes the root agent to call the sub-agent and then respond.
3. The session's LLM calls end up ordered e.g. `root -> sub -> sub -> ... -> root`.
4. The warning appears, and the diff shows the root agent's prompt vs the sub-agent's prompt.

Neither system instruction was edited; the only "change" is the agent boundary.

## Expected

- No warning when the system instruction is stable *within a given agent's* own consecutive turns.
- The comparison should be grouped per agent (e.g. per event `author` / agent invocation), not across the whole session timeline.

## Proposed fix

In `updateSystemInstructionFlags()`, group `llmSpans` by agent (e.g. the event `author` / agent invocation, via `eventData.get(span.attrEventId)`) before the consecutive comparison, so `systemInstructionChanged` only reflects a real change within a single agent's own turns (e.g. a dynamic / state-templated instruction that genuinely changes between that agent's calls).

## Environment

- adk-web dev UI (bundled with `adk web`)
- Multi-agent setup (root agent + sub-agents)

Contributor guide

Open the contributing guide

Research direction

Start in src/app/components/chat/chat.component.ts at updateSystemInstructionFlags() and trace how eventData.get(span.attrEventId) identifies an event author or agent. Review the flatSpans filtering and current consecutive comparison, then group spans by agent before comparing system instructions. Done means cross-agent prompts no longer trigger the warning, while genuine changes within one agent's sequence still do.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.