google / google/adk-python

Session state does not cross a RemoteA2aAgent boundary in either direction, silently

Open
#6,854 8 comments 0 reactions 2 assignees Claimed by @GWeale View on GitHub
a2a needs review
Dominant language
Python
Stars
21.5k
Forks
4k
Avg merge
1d 14h
Merged PRs (30d)
37

Description

### Summary

Session state does not cross a `RemoteA2aAgent` boundary **in either direction**, and nothing in
the API surface, logs, or docs indicates it. Because the same agent graph works correctly
in-process, this is invisible to unit tests, integration tests and CI, and only appears once an
agent is actually deployed behind A2A.

We hit it twice, in opposite directions, in a `SequentialAgent` pipeline that ran fine locally for
weeks.

### Environment

- `google-adk==2.7.1`, `a2a-sdk==1.1.2`, Python 3.12
- Orchestrator is a `SequentialAgent` on Agent Engine; two peers are `RemoteA2aAgent`s backed by
Cloud Run services

### Direction 1 — `output_key` on a remote agent never reaches the caller

```python
# runs in the worker process
auditor = LlmAgent(name="auditor", output_schema=Report, output_key="findings", ...)

# runs in the orchestrator
class NextStep(BaseAgent):
async def _run_async_impl(self, ctx):
report = ctx.session.state.get("findings") # populated in-process, always None over A2A
```

`output_key` writes into the session of the agent that *declares* it. In-process the declaring
agent and the reading agent share one session, so this works. Over A2A the worker has its own
session and that state never returns to the caller.

### Direction 2 — a caller-side `state_delta` never reaches the remote agent

```python
class PolicyStep(BaseAgent):
async def _run_async_impl(self, ctx):
yield Event(author=self.name, invocation_id=ctx.invocation_id,
actions=EventActions(state_delta={"routing": routing})) # no content

# next in the SequentialAgent
RemoteA2aAgent(name="notifier", ...) # never sees "routing"
```

`RemoteA2aAgent._construct_message_parts_from_session` builds the outgoing message from
`event.content.parts` and skips events whose `content` is `None`. A state-only event therefore
contributes nothing to the A2A request, and the remote agent receives the *previous* content
event instead — in our case an earlier agent's output, which looked plausible and was silently
acted upon.

### Why this is worth a warning rather than only a doc note

The failure is silent, and the two directions fail differently:

- Direction 1 raises, if you happen to have a downstream check. If you don't, an `LlmAgent`
reading the missing key will often invent plausible content instead.
- Direction 2 does not raise at all. The remote agent gets *some* content and proceeds.

Because in-process runs and A2A runs are described as the same composition with a different
transport, the natural assumption is that the hand-off is also the same. Our test suite has 100%
statement and branch coverage and could not have caught either one — the state key is populated in
exactly the topology tests run in.

### Suggested remedies, in the order we'd value them

1. **Warn when a `SequentialAgent` contains a `RemoteA2aAgent` and a sibling declares `output_key`**
— the combination is almost always a mistake, and it is statically detectable at construction.
2. **Warn (or debug-log) when an event with a non-empty `state_delta` and no `content` is the last
event before a `RemoteA2aAgent` runs**, since that state is about to be dropped.
3. **Document the boundary explicitly** in the A2A guide: state is per-session, sessions are
per-agent across A2A, and only event content crosses.
4. Optionally, an opt-in `forward_state=` / `state_keys=` on `RemoteA2aAgent` for callers who want
specific keys serialized into the request.

We worked around it by having the caller read the remote agent's reply from session events
(sound in our case only because `output_schema` makes that reply validated JSON rather than
prose), and by emitting the hand-off as event content rather than as `state_delta`. Both are fine
once you know; the cost was entirely in not knowing.

Happy to open a PR for (1) and (3) if that's a direction you'd take.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.