a2ui-project / a2ui-project/a2ui

generate_system_prompt() conflicts with SendA2uiToClientToolset — injects <a2ui-json> tag instructions causing non-deterministic tool use

オープン
#1,045 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る
component: agent_sdk P2 type: bug type: feature/enhancement
主要言語
TypeScript
スター
16.4k
フォーク
1.3k
平均マージ
2日 13時間
マージ済み PR(30日)
134

説明

**Severity:** Medium — affects all users of tool-based A2UI emission

---

## Summary

`A2uiSchemaManager.generate_system_prompt()` unconditionally injects `` tag workflow instructions into the system prompt, even when `include_schema=False`. When used alongside `SendA2uiToClientToolset` (the tool-based approach), this creates conflicting instructions:

- The system prompt tells the LLM: *"wrap A2UI JSON in `` tags"* (inline text output)
- The toolset tells the LLM: *"call `send_a2ui_json_to_client`"* (structured function call)

With Gemini 2.5 Flash, this causes **non-deterministic behavior** — the model randomly chooses between proper tool use, malformed code-style output, or inline text with tags.

## Reproduction

### Setup

```python
# agent.py
from google.adk.agents.llm_agent import LlmAgent
from a2ui.core.schema.manager import A2uiSchemaManager
from a2ui.basic_catalog.provider import BasicCatalog
from a2ui.core.schema.constants import VERSION_0_8
from a2ui.adk.a2a_extension.send_a2ui_to_client_toolset import SendA2uiToClientToolset

schema_manager = A2uiSchemaManager(
version=VERSION_0_8,
catalogs=[BasicCatalog.get_config(
version=VERSION_0_8,
examples_path="examples/standard",
)],
)
selected_catalog = schema_manager.get_selected_catalog()
examples = schema_manager.load_examples(selected_catalog, validate=False)

# This injects tag instructions — even with include_schema=False
instruction = schema_manager.generate_system_prompt(
role_description="You are a helpful assistant.",
include_schema=False,
include_examples=False,
)

root_agent = LlmAgent(
model="gemini-2.5-flash",
name="my_agent",
instruction=instruction,
tools=[
SendA2uiToClientToolset(
a2ui_enabled=True,
a2ui_catalog=selected_catalog,
a2ui_examples=examples or "",
),
],
)
```

Run with `adk api_server` and send via `/run_sse` with `streaming: true`.

### What gets injected

Even with `include_schema=False, include_examples=False`, `generate_system_prompt()` always adds:

```
The generated response MUST follow these rules:
- The response can contain one or more A2UI JSON blocks.
- Each A2UI JSON block MUST be wrapped in `` and `` tags.
- Between or around these blocks, you can provide conversational text.
- The JSON part MUST be a single, raw JSON object...
```

This directly conflicts with `SendA2uiToClientToolset`, which registers `send_a2ui_json_to_client` as a structured function call.

### Observed behavior

Sending "Create a contact card for Alice, email alice@test.com" produces three different outcomes non-deterministically:

**1. Proper tool call (desired)** — ~33% of the time with `gemini-2.5-flash`:
```
functionCall: { name: "send_a2ui_json_to_client", args: { a2ui_json: "[...]" } }
```

**2. Malformed code-style call** — ~20%:
```
Malformed function call: print(default_api.send_a2ui_json_to_client(a2ui_json='[...]'))
```

**3. Inline text with `` tags** — ~40%:
```
Here is the contact card:

[{"beginRendering": ...}, {"surfaceUpdate": ...}]

```

### Test results

We ran systematic tests across models and `FunctionCallingConfigMode`:

| Model | FunctionCallingConfigMode | Success Rate | Failures |
|-------|--------------------------|-------------|----------|
| gemini-2.5-flash | AUTO (default) | 1/3 (33%) | 1 malformed, 1 inline |
| gemini-2.5-flash | ANY | 3/3 (100%) | — |
| gemini-2.5-pro | AUTO (default) | 2/3 (67%) | 1 unknown |
| gemini-2.5-pro | ANY | 3/3 (100%) | — |

## Current workarounds

### Workaround 1: Skip `generate_system_prompt` entirely

Write the instruction directly. `SendA2uiToClientToolset.process_llm_request()` injects schema and examples automatically — `generate_system_prompt` is redundant when using the toolset.

```python
# Instead of generate_system_prompt(), write directly:
instruction = """You are a helpful assistant.
When visual presentation helps, call the send_a2ui_json_to_client tool.
For simple questions, respond with text only."""

# Result: 15/15 success with gemini-2.5-flash + streaming=true
```

### Workaround 2: Force `FunctionCallingConfigMode.ANY`

```python
from google.genai import types

root_agent = LlmAgent(
model="gemini-2.5-flash",
instruction=instruction,
generate_content_config=types.GenerateContentConfig(
tool_config=types.ToolConfig(
function_calling_config=types.FunctionCallingConfig(
mode=types.FunctionCallingConfigMode.ANY,
)
)
),
tools=[SendA2uiToClientToolset(...)],
)
# Result: 3/3 success even with the large 52k prompt
```

**Drawback of `ANY` mode:** The model *must* call a tool on every turn. Simple text queries like "What is 2+2?" will still trigger a `send_a2ui_json_to_client` call instead of responding with plain text. This breaks the natural mixed-mode behavior where the agent decides whether UI is appropriate.

## Suggested fix

### Option A: Add `include_workflow` parameter

```python
instruction = schema_manager.generate_system_prompt(
role_description="You are a helpful assistant.",
include_schema=False,
include_examples=False,
include_workflow=False, # NEW: skip tag instructions
)
```

### Option B: Auto-detect tool-based usage

If `SendA2uiToClientToolset` is registered, `generate_system_prompt` should not inject the `` tag workflow since the toolset handles prompt injection via `process_llm_request`.

### Option C: Document incompatibility

At minimum, document that `generate_system_prompt()` should NOT be used with `SendA2uiToClientToolset`. The rizzcharts sample works around this by using a strong `WORKFLOW_DESCRIPTION` that overrides the injected tag instructions, but this is fragile and non-obvious.

## Note on the rizzcharts sample

The `rizzcharts` sample in this repo uses both `generate_system_prompt()` and `SendA2uiToClientToolset` together. It works because:

1. Its `WORKFLOW_DESCRIPTION` explicitly says "Call the `send_a2ui_json_to_client` tool" — overriding the tag instructions
2. It uses `BuiltInPlanner` with `ThinkingConfig(include_thoughts=True)` which may improve function call reliability
3. It may be tested with models or configurations that handle the conflict better

However, following the same pattern without the strong workflow override produces the non-deterministic behavior described above.

## Environment

- `google-adk==1.28.0`
- `a2ui-agent==0.1.0` (from source, `agent_sdks/python`)
- `gemini-2.5-flash` and `gemini-2.5-pro` via Vertex AI
- Python 3.14, macOS

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Look at a2ui/core/schema/manager.py for the generate_system_prompt method and a2ui/adk/a2a_extension/send_a2ui_to_client_toolset.py for the toolset. The conflict is in the injected instructions. Test with the provided reproduction script to see the non-deterministic behavior. The fix likely involves modifying generate_system_prompt to conditionally skip the tag instructions when used with the toolset.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python, typescript
領域
backend-api-design, tooling
issue の種類
バグ
難易度
3/5
見積もり時間
1〜2日
活発さ
静か
明瞭さ
明確に書かれている
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。