a2ui-project / a2ui-project/a2ui

We need a way to show an agent what a surface is displaying

Open
#646 6 comments 0 reactions 0 assignees View on GitHub
P2 type: feature/enhancement
Dominant language
TypeScript
Stars
16.4k
Forks
1.3k
Avg merge
2d 13h
Merged PRs (30d)
134

Description

In order for the user to query the agent about things they see (e.g. "Tell me about the one on the left"), the agent needs to be able to understand the layout and content of the surface.

The agent has no way to query this at the moment, and although we do have a way to send the current data model with every A2A message, we don't have any way to do the same with the UI tree.

One proposal might be to introduce a new A2UI message `querySurface` that allows the agent to get the current state of the surface, both the UI tree and the data model.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.