a2ui-project / a2ui-project/a2ui

Ability to declare client-side event handling where one component can affect another component

Đang mở
#302 8 bình luận 0 reaction 0 người được giao Xem trên GitHub
P2 status: needs-team-input status: needs-triage type: feature/enhancement
Ngôn ngữ chính
TypeScript
Star
16.4k
Fork
1.3k
Merge trung bình
2 ngày 13 giờ
Pull request đã merge (30 ngày)
134

Mô tả

Since I, in parallel, worked on a (much smaller) project that also returns agent-generated UI specs based on user prompts, I want to state the following proposal for discussion (and possibly refinement, if not outright-rejected). I purposely keep it relatively vague without going into implementation details in order to keep it short & the discussion on a high level initially, since it may introduce principle target use case incompatibilities with A2UI and thus cause a "desk reject", not warranting deeper investigation.

# Use Case

As a _developer_, I want to be able to enable users to "live" preview the results of their UI interactions without requiring re-prompting the LLM for efficiency and UX, at least for simple transformations/calculations.

As a _user_, I want to be able to make fine-adjustments within the generated UI and receive feedback on them without delays. E.g. in an LLM-based color manipulator.

```
1. Initial prompt: "dark red"
2. Output: Multi-choice selector with color previews of 3 "dark red" variants

3. Second prompt: "make it more saturated"
4. Output: Slider with saturation adjustment and live preview of saturation change of the chosen color (variant A) // <-- this requires either a specialized component, **or a generated callback function that is executed on the client-side**

```

# Current limitations

The current protocol is limited to pre-defined components with built-in business logic (e.g. color saturation change of color in the above example) limiting user-defined actions to strongly pre-defined set OR a whole server<->client roundtrip with A2A messages etc. introducing potentially significant delays, especially as soon as the LLM is re-involved.

# Extension Concept

1. Enable the (LLM) agent to generate expressions or functions with widget-specific parameters (e.g. slider min, max, position) by introducing the required protocol extensions, e.g. based on a python slang or pure math expressions
2. The functions are executed as a callback to the respective UI component's action (e.g. a slider and a slider position update) client-sided in a sandboxed logic interpreter / expression evaluator, such as python's `asteval` or `simpleeval` or typescript `filtrex` or similar
3. The respective UI component is updated purely client-sided with the function result (e.g. the color preview)

## Benefits

This allows for much more flexible UIs with instant, client-side-computed previews, etc. which are particularly important for fine-tuning results, such as small percentage adjustments in relative changes (for colors: hue, saturation, vibrance, ..) or non-conforming input formats (e.g. a color preview requiring hex input but the user providing rgba in floating point).

## Potential Risks
1. Security issues: expression/script evaluators come with the risk of suffering from security issues, e.g. `expr-eval` with [CVE-2025-12735](https://nvd.nist.gov/vuln/detail/CVE-2025-12735)
2. Incompatibility with the target use case of the A2UI protocol
3. ...?

---

I am looking forward to your feedback, hopefully a discussion and will try to answer all potential questions.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.