a2ui-project / a2ui-project/a2ui
Ability to declare client-side event handling where one component can affect another component
- Ngôn ngữ chính
- TypeScript
- Star
- 16.4k
- Fork
- 1.3k
- Merge trung bình
- 2 ngày 13 giờ
- Pull request đã merge (30 ngày)
- 134
Mô tả
Since I, in parallel, worked on a (much smaller) project that also returns agent-generated UI specs based on user prompts, I want to state the following proposal for discussion (and possibly refinement, if not outright-rejected). I purposely keep it relatively vague without going into implementation details in order to keep it short & the discussion on a high level initially, since it may introduce principle target use case incompatibilities with A2UI and thus cause a "desk reject", not warranting deeper investigation.
# Use Case
As a _developer_, I want to be able to enable users to "live" preview the results of their UI interactions without requiring re-prompting the LLM for efficiency and UX, at least for simple transformations/calculations.
As a _user_, I want to be able to make fine-adjustments within the generated UI and receive feedback on them without delays. E.g. in an LLM-based color manipulator.
```
1. Initial prompt: "dark red"
2. Output: Multi-choice selector with color previews of 3 "dark red" variants
3. Second prompt: "make it more saturated"
4. Output: Slider with saturation adjustment and live preview of saturation change of the chosen color (variant A) // <-- this requires either a specialized component, **or a generated callback function that is executed on the client-side**
```
# Current limitations
The current protocol is limited to pre-defined components with built-in business logic (e.g. color saturation change of color in the above example) limiting user-defined actions to strongly pre-defined set OR a whole server<->client roundtrip with A2A messages etc. introducing potentially significant delays, especially as soon as the LLM is re-involved.
# Extension Concept
1. Enable the (LLM) agent to generate expressions or functions with widget-specific parameters (e.g. slider min, max, position) by introducing the required protocol extensions, e.g. based on a python slang or pure math expressions
2. The functions are executed as a callback to the respective UI component's action (e.g. a slider and a slider position update) client-sided in a sandboxed logic interpreter / expression evaluator, such as python's `asteval` or `simpleeval` or typescript `filtrex` or similar
3. The respective UI component is updated purely client-sided with the function result (e.g. the color preview)
## Benefits
This allows for much more flexible UIs with instant, client-side-computed previews, etc. which are particularly important for fine-tuning results, such as small percentage adjustments in relative changes (for colors: hue, saturation, vibrance, ..) or non-conforming input formats (e.g. a color preview requiring hex input but the user providing rgba in floating point).
## Potential Risks
1. Security issues: expression/script evaluators come with the risk of suffering from security issues, e.g. `expr-eval` with [CVE-2025-12735](https://nvd.nist.gov/vuln/detail/CVE-2025-12735)
2. Incompatibility with the target use case of the A2UI protocol
3. ...?
---
I am looking forward to your feedback, hopefully a discussion and will try to answer all potential questions.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.