a2ui-project / a2ui-project/a2ui

Ability to declare client-side event handling where one component can affect another component

Ouverte
#302 8 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
P2 status: needs-team-input status: needs-triage type: feature/enhancement
Langage dominant
TypeScript
Étoiles
16.4k
Forks
1.3k
Merge moyen
2 j 13 h
PR mergées (30 j)
134

Description

Since I, in parallel, worked on a (much smaller) project that also returns agent-generated UI specs based on user prompts, I want to state the following proposal for discussion (and possibly refinement, if not outright-rejected). I purposely keep it relatively vague without going into implementation details in order to keep it short & the discussion on a high level initially, since it may introduce principle target use case incompatibilities with A2UI and thus cause a "desk reject", not warranting deeper investigation.

# Use Case

As a _developer_, I want to be able to enable users to "live" preview the results of their UI interactions without requiring re-prompting the LLM for efficiency and UX, at least for simple transformations/calculations.

As a _user_, I want to be able to make fine-adjustments within the generated UI and receive feedback on them without delays. E.g. in an LLM-based color manipulator.

```
1. Initial prompt: "dark red"
2. Output: Multi-choice selector with color previews of 3 "dark red" variants

3. Second prompt: "make it more saturated"
4. Output: Slider with saturation adjustment and live preview of saturation change of the chosen color (variant A) // <-- this requires either a specialized component, **or a generated callback function that is executed on the client-side**

```

# Current limitations

The current protocol is limited to pre-defined components with built-in business logic (e.g. color saturation change of color in the above example) limiting user-defined actions to strongly pre-defined set OR a whole server<->client roundtrip with A2A messages etc. introducing potentially significant delays, especially as soon as the LLM is re-involved.

# Extension Concept

1. Enable the (LLM) agent to generate expressions or functions with widget-specific parameters (e.g. slider min, max, position) by introducing the required protocol extensions, e.g. based on a python slang or pure math expressions
2. The functions are executed as a callback to the respective UI component's action (e.g. a slider and a slider position update) client-sided in a sandboxed logic interpreter / expression evaluator, such as python's `asteval` or `simpleeval` or typescript `filtrex` or similar
3. The respective UI component is updated purely client-sided with the function result (e.g. the color preview)

## Benefits

This allows for much more flexible UIs with instant, client-side-computed previews, etc. which are particularly important for fine-tuning results, such as small percentage adjustments in relative changes (for colors: hue, saturation, vibrance, ..) or non-conforming input formats (e.g. a color preview requiring hex input but the user providing rgba in floating point).

## Potential Risks
1. Security issues: expression/script evaluators come with the risk of suffering from security issues, e.g. `expr-eval` with [CVE-2025-12735](https://nvd.nist.gov/vuln/detail/CVE-2025-12735)
2. Incompatibility with the target use case of the A2UI protocol
3. ...?

---

I am looking forward to your feedback, hopefully a discussion and will try to answer all potential questions.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.