[Tech Debt][API][Chat Model] Review the chat model invocation flow in custom Actions
- Dominant language
- Java
- Stars
- 452
- Forks
- 167
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 49
Description
### Search before asking
- [x] I searched the existing issues and did not find a duplicate.
### Description
This is a child issue of #1055.
Today, invoking a chat model from a custom Agent Action is expressed through the built-in event flow: one Action sends a `ChatRequestEvent`, the built-in chat model Action processes it, and another user Action receives the resulting `ChatResponseEvent`.
This event-oriented model is useful for advanced workflows, but making it the required path for ordinary chat model invocations introduces usability concerns:
- A single logical model invocation is split across multiple user Actions.
- Data that would otherwise be local to the invocation often needs to be stored in Agent memory and recovered by the response Action.
- Concurrent requests require users to correlate responses with request IDs and maintain additional mappings.
- Users need to understand the framework's built-in request/response protocol before they can perform a common operation.
- Success, failure, and continuation semantics are distributed across the request Action, built-in Action, and response Action.
We should review this user journey and clarify the public API boundary for invoking chat models from custom Actions.
#### Direction and constraints
The existing `ChatRequestEvent` / `ChatResponseEvent` API and the built-in event-processing flow should be retained as a supported event-oriented API. It remains valuable for workflows that need explicit event routing, parallel fan-out/fan-in, custom correlation, branching, or direct control over event-driven orchestration.
This issue intentionally does not prescribe a concrete alternative API. The exact API shape should be discussed separately and evaluated across Java, Python, and YAML-facing workflows. Any future refinement should preserve the behavior currently provided by the built-in flow, including model selection, retries and fallback, tool-call rounds, durable execution, structured output, metrics, tracing, and well-defined failure handling.
Compatibility and migration mechanisms are outside the scope of this issue.
#### Acceptance criteria
- The common chat model invocation journey and its usability problems are documented for Java and Python custom Actions.
- The existing Event API is retained and clearly positioned as a supported event-oriented API.
- Requirements and semantic constraints for any API refinement are agreed before a concrete API design is selected.
- Java, Python, and relevant YAML-facing behavior are considered consistently.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start by reviewing the ChatRequestEvent and ChatResponseEvent flow and documenting the common chat model invocation journey for Java and Python custom Actions. Consider the YAML-facing behavior and record the requirements and semantic constraints before selecting an API design; done means the existing Event API is clearly positioned and the acceptance criteria are agreed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python, yaml
- Domain
- api, backend-api-design
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100