apache / apache/flink-agents

[Tech Debt][API][Chat Model] Review the chat model invocation flow in custom Actions

Open
#1,086 0 comments 0 reactions 0 assignees Claimed by @wenjin272 View on GitHub
fixVersion/0.4.0 priority/major tech debt
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 9h
Merged PRs (30d)
49

Description

### Search before asking

- [x] I searched the existing issues and did not find a duplicate.

### Description

This is a child issue of #1055.

Today, invoking a chat model from a custom Agent Action is expressed through the built-in event flow: one Action sends a `ChatRequestEvent`, the built-in chat model Action processes it, and another user Action receives the resulting `ChatResponseEvent`.

This event-oriented model is useful for advanced workflows, but making it the required path for ordinary chat model invocations introduces usability concerns:

- A single logical model invocation is split across multiple user Actions.
- Data that would otherwise be local to the invocation often needs to be stored in Agent memory and recovered by the response Action.
- Concurrent requests require users to correlate responses with request IDs and maintain additional mappings.
- Users need to understand the framework's built-in request/response protocol before they can perform a common operation.
- Success, failure, and continuation semantics are distributed across the request Action, built-in Action, and response Action.

We should review this user journey and clarify the public API boundary for invoking chat models from custom Actions.

#### Direction and constraints

The existing `ChatRequestEvent` / `ChatResponseEvent` API and the built-in event-processing flow should be retained as a supported event-oriented API. It remains valuable for workflows that need explicit event routing, parallel fan-out/fan-in, custom correlation, branching, or direct control over event-driven orchestration.

This issue intentionally does not prescribe a concrete alternative API. The exact API shape should be discussed separately and evaluated across Java, Python, and YAML-facing workflows. Any future refinement should preserve the behavior currently provided by the built-in flow, including model selection, retries and fallback, tool-call rounds, durable execution, structured output, metrics, tracing, and well-defined failure handling.

Compatibility and migration mechanisms are outside the scope of this issue.

#### Acceptance criteria

- The common chat model invocation journey and its usability problems are documented for Java and Python custom Actions.
- The existing Event API is retained and clearly positioned as a supported event-oriented API.
- Requirements and semantic constraints for any API refinement are agreed before a concrete API design is selected.
- Java, Python, and relevant YAML-facing behavior are considered consistently.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ChatRequestEvent and ChatResponseEvent flow and documenting the common chat model invocation journey for Java and Python custom Actions. Consider the YAML-facing behavior and record the requirements and semantic constraints before selecting an API design; done means the existing Event API is clearly positioned and the acceptance criteria are agreed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python, yaml
Domain
api, backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.