apache / apache/flink-agents

[Tech Debt][API][ChatMessage] Review ChatMessage responsibilities and data model

Open
#1,056 0 comments 0 reactions 0 assignees View on GitHub
fixVersion/0.4.0 priority/major tech debt
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 9h
Merged PRs (30d)
49

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.

### Description

`ChatMessage` is a public contract shared by the Java and Python APIs, model integrations, runtime actions, Event Log serialization, state, and the pemja bridge. As these areas have evolved, the class has accumulated responsibilities that are not clearly separated:

- `content` is limited to text, while provider messages may contain richer, ordered content. This is also being discussed in #1031.
- `tool_calls` is represented as untyped maps whose required keys and value shapes are enforced by convention in provider and runtime code.
- `extra_args` is used for several unrelated purposes, including provider-native message data, tool-call correlation, token usage, refusal/reasoning details, structured output, and routing metadata.
- Provider connections store response-scoped values such as `model_name`, `promptTokens`, `completionTokens`, and `response_id` in `ChatMessage.extra_args`. `ChatModelAction` later reads the same map to record metrics and also attaches values such as `structured_output` and `model_routing`. Message data and per-invocation results therefore share the same untyped container.
- There are concrete Java/Python differences in this implicit contract. Tool-result messages use `extraArgs["externalId"]` in Java but `extra_args["external_id"]` in Python. The Python OpenAI converter merges all `extra_args` into USER and ASSISTANT provider messages, whereas the Java converter only reads specific recognized keys. The same serialized map can therefore be interpreted differently depending on the execution path.
- A `ChatMessage` is serialized and reconstructed through several independent paths: Jackson/Pydantic models, nested Event Log event attributes, raw-map parsing in prompt adapters, and explicit field-by-field pemja conversions. `RoutingContext` also implements a custom deep copy because `extraArgs` and `toolCalls` have different copy behavior. Adding or changing a message field requires each path to be updated consistently; otherwise data may be dropped at a boundary or mutable state may be shared unexpectedly.

This makes it difficult to distinguish stable message semantics from provider-specific or runtime metadata, validate messages consistently, and evolve the API without adding more implicit conventions. Related provider-level parity problems are tracked in #936.

Before establishing the post-0.4 compatibility baseline, the responsibilities and public contract of `ChatMessage` should be reviewed across Java, Python, provider adapters, runtime call paths, and cross-language serialization.

This issue intentionally records the current problems only. The target data model, field names, compatibility impact, and implementation scope should be agreed separately before implementation.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with ChatMessage and trace its use through provider adapters, ChatModelAction, Event Log serialization, Jackson/Pydantic models, raw-map prompt adapters, pemja conversions, and RoutingContext copying. Compare the Java and Python contracts and the separate metadata paths. Done means an agreed post-0.4 compatibility baseline, target data model, compatibility impact, and implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
api, backend-api-design, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.