[Tech Debt][API][ChatMessage] Review ChatMessage responsibilities and data model
- Dominant language
- Java
- Stars
- 452
- Forks
- 167
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 49
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.
### Description
`ChatMessage` is a public contract shared by the Java and Python APIs, model integrations, runtime actions, Event Log serialization, state, and the pemja bridge. As these areas have evolved, the class has accumulated responsibilities that are not clearly separated:
- `content` is limited to text, while provider messages may contain richer, ordered content. This is also being discussed in #1031.
- `tool_calls` is represented as untyped maps whose required keys and value shapes are enforced by convention in provider and runtime code.
- `extra_args` is used for several unrelated purposes, including provider-native message data, tool-call correlation, token usage, refusal/reasoning details, structured output, and routing metadata.
- Provider connections store response-scoped values such as `model_name`, `promptTokens`, `completionTokens`, and `response_id` in `ChatMessage.extra_args`. `ChatModelAction` later reads the same map to record metrics and also attaches values such as `structured_output` and `model_routing`. Message data and per-invocation results therefore share the same untyped container.
- There are concrete Java/Python differences in this implicit contract. Tool-result messages use `extraArgs["externalId"]` in Java but `extra_args["external_id"]` in Python. The Python OpenAI converter merges all `extra_args` into USER and ASSISTANT provider messages, whereas the Java converter only reads specific recognized keys. The same serialized map can therefore be interpreted differently depending on the execution path.
- A `ChatMessage` is serialized and reconstructed through several independent paths: Jackson/Pydantic models, nested Event Log event attributes, raw-map parsing in prompt adapters, and explicit field-by-field pemja conversions. `RoutingContext` also implements a custom deep copy because `extraArgs` and `toolCalls` have different copy behavior. Adding or changing a message field requires each path to be updated consistently; otherwise data may be dropped at a boundary or mutable state may be shared unexpectedly.
This makes it difficult to distinguish stable message semantics from provider-specific or runtime metadata, validate messages consistently, and evolve the API without adding more implicit conventions. Related provider-level parity problems are tracked in #936.
Before establishing the post-0.4 compatibility baseline, the responsibilities and public contract of `ChatMessage` should be reviewed across Java, Python, provider adapters, runtime call paths, and cross-language serialization.
This issue intentionally records the current problems only. The target data model, field names, compatibility impact, and implementation scope should be agreed separately before implementation.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start with ChatMessage and trace its use through provider adapters, ChatModelAction, Event Log serialization, Jackson/Pydantic models, raw-map prompt adapters, pemja conversions, and RoutingContext copying. Compare the Java and Python contracts and the separate metadata paths. Done means an agreed post-0.4 compatibility baseline, target data model, compatibility impact, and implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- api, backend-api-design, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100