apache / apache/flink-agents

[Tech Debt][API][Tool] Align Java and Python Tool outcome contracts

Closed
#956 0 comments 0 reactions 1 assignee Claimed by @joeyutong View on GitHub
fixVersion/0.4.0 tech debt
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 9h
Merged PRs (30d)
49

Description

### Search before asking

- [x] I searched the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.

### Description

## Background

Java and Python currently expose different Tool result contracts:

- Java `Tool.call` returns `ToolResponse`, which can explicitly represent success or failure without throwing an exception.
- Python `Tool.call` returns `Any`. An exception represents failure, while every normal return is treated as success because Python has no equivalent explicit Tool result type.

[PR #926](https://github.com/apache/flink-agents/pull/926) has introduced parallel Tool calls and the cross-language `Outcome` abstraction for durable batch execution. The resulting model has two distinct outcome layers:

```text
Outcome = whether the durable invocation returned or raised
ToolResponse = whether the Tool operation itself succeeded or failed
```

Java handles both layers for sequential and parallel calls: a successful `Outcome` may still contain an unsuccessful `ToolResponse`. Python can currently represent only the invocation layer, so every `Outcome.success(value)` is treated as a successful Tool operation.

This difference is now visible through several framework surfaces:

- `ToolResponseEvent.success` and `ToolResponseEvent.error`;
- Tool Execution Events and Agent Trace;
- Tool and MCP operational metrics proposed in [PR #955](https://github.com/apache/flink-agents/pull/955);
- Java/Python Tool bridges;
- MCP protocol-level error results.

There are also concrete built-in cases that cannot be represented consistently today. For example, Java and Python `load_skill` return normal success values when the Skill manager is unavailable, a Skill does not exist, or a requested Skill resource does not exist. These responses contain useful guidance for the model, but Trace and metrics classify them as successful Tool calls.

## Proposed direction

Define one Tool outcome contract across Java and Python while preserving compatibility for existing Python Tools:

- A normal raw Python return remains a successful Tool result.
- Both languages can explicitly return a successful or failed Tool result; Python exposes `ToolResponse.success(...)` and `ToolResponse.error(...)` for this purpose.
- An invocation exception remains a failed Tool result.
- Generic durable `Outcome` keeps its invocation-level meaning and does not absorb Tool-specific semantics.
- Sequential and parallel Tool-call paths normalize and record results through the same logic.
- Java/Python bridges preserve explicit Tool success, failure, and error details.
- `ToolResponseEvent`, Execution Events, and operational metrics consume the same normalized Tool outcome.
- `load_skill` missing-manager, missing-Skill, and missing-resource responses become explicit Tool failures while retaining their current model-facing messages.
- MCP protocol-level error results map to failed Tool outcomes instead of successful content.

The runtime must not infer failure by inspecting arbitrary Tool payloads such as strings or dictionaries. Aligning the wire representation of all existing `ToolResponseEvent.responses` payloads is not required for this change; existing event payload compatibility should be preserved.

## Expected validation

Cover native Java, native Python, and both cross-language directions for:

- normal raw success;
- explicit returned success and failure;
- thrown exception;
- sequential and parallel execution;
- `load_skill` failure responses;
- MCP protocol-level failure;
- consistent `ToolResponseEvent` and Execution Event outcomes.

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.