google / google/adk-java

SetModelResponseTool: schema validation failure terminates the tool flow instead of returning feedback

Open
#1,430 3 comments 0 reactions 1 assignee Claimed by @hemasekhar-p View on GitHub
needs review
Dominant language
Java
Stars
1.7k
Forks
420
Avg merge
4d 12h
Merged PRs (30d)
31

Description

## 🔴 Required Information

### Describe the Bug

`SetModelResponseTool` currently propagates output-schema validation failures as tool execution errors.

In the current Java implementation, `SchemaUtils.validateMapOnSchema(...)` is executed inside `Single.fromCallable(...)`. When validation throws an `IllegalArgumentException`, the `Single` terminates with an error instead of returning validation feedback as a tool response.

Under the default tool execution flow, no validation-feedback `FunctionResponse` is returned to the model. As a result, the model cannot correct the arguments and call `set_model_response` again.

ADK Python now returns structured-output validation failures to the model as tool feedback and records successfully validated output separately. This appears to be a Java/Python parity gap.

There is also a related finalization concern: `OutputSchema.getStructuredModelResponse(...)` currently promotes a function response named `set_model_response` to the final structured response. Validation feedback must therefore be distinguishable from successfully validated output.

### Steps to Reproduce

Create an output schema containing a required field:

```java
Schema outputSchema =
Schema.builder()
.type("OBJECT")
.properties(
ImmutableMap.of(
"field1",
Schema.builder().type("STRING").build()))
.required(ImmutableList.of("field1"))
.build();
```

Create and invoke `SetModelResponseTool` without the required field:

```java
SetModelResponseTool tool =
new SetModelResponseTool(outputSchema);

Map invalidArgs = ImmutableMap.of();

tool.runAsync(invalidArgs, null).blockingGet();
```

The call terminates with:

```text
java.lang.IllegalArgumentException:
Output args does not contain required field1
```

### Expected Behavior

For parity with ADK Python, an output-schema validation failure from the internal `set_model_response` tool should be returned to the model as validation feedback:

```text
LLM calls set_model_response

schema validation fails

validation feedback is returned to the model

LLM corrects the arguments

LLM calls set_model_response again

validated result becomes the final response
```

An invalid `set_model_response` call must not become the final structured model response. Only successfully validated output should be finalized.

The exact Java implementation does not need to identify validation feedback solely through an `"error"` field, since a valid user-defined output schema could legitimately contain a field with that name.

### Observed Behavior

`SetModelResponseTool` currently performs:

```java
return Single.fromCallable(
() -> {
SchemaUtils.validateMapOnSchema(
args, outputSchema, /* isInput= */ false);
return args;
});
```

The validation exception therefore becomes an RxJava error signal and propagates through the tool flow.

The existing upstream `SetModelResponseToolTest` explicitly verifies this behavior using `assertThrows(...)`, confirming that exception propagation is the current expected Java behavior.

Additionally, `OutputSchema.getStructuredModelResponse(...)` currently identifies the final structured response using the `set_model_response` function name. A change that only converts the exception into a Python-style feedback map could therefore incorrectly promote that feedback to the final response unless validated output is tracked or distinguished separately.

### Environment Details

- ADK Java version: commit `2b87d65d9704a61ff4668b8c9482a79fef9fe0d4`
- Java: Microsoft OpenJDK 17.0.15
- OS: Windows 11
- Test framework: JUnit 4

### Model Information

- Model: N/A
- The issue is reproducible deterministically at the tool level without a live model

---

## 🟡 Optional Information

### Regression

No known Java regression.

This appears to be a behavioral parity gap following the newer structured-output validation recovery behavior in ADK Python.

### Logs

```text
IllegalArgumentException: Output args does not contain required field1
at com.google.adk.SchemaUtils.validateMapOnSchema(SchemaUtils.java:124)
at com.google.adk.tools.SetModelResponseTool.lambda$runAsync$0(
SetModelResponseTool.java:63)
```

### Additional Context

The corresponding Python behavior is present on `adk-python` main in:

https://github.com/google/adk-python/commit/b5b27cb074d00f2e6889098514d8f716bc1cdfad

The change originated from:

https://github.com/google/adk-python/pull/6368

The Python implementation catches structured-output validation failures and returns validation feedback to the model. Successfully validated output is recorded separately and is the only result promoted to the final structured response.

Relevant Java files:

- `core/src/main/java/com/google/adk/tools/SetModelResponseTool.java`
- `core/src/main/java/com/google/adk/flows/llmflows/OutputSchema.java`
- `core/src/test/java/com/google/adk/tools/SetModelResponseToolTest.java`

The ADK Java contribution guide states that ADK Java aims to align with ADK Python and leans on ADK Python as the source of truth for behavioral validation:

https://github.com/google/adk-java/blob/2b87d65d9704a61ff4668b8c9482a79fef9fe0d4/CONTRIBUTING.md#alignment-with-adk-python

Would maintainers agree that ADK Java should align with the current Python recovery behavior here?

### How often has this issue occurred?

Always (100%) when `SetModelResponseTool` receives arguments that fail output-schema validation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.