SetModelResponseTool: schema validation failure terminates the tool flow instead of returning feedback
- Dominant language
- Java
- Stars
- 1.7k
- Forks
- 420
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 31
Description
## 🔴 Required Information
### Describe the Bug
`SetModelResponseTool` currently propagates output-schema validation failures as tool execution errors.
In the current Java implementation, `SchemaUtils.validateMapOnSchema(...)` is executed inside `Single.fromCallable(...)`. When validation throws an `IllegalArgumentException`, the `Single` terminates with an error instead of returning validation feedback as a tool response.
Under the default tool execution flow, no validation-feedback `FunctionResponse` is returned to the model. As a result, the model cannot correct the arguments and call `set_model_response` again.
ADK Python now returns structured-output validation failures to the model as tool feedback and records successfully validated output separately. This appears to be a Java/Python parity gap.
There is also a related finalization concern: `OutputSchema.getStructuredModelResponse(...)` currently promotes a function response named `set_model_response` to the final structured response. Validation feedback must therefore be distinguishable from successfully validated output.
### Steps to Reproduce
Create an output schema containing a required field:
```java
Schema outputSchema =
Schema.builder()
.type("OBJECT")
.properties(
ImmutableMap.of(
"field1",
Schema.builder().type("STRING").build()))
.required(ImmutableList.of("field1"))
.build();
```
Create and invoke `SetModelResponseTool` without the required field:
```java
SetModelResponseTool tool =
new SetModelResponseTool(outputSchema);
Map invalidArgs = ImmutableMap.of();
tool.runAsync(invalidArgs, null).blockingGet();
```
The call terminates with:
```text
java.lang.IllegalArgumentException:
Output args does not contain required field1
```
### Expected Behavior
For parity with ADK Python, an output-schema validation failure from the internal `set_model_response` tool should be returned to the model as validation feedback:
```text
LLM calls set_model_response
↓
schema validation fails
↓
validation feedback is returned to the model
↓
LLM corrects the arguments
↓
LLM calls set_model_response again
↓
validated result becomes the final response
```
An invalid `set_model_response` call must not become the final structured model response. Only successfully validated output should be finalized.
The exact Java implementation does not need to identify validation feedback solely through an `"error"` field, since a valid user-defined output schema could legitimately contain a field with that name.
### Observed Behavior
`SetModelResponseTool` currently performs:
```java
return Single.fromCallable(
() -> {
SchemaUtils.validateMapOnSchema(
args, outputSchema, /* isInput= */ false);
return args;
});
```
The validation exception therefore becomes an RxJava error signal and propagates through the tool flow.
The existing upstream `SetModelResponseToolTest` explicitly verifies this behavior using `assertThrows(...)`, confirming that exception propagation is the current expected Java behavior.
Additionally, `OutputSchema.getStructuredModelResponse(...)` currently identifies the final structured response using the `set_model_response` function name. A change that only converts the exception into a Python-style feedback map could therefore incorrectly promote that feedback to the final response unless validated output is tracked or distinguished separately.
### Environment Details
- ADK Java version: commit `2b87d65d9704a61ff4668b8c9482a79fef9fe0d4`
- Java: Microsoft OpenJDK 17.0.15
- OS: Windows 11
- Test framework: JUnit 4
### Model Information
- Model: N/A
- The issue is reproducible deterministically at the tool level without a live model
---
## 🟡 Optional Information
### Regression
No known Java regression.
This appears to be a behavioral parity gap following the newer structured-output validation recovery behavior in ADK Python.
### Logs
```text
IllegalArgumentException: Output args does not contain required field1
at com.google.adk.SchemaUtils.validateMapOnSchema(SchemaUtils.java:124)
at com.google.adk.tools.SetModelResponseTool.lambda$runAsync$0(
SetModelResponseTool.java:63)
```
### Additional Context
The corresponding Python behavior is present on `adk-python` main in:
https://github.com/google/adk-python/commit/b5b27cb074d00f2e6889098514d8f716bc1cdfad
The change originated from:
https://github.com/google/adk-python/pull/6368
The Python implementation catches structured-output validation failures and returns validation feedback to the model. Successfully validated output is recorded separately and is the only result promoted to the final structured response.
Relevant Java files:
- `core/src/main/java/com/google/adk/tools/SetModelResponseTool.java`
- `core/src/main/java/com/google/adk/flows/llmflows/OutputSchema.java`
- `core/src/test/java/com/google/adk/tools/SetModelResponseToolTest.java`
The ADK Java contribution guide states that ADK Java aims to align with ADK Python and leans on ADK Python as the source of truth for behavioral validation:
https://github.com/google/adk-java/blob/2b87d65d9704a61ff4668b8c9482a79fef9fe0d4/CONTRIBUTING.md#alignment-with-adk-python
Would maintainers agree that ADK Java should align with the current Python recovery behavior here?
### How often has this issue occurred?
Always (100%) when `SetModelResponseTool` receives arguments that fail output-schema validation.
Contributor guide
Assessment
This issue has not been assessed yet.