kestra-io / kestra-io/plugin-ai

Combining `tools` with `responseFormat` on Gemini is documented as unsupported but not validated

Open
#406 2 comments 0 reactions 1 assignee Claimed by @sh0723 View on GitHub
area/plugin good first issue
Dominant language
Java
Stars
9
Forks
24
Avg merge
1d 23h
Merged PRs (30d)
15

Description

On the Gemini provider it is documented that tools cannot be combined with `responseFormat` and that only one of the two should be configured. Currently a flow that sets both is accepted by the editor and at save time, and the failure happens at the provider:

```
Function calling with a response mime type: 'application/json' is unsupported
```

Neither `tools` nor `responseFormat` is named in that message, so the documented rule has to be found before the cause can be understood. The same error was reported in https://github.com/kestra-io/plugin-ai/issues/217, which was closed by documenting the constraint. On the Anthropic provider, a configured response format is already rejected before the call, so there is precedent for the check.

Minimal example:

```yaml
id: tools_and_response_format
namespace: company.team

tasks:
- id: agent
type: io.kestra.plugin.ai.agent.AIAgent
prompt: Which flow should be executed to restart the database?
provider:
type: io.kestra.plugin.ai.provider.GoogleGemini
modelName: gemini-3.5-flash
apiKey: "{{ secret('GEMINI_API_KEY') }}"
configuration:
responseFormat:
type: JSON
jsonSchema:
type: object
properties:
decision:
type: string
tools:
- type: io.kestra.plugin.ai.tool.KestraFlow
namespace: company.team
flowId: hello-world
description: Say hello
```

## To do

- A flow that configures both `tools` and `responseFormat` on a provider where the two are incompatible should be rejected with a message naming both properties and the provider.
- The rejection should happen at validation time so the conflict is visible in the editor before the flow is executed.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.