JetBrains / JetBrains/koog

onToolCalls with filter breaks parallel tools

Open
#2,153 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Kotlin
Stars
4.6k
Forks
474
Avg merge
58m
Merged PRs (30d)
1

Description

When `onToolCalls` is used in strategy with a filter different than `{true}`, all tools which don't pass filter are dropped, even if there is a catch-all unconditional edge.

A conditional edge built with onToolCalls { predicate } is meant to route an assistant message to a node when at least one tool call matches the predicate. Instead it also mutates the payload: it forwards only the tool calls that match the predicate and silently discards the rest. When the LLM emits several different tool calls in a single turn (parallel / fan-out tool calling), every non-matching call is dropped and never executed, leaving the conversation with tool calls that have no corresponding tool result.

For Azure OpenAI, this results with error and conversation is terminated:

```
Error from client: Message: Expected status code 200 but was 400 Status code:
400 Error body: {"status":"failure","message":"azure-openai error: An assistant
message with 'tool_calls' must be followed by tool messages responding to
each 'tool_call_id'. The following tool_call_ids did not have response
messages: call_FSj9aB63JsQRAKn9OM9WAL5h","error":{"message":
"azure-openai error: An assistant message with 'tool_calls' must be
followed by tool messages responding to each 'tool_call_id'. The following
tool_call_ids did not have response messages: call_FSj9aB63JsQRAKn9OM9WAL5h","type":"APIError","code":"400"},
"error_origin_level":"api_error","provider":"azure-openai"}
```

```mermaid
---
title: buggy
---
stateDiagram
state "callLLM" as callLLM
state "sideLogic" as sideLogic
state "executeTools" as executeTools
state "sendResults" as sendResults

[*] --> callLLM
callLLM --> sideLogic : transformed
callLLM --> executeTools : transformed
callLLM --> [*] : transformed
sideLogic --> executeTools
executeTools --> sendResults
sendResults --> executeTools : transformed
sendResults --> [*] : transformed
```

Workaround is to always go to sideLogic and add condition inside.

```mermaid
---
title: workaround
---
stateDiagram
state "callLLM" as callLLM
state "sideLogic" as sideLogic
state "executeTools" as executeTools
state "sendResults" as sendResults

[*] --> callLLM
callLLM --> sideLogic : transformed
callLLM --> [*] : transformed
sideLogic --> executeTools
executeTools --> sendResults
sendResults --> sideLogic : transformed
sendResults --> [*] : transformed
```
## Scenario

LLM requests call tools: x, y

## Expected

- x passes through sideLogic and then to executeTools
- y directly to executeTools

## Actual

- x passes through sideLogic and then to executeTools (correct)
- y tool call is dropped.

Contributor guide

Open the contributing guide

Research direction

Start at the strategy implementation of onToolCalls and trace how filtered tool-call payloads are forwarded when multiple calls are present. Reproduce the x/y scenario with a matching conditional edge and an unconditional catch-all; done means both tool calls reach execution and each receives a tool result.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.