spring-projects / spring-projects/spring-ai

Tool-call limit breach text is not JSON-safe: bypasses `ToolExecutionExceptionProcessor`, and `GoogleGenAiChatModel` rejects it

Open
#6,902 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Spring AI 2.0.1, spring-ai-model + spring-ai-google-genai.

Summary

With spring.ai.tools.limits.on-limit-exceeded=RETURN_ERROR_RESPONSE, a breached per-tool or total limit is returned to the model as the raw ToolCallLimits.Breach.message() text. On Google GenAI that kills the whole call: GoogleGenAiChatModel JSON-parses every tool response and throws on plain text. The tool loop therefore ends with an exception instead of the model seeing "limit exceeded", which is the opposite of what RETURN_ERROR_RESPONSE promises.

Two independent faults contribute.

1. DefaultToolCallingManager bypasses ToolExecutionExceptionProcessor on a breach

DefaultToolCallingManager.executeToolCall (2.0.1, ~L248):

ToolCallLimits.Breach limitBreach = this.toolCallLimits.check(toolName, toolCallCount, totalToolCallCount);
if (limitBreach != null) {
    toolResponses.add(new ToolResponseMessage.ToolResponse(toolCall.id(), toolName, limitBreach.message()));
    ...
    continue;
}

Every other tool failure goes through toolExecutionExceptionProcessor.process(...), which is the documented hook for shaping error text for a provider (we use it to emit {"error": "..."} for Gemini). The limit breach is the one error path that writes bare text directly, so a processor configured for provider safety cannot reach it.

Suggested fix: route the breach through the processor, e.g. this.toolExecutionExceptionProcessor.process(new ToolExecutionException(toolDefinition, new ToolCallLimitExceededException(...))), or give ToolCallLimits a message-shaping hook.

2. GoogleGenAiChatModel.parseJsonToMap rejects non-JSON tool text

GoogleGenAiChatModel converts each ToolResponse.responseData() with parseJsonToMap: a JSON object is used as-is, a non-object JSON value is wrapped as {"result": value}, but text that is not JSON at all throws (Failed to parse JSON: Tool call limit (10) exceeded for tool 'x' ...).

Since the model already wraps non-object JSON, wrapping non-JSON text the same way ({"result": "<text>"}) would make every tool result Gemini-safe regardless of what produced it, including the breach message above and any custom ToolCallback returning plain text.

Reproduction

  • Google GenAI chat model, ChatClient with tools, spring.ai.tools.limits.max-calls-per-tool-default=1, on-limit-exceeded=RETURN_ERROR_RESPONSE.
  • Model issues two calls to the same tool in one round (or two rounds).
  • Second call → ToolResponse(id, name, "Tool call limit (1) exceeded for tool '...'...") → next model call fails with IllegalStateException: Failed to parse JSON.

THROW is worse for us: ToolCallLimitExceededException short-circuits past our session-memory advisor, leaving a dangling ASSISTANT(functionCall) in the persisted transcript that Gemini rejects on every later turn.

Workaround

A CallAdvisor inside the tool loop (order between ToolCallingAdvisor and any memory advisor) that rewrites every ToolResponseMessage response whose responseData is not valid JSON into {"error": "<text>"} before the request reaches the model:

static final class GeminiToolTextAdvisor implements CallAdvisor {
  private static final JsonHelper JSON = new JsonHelper();
  public int getOrder() { return Ordered.HIGHEST_PRECEDENCE + 350; }
  public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
    var messages = request.prompt().getInstructions();
    return switch (messages.getLast()) {
      case ToolResponseMessage tool -> {
        var responses = tool.getResponses().stream().map(GeminiToolTextAdvisor::mkJsonResponse).toList();
        var fixed = ToolResponseMessage.builder().responses(responses).metadata(tool.getMetadata()).build();
        var all = new ArrayList<>(messages);
        all.set(all.size() - 1, fixed);
        var prompt = request.prompt().mutate().messages(all).build();
        yield chain.nextCall(request.mutate().prompt(prompt).build());
      }
      default -> chain.nextCall(request);
    };
  }
  private static ToolResponse mkJsonResponse(ToolResponse r) {
    return isJson(r.responseData()) ? r : new ToolResponse(r.id(), r.name(), JSON.toJson(Map.of("error", r.responseData())));
  }
  // every MethodToolCallback result is JSON (records, lists, quoted strings); only the breach text is bare
  private static boolean isJson(String text) { return !text.isEmpty() && "{[\"".indexOf(text.charAt(0)) >= 0; }
}

Either fix upstream (1 alone, or 2 alone) removes the need for it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading DefaultToolCallingManager.executeToolCall and GoogleGenAiChatModel.parseJsonToMap, then run the described Google GenAI reproduction with RETURN_ERROR_RESPONSE and a per-tool limit of one. Done means the second tool call reaches the model as a JSON-safe error response instead of causing a JSON parse failure, with the relevant behavior covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, java, spring
Domain
ai, api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.