spring-projects / spring-projects/spring-ai

MessageWindowChatMemory truncation breaks tool_use/tool_result pairs (Anthropic 400)

Open
#5,940 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Summary

MessageWindowChatMemory performs naïve count-based head truncation when the message count exceeds maxMessages. The cut point is not aware of AssistantMessage (with tool_use blocks) ↔ ToolResponseMessage (with tool_result blocks) pairs. When the cut falls between such a pair, the assistant message is dropped but the tool result survives — and the next request to the provider fails because the conversation now starts with an orphan tool_result.

Reproducer

Anthropic's API enforces this strictly:

400 invalid_request_error
messages.0.content.0: unexpected tool_use_id found in tool_result blocks: toolu_bdrk_…. Each tool_result block must have a corresponding tool_use block in the previous message.

In our environment (Spring AI 2.0.0-M5, Anthropic Claude Sonnet, MessageWindowChatMemory with maxMessages=100, tool-heavy ReAct conversations) we hit this every few hours on long-lived threads. A single user turn often produces 4–10 tool calls, so 100 messages is reached after only ~15–20 user turns. Once the window starts sliding, any cut that lands inside a tool_use→tool_result pair corrupts the conversation.

A self-contained reproducer with MessageWindowChatMemory.builder().maxMessages(4).build():

// Pretend the LLM made a tool call in turn 1 and we kept appending.
chatMemory.add(conversationId, List.of(
    new UserMessage("u1"),
    AssistantMessage.builder().content("").toolCalls(List.of(toolCall("t1"))).build(),
    new ToolResponseMessage(List.of(new ToolResponse("t1", "tool_a", "result_a"))),
    AssistantMessage.builder().content("done").build(),
    new UserMessage("u2")));   // window slides; messages[0] is now ToolResponseMessage(tool_use_id=t1)
                               // but the AssistantMessage that contained t1 was dropped.

chatMemory.get(conversationId) returns a list whose head is a ToolResponseMessage referencing a tool_use that no longer exists in the history.

Why the bot's recovery isn't enough

Anthropic returns the 400 every request until the corrupt state is cleared. Our app catches BadRequestException and resets the entire conversation — which is annoying and loses context the user expects to be there. We can write our own pair-aware wrapper, but a fix at the framework layer would benefit every user.

Suggested fix

MessageWindowChatMemory (or a pair-aware sibling) should, after computing the window cut, advance the cut forward while messages[0] instanceof ToolResponseMessage (or while messages[0] is a tool_result-bearing message whose matching tool_use is no longer in the kept window). Equivalent forms:

  • Keep cutting from the head while the first kept message is a ToolResponseMessage.
  • Or: track tool_use_id → tool_use position and never cut between a tool_use and its matching tool_result.

Either approach prevents the orphan-tool_result corruption without changing the public API. The same idea applies to OpenAI's tool messages — same constraint, same wire-format requirement.

Related but distinct issues

  • #2101 — tool calls not persisted at all (different bug)
  • #2529 — tool calls duplicated in memory (different bug)
  • #3366 — streaming mode loses toolCalls on AssistantMessage (different bug)
  • #4315 — tool messages not stored when using .stream.chatClientResponse() (different bug)

This issue is specifically about truncation in the persisted store breaking pairs.

Environment

  • Spring AI 2.0.0-M5
  • MessageWindowChatMemory + RedisChatMemoryRepository (Redis JSON)
  • Anthropic Claude Sonnet 4.6
  • Java 25, Spring Boot 4.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the MessageWindowChatMemory implementation and the maxMessages reproducer described in the issue. Trace how the window cut is applied, then add coverage showing that get(conversationId) never begins with an orphan ToolResponseMessage and that tool_use/tool_result pairs remain intact after truncation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, redis, spring
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.