Unable to build a proper ChatMemory including ToolCalls requests and responses.

As Spring AI does not store tool calls and results in the ChatMemory a couple of solutions were proposed here.

I tried solution 1 but I land up with duplicate messages in the ChatMemory no matter where/how I add the messages to the memory. I suspect this only shows up if multiple tool calls happen as part of the same user prompt.

Let's take a simple example of Authors and Books. We want to get the books written by an author. This results in 2 tool calls. First, getAuthor is called to get the author id and then getBooks is called using author id.

Prompt: List the books by Dan Brown
Tools: [getAuthor(name), getBooks(authorId)]

When the response is complete the memory should look like this so follow up questions can be asked without having the tool calls repeated.

UserMessage [List books by Dan Brown]
AssistantToolCall[getAuthor(name = Dan Brown)]
AssistantToolResponse[623]
AssistantToolCall[getBooks(authorId = 623)
AssistantToolResponse[[{
“name”: "Angels and Daemons”,
“publishedOn”: date,
“copiesSold”: “3 million”
“plot”: "...", 
 }, {"name": “Da Vinci Code”, "publishedOn": ... }]

I have written my own ChatMemoryAdvisor which adds AssistantMessages to the ChatMemory as below.

    private void observeAfter(AdvisedResponse advisedResponse)
    {
        List<Message> assistantMessages = advisedResponse.response()
            .getResults()
            .stream()
            .map(g -> (Message) g.getOutput())
            .toList();

        this.getChatMemoryStore().add(this.doGetConversationId(advisedResponse.adviseContext()), assistantMessages);
    }

I also use the User-Controlled tool execution flow as below..

public ChatResponse chat(String userText) {

    ToolCallingChatOptions chatOptions = …; // built with internalToolExecutionEnabled = false
    Prompt prompt = new Prompt(userText, chatOptions);
    ChatResponse chatResponse = _chat(prompt, systemText); // Internal chat implementation so we can call it while loop below

    while (chatResponse.hasToolCalls()) {
        ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, chatResponse);

        ToolResponseMessage toolResponseMessage = (ToolResponseMessage) toolExecutionResult.conversationHistory().get(toolExecutionResult.conversationHistory().size() - 1);

        // UserMessage should be pushed before the following else sequence of messages won't be right
        chatMemory.add(memoryId, toolResponseMessage); // add result of tool call to Memory

        prompt = new Prompt(toolExecutionResult.conversationHistory(), chatOptions);

        chatResponse = _chat(toolResponseMessage.getMemory(), …);
    }
}

private ChatResponse _chat(Prompt prompt, String systemText) {
    // Can't add prompt to ChatMemory here because the userText itself will be a UserMessage finally and causes duplication.
    // also, _chat is called after each tool call. So that will again add duplication.
    return chatClient.prompt(prompt).system(systemText).advisors(…CustomChatMemoryAdvisor).call();
}

Environment
Spring AI : 1.0.0-SNAPSHOT
SpringBoot : 3.4.3
Java 21

Start with the custom ChatMemoryAdvisor's observeAfter method and the chat/_chat user-controlled tool execution flow shown in the issue. Trace how conversationHistory, tool responses, and assistant messages enter ChatMemory during multiple tool calls. Done means the memory preserves one ordered user message, each tool call, and each tool response without duplication, enabling follow-up questions without repeating the calls.