spring-projects / spring-projects/spring-ai

ToolCallingAdvisor (stream mode): advisorContext modifications inside recursive tool‑loop are lost, only last‑round context retained

Open
#6,903 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description
When using ToolCallingAdvisor in stream() mode with multi‑round recursive tool calls:
Modifications to advisorContext made by custom BaseAdvisor inside tool‑loop iterations are not propagated to the next recursive tool‑call iteration.
Only context changes from the last iteration are retained. All context list accumulations written in earlier loop iterations are lost silently.

The non‑stream .call() path works correctly, this was fixed by PR #5747.
But the streaming branch did NOT adopt the same context‑merge logic.

In streaming tool‑loop implementation, next request is built as:

advisorContext(new HashMap<>(originalRequest.advisorContext()))

It only snapshots advisorContext at the very beginning of the whole tool loop.
Context mutations / list accumulations which happen inside each loop iteration are not carried forward to subsequent recursion rounds.

Note: This is NOT the same as #6842.
#6842 fixes lost advisor‑injected Message objects. This bug is about lost state inside the advisorContext Map.

Environment

  • Spring AI version: 2.0.1
  • Java Version: JDK 21
  • No vector store required for reproduction.

Steps to reproduce

  1. Configure ChatClient with ToolCallingAdvisor and a custom BaseAdvisor (CompleteTrackingAdvisor).
  2. In advisor before() / after() hook, append track items into a List stored inside advisorContext key COMPLETE_TRACKS.
  3. Trigger LLM to produce multiple sequential tool‑calls, so recursive tool‑loop executes more than one iteration.
  4. Observe the COMPLETE_TRACKS list inside advisorContext: tracks added in earlier iterations disappear. Only items added in final iteration remain.

Expected behavior
In stream mode, modifications and list accumulations to advisorContext made in each tool‑loop iteration should be propagated across recursive tool‑call iterations, consistent with existing .call() behavior.
State written in previous rounds should be visible for advisors in next tool‑call round.

Minimal Complete Reproducible example

Context utility class:

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.UUID;

public class AgentAdvisorContextOperator {
    public static final String COMPLETE_TRACKS = "COMPLETE_TRACKS";
    private static final ObjectMapper JsonMapper = new ObjectMapper();

    public static String uuid() {
        return UUID.randomUUID().toString();
    }

    public static void addCompleteTracks(ChatClientRequest chatClientRequest) {
        System.out.println("addCompleteTracks ChatClientRequest");
        addCompleteTracksToContext(chatClientRequest.context(), uuid());
    }

    public static void addCompleteTracks(ChatClientResponse chatClientResponse) {
        System.out.println("addCompleteTracks ChatClientResponse");
        addCompleteTracksToContext(chatClientResponse.context(), uuid());
    }

    @SuppressWarnings("unchecked")
    private static void addCompleteTracksToContext(Map<String, Object> context, Object track) {
        try {
            System.out.println(JsonMapper.writeValueAsString(track));
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }
        Object existingCompleteTracks = context.get(COMPLETE_TRACKS);
        if (existingCompleteTracks instanceof List<?> existingList) {
            ((List<Object>) existingList).add(track);
            System.out.println("addCompleteTracksToContext existingList size: " + existingList.size());
        } else {
            List<Object> tracks = new ArrayList<>();
            tracks.add(track);
            context.put(COMPLETE_TRACKS, tracks);
            System.out.println("addCompleteTracksToContext tracks size: " + tracks.size());
        }
    }
}

Custom BaseAdvisor implementation:

import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.AdvisorChain;
import org.springframework.ai.chat.client.advisor.api.BaseAdvisor;
import org.jspecify.annotations.NonNull;

public class CompleteTrackingAdvisor implements BaseAdvisor {

    public static final int ORDER = 10000;

    @Override
    public @NonNull String getName() {
        return CompleteTrackingAdvisor.class.getSimpleName();
    }

    @Override
    public @NonNull ChatClientRequest before(@NonNull ChatClientRequest chatClientRequest, @NonNull AdvisorChain advisorChain) {
        System.out.println("CompleteTrackingAdvisor before");
        AgentAdvisorContextOperator.addCompleteTracks(chatClientRequest);
        return chatClientRequest;
    }

    @Override
    public @NonNull ChatClientResponse after(@NonNull ChatClientResponse chatClientResponse, @NonNull AdvisorChain advisorChain) {
        System.out.println("CompleteTrackingAdvisor after");
        AgentAdvisorContextOperator.addCompleteTracks(chatClientResponse);
        return chatClientResponse;
    }

    @Override
    public int getOrder() {
        return ORDER;
    }
}

ChatClient setup & trigger multi‑round tool‑call stream:

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(
                ToolCallingAdvisor.builder().build(),
                new CompleteTrackingAdvisor()
        )
        .build();

// use prompt that forces multiple rounds of tool calling
chatClient.prompt("prompt‑that‑triggers‑multi‑tool‑calls")
        .stream()
        .collectList()
        .block();

Observed result:
The COMPLETE_TRACKS list only contains items from the last tool‑loop iteration.
All track entries appended in previous iterations are lost.

Related references:

  • PR #5747: fixed advisorContext propagation for .call() path, missing streaming branch
  • Issue #6842: fixes advisor‑injected Message loss, unrelated to advisorContext Map

ToolCallingAdvisor(流式模式):递归工具循环内对 advisorContext 的修改丢失,仅保留最后一轮上下文

Bug 描述
ToolCallingAdvisor stream() 流式模式执行多轮递归工具调用时:
自定义 BaseAdvisor 在每一轮工具循环迭代内对 advisorContext 的修改,不会传递到下一轮递归工具调用。
只保留最后一轮迭代的上下文变更,前面轮次向上下文List累加的数据全部静默丢失。

非流式 .call() 模式行为正常,该能力由 PR #5747 修复。
但流式分支没有实现相同的上下文合并透传逻辑。

流式工具循环内部构建下一轮请求:

advisorContext(new HashMap<>(originalRequest.advisorContext()))

只会对整个工具循环最开始那一刻的advisorContext做快照拷贝。
每一轮循环内部产生的上下文变更、List追加操作,不会带入后续递归轮次。

注意:该Bug与 #6842 不是同一个问题。
#6842 解决Advisor注入的Message消息丢失;本问题是 advisorContext Map内部状态丢失。

运行环境

  • Spring AI 版本:2.0.1
  • Java 版本:JDK 21
  • 复现无需向量库。

复现步骤

  1. ChatClient装配 ToolCallingAdvisor 与自定义 BaseAdvisor(CompleteTrackingAdvisor)。
  2. 在Advisor的 before() / after() 钩子中,向 advisorContextCOMPLETE_TRACKS List追加跟踪记录。
  3. 让大模型返回多轮连续工具调用,触发工具递归循环执行多次。
  4. 观察advisorContext中的COMPLETE_TRACKS列表:前面轮次写入的跟踪记录消失,仅保留最后一轮追加的条目。

预期行为
流式模式下,每一轮工具循环迭代对advisorContext的修改、列表累加,应当可以跨递归工具调用透传,行为对齐 .call() 非流式模式。上一轮写入的状态,下一轮Advisor应当可以读取。

最小可复现代码

上下文工具类 AgentAdvisorContextOperator

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.UUID;

public class AgentAdvisorContextOperator {
    public static final String COMPLETE_TRACKS = "COMPLETE_TRACKS";
    private static final ObjectMapper JsonMapper = new ObjectMapper();

    public static String uuid() {
        return UUID.randomUUID().toString();
    }

    public static void addCompleteTracks(ChatClientRequest chatClientRequest) {
        System.out.println("addCompleteTracks ChatClientRequest");
        addCompleteTracksToContext(chatClientRequest.context(), uuid());
    }

    public static void addCompleteTracks(ChatClientResponse chatClientResponse) {
        System.out.println("addCompleteTracks ChatClientResponse");
        addCompleteTracksToContext(chatClientResponse.context(), uuid());
    }

    @SuppressWarnings("unchecked")
    private static void addCompleteTracksToContext(Map<String, Object> context, Object track) {
        try {
            System.out.println(JsonMapper.writeValueAsString(track));
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }
        Object existingCompleteTracks = context.get(COMPLETE_TRACKS);
        if (existingCompleteTracks instanceof List<?> existingList) {
            ((List<Object>) existingList).add(track);
            System.out.println("addCompleteTracksToContext existingList size: " + existingList.size());
        } else {
            List<Object> tracks = new ArrayList<>();
            tracks.add(track);
            context.put(COMPLETE_TRACKS, tracks);
            System.out.println("addCompleteTracksToContext tracks size: " + tracks.size());
        }
    }
}

自定义Advisor CompleteTrackingAdvisor

import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.AdvisorChain;
import org.springframework.ai.chat.client.advisor.api.BaseAdvisor;
import org.jspecify.annotations.NonNull;

public class CompleteTrackingAdvisor implements BaseAdvisor {

    public static final int ORDER = 10000;

    @Override
    public @NonNull String getName() {
        return CompleteTrackingAdvisor.class.getSimpleName();
    }

    @Override
    public @NonNull ChatClientRequest before(@NonNull ChatClientRequest chatClientRequest, @NonNull AdvisorChain advisorChain) {
        System.out.println("CompleteTrackingAdvisor before");
        AgentAdvisorContextOperator.addCompleteTracks(chatClientRequest);
        return chatClientRequest;
    }

    @Override
    public @NonNull ChatClientResponse after(@NonNull ChatClientResponse chatClientResponse, @NonNull AdvisorChain advisorChain) {
        System.out.println("CompleteTrackingAdvisor after");
        AgentAdvisorContextOperator.addCompleteTracks(chatClientResponse);
        return chatClientResponse;
    }

    @Override
    public int getOrder() {
        return ORDER;
    }
}

ChatClient 构建,触发多轮工具调用流式请求

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(
                ToolCallingAdvisor.builder().build(),
                new CompleteTrackingAdvisor()
        )
        .build();

// 使用会触发多轮工具调用的提示词
chatClient.prompt("prompt‑that‑triggers‑multi‑tool‑calls")
        .stream()
        .collectList()
        .block();

实际现象:
COMPLETE_TRACKS 列表中只有最后一轮工具循环的记录,前面迭代追加的跟踪记录全部丢失。

相关引用:

  • PR #5747:修复call模式advisorContext透传,流式分支缺失该逻辑
  • Issue #6842:修复Advisor注入Message丢失,与advisorContext Map无关

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the ToolCallingAdvisor streaming entry point and compare its recursive tool-loop request construction with the working .call() path fixed by PR #5747. Reproduce the multi-round stream using the CompleteTrackingAdvisor and COMPLETE_TRACKS context from the issue, then add regression coverage showing earlier-round entries remain visible in later rounds.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.