spring-projects / spring-projects/spring-ai
Streaming Tool Calls problem in Observation
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Question
In flux streaming request, ChatClientObservationContext and ToolCallingObservationContext do not have a parent-child inheritance relationship, perhaps due to multithreading considerations. ChatClientObservationContext has an important indicator, spring.ai.chat.client.conversation.id, which could be passed to ToolCallingObservationContext to print when a tool call occurs. However, in the current Spring AI version 1.1.0, this doesn't seem to be possible yet. Is there any solution for this?
However, for call request, ChatClientObservationContext and ToolCallingObservationContext do have a parent-child inheritance relationship. This allows me to obtain the parent class ChatClientObservationContext, and thus the spring.ai.chat.client.conversation.id can be obtained.
public static ChatClientObservationContext findParentClientContext(Observation.ContextView ctx) {
if (ctx == null) {
return null;
}
try {
ObservationView current = ctx.getParentObservation();
while (current != null) {
Observation.ContextView parentCtx = current.getContextView();
if (parentCtx instanceof ChatClientObservationContext clientCtx) {
return clientCtx;
}
current = parentCtx.getParentObservation();
}
}catch (Throwable ignore) {
}
return null;
}
public class ToolObservationHandler implements ObservationHandler<ToolCallingObservationContext> {
@Override
public void onStop(ToolCallingObservationContext context) {
ChatClientObservationContext clientCtx = findParentClientContext(context);
// note: for flux streaming request, clientCtx will always be null
if (clientCtx != null) {
String conversationId = Optional.ofNullable(
clientCtx.getHighCardinalityKeyValue(
ChatClientObservationDocumentation.HighCardinalityKeyNames.CHAT_CLIENT_CONVERSATION_ID.asString()))
.map(KeyValue::getValue)
.orElse("");
}
}
}
// flux streaming request
public Flux<String> flux(String message, Long chatId) {
AtomicBoolean hasSentSeparator = new AtomicBoolean(false);
return chatClient
.prompt(promptService.getDefault())
.user(message)
.toolCallbacks(toolCallbackProvider.getToolCallbacks())
.tools(localToolsService)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "abc-1"))
.stream()
.content().contextWrite(ctx -> ctx.put(ChatMemory.CONVERSATION_ID, "abc-1"));
}
// `call` request
public String chat(String message, Long chatId) {
return chatClient
.prompt(promptService.getDefault())
.user(message)
.toolCallbacks(toolCallbackProvider.getToolCallbacks())
.tools(localToolsService)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "abc-1"))
.call()
.content();
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ChatClientObservationContext, ToolCallingObservationContext, and the ObservationHandler.onStop entry point, comparing the provided Flux streaming and call examples. Trace how parent observations and the conversation ID are handled in each path; done should be a defined, verified outcome for conversation-ID visibility during streaming tool calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100