Introduce events in ChatClient streaming API
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Domain
- api, backend-api-design
Research direction
Start by reading the existing StreamAdvisor and ChatClientResponse APIs and tracing the streaming call chain. Compare the proposed ChatClientEvent flow with current response handling; done means advisors can signal events through both streaming and blocking ChatClient APIs without using response context as a workaround.
Written by the indexing model from the issue text.
Description
The advisor chain is a very powerful tool. However, I feel like there is a piece missing to the puzzle.
Whilst you can alter a ChatClientRequest or a ChatClientResponse in any way you'd like through an advisor, there is no clean way of signaling that something happened. The workaround I've found here is to inject data into the response's context and then looking for it after calling the ChatClient API, but to me that's a very hacky bandaid fix.
A practical example of this limitation is #5792, where the current API does not offer a clean way of propagating the tool responses to the user calling the ChatClient API.
To solve this, I'd like to propose introducing what I call a ChatClientEvent. This would be an interface used to signal anything that happens throughout the entire API call. Here's what it could look like:
interface ChatClientEvent {
data class ModelGeneration(val generation: Generation) : ChatClientEvent
data class ToolCalled(val id: String, val name: String, val responseData: String) : ChatClientEvent
// any other events: InputValidationFailed, etc.
}
This interface could also be extended by users to define their own custom events.
The StreamAdvisor interface would be modified to return a Flux<ChatClientEvent>, like so:
public interface StreamAdvisor extends Advisor {
Flux<ChatClientEvent> adviseStream(ChatClientRequest chatClientRequest, StreamAdvisorChain streamAdvisorChain);
}
Then, users could act on events just like they do with #chatClientResponse(), #chatResponse() or #content():
chatClient.prompt("Hello World!")
.stream()
.events()
.doOnNext { event ->
when(event) {
is ModelGeneration -> logger.info("AI response: ${event.generation.output.text}, tool calls: ${event.generation.output.toolCalls}")
is ToolCalled -> logger.info("Tool execution completed: id=${event.id}, tool=${event.name}, responseData=${event.responseData}")
}
}
The blocking API could also be updated to keep track of all events, adding an events field to the ChatClientResponse:
public record ChatClientResponse(@Nullable ChatResponse chatResponse, Map<String, @Nullable Object> context, List<ChatClientEvent> events)
I recognize that by adding the events field to ChatClientResponse, you could technically avoid introducing the #events() method to the streaming API and having to modify StreamAdvisor entirely. However - and this is more of a personal design choice, - I find it odd that the streaming API would return multiple ChatClientResponse objects, when conceptually it's all just one response that's being delivered in smaller chunks.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from spring-projects/spring-ai
-
status: waiting-for-triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
spring-projects/spring-ai#7000 ·
-
status: waiting-for-triage
Difficulty 1/5 Under an hour Newbie friendliness 90/100
spring-projects/spring-ai#6998 ·
-
[Bug - MCP server] @McpTool error messages are emitted twice when the thrown exception has no cause Openstatus: waiting-for-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
spring-projects/spring-ai#6948 · 1 comment · 1 reaction ·
-
status: waiting-for-triage
Difficulty 1/5 Under an hour Newbie friendliness 90/100
spring-projects/spring-ai#6940 · 1 comment ·
-
status: waiting-for-triage
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
spring-projects/spring-ai#6928 · 1 comment ·
All issues in spring-projects/spring-ai
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
objectionary/hone-maven-plugin#1061 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
spring-projects/spring-modulith#1895 ·