Advisor context

Description

In the latest updates of SpringAI the tools approach has been changed and now you can define context dependent tools (tools with context available only on invocation) as a default tools during chat client creation. To provide context on a call you can use toolContext method:

ChatResponse response = chatClient.prompt(new Prompt(message))
    .toolContext(Map.of("foo", foo))
    .call().chatResponse();

But we doesn't have the same for the advisors. In case the advisor is context dependent we have no choice but to create it on a per call basis passing the context via the constructor:

ChatResponse response = chatClient.prompt(new Prompt(message))
    .advisors(new FooAdvisor(foo))
    .call().chatResponse();

That would be nice to provide a method advisorContext similar to toolContext to be able to pass some context related data to the advisors as well. This method can be used to prepopulate context of the ChatClientRequest to make this data available in the advisor, or alternatively can be passed as an additional external context parameter to the advisors methods.

Workaround

You can create a dedicated advisor that will populate internal advisor context with an external advisor context:

public class ExternalContextAdvisor implements BaseAdvisor {

    private final Map<String, Object> externalContext;

    public ExternalContextChatAdvisor(Map<String, Object> externalContext) {
        this.externalContext = externalContext;
    }

    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE;
    }

    @Override
    public ChatClientRequest before(ChatClientRequest request, AdvisorChain chain) {
        request.context().put("external-context", externalContext);
        return request;
    }

    @Override
    public ChatClientResponse after(ChatClientResponse response, AdvisorChain chain) {
        return response;
    }
}

And use it to pass an external context to advisors during the client call:

Map<String, Object> externalContext = new HashMap<>();

externalContext.push("foo", foo);
externalContext.push("bar", bar);

ChatResponse response = chatClient.prompt(new Prompt(message))
        .advisors(new ExternalContextAdvisor(externalContext)
        .call().chatResponse();

Using this external context advisor you can now register all other context dependent advisors as a default advisors during chat client creation:

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(fooAdvisor, barAdvisor)
        .build();

Inside any of the context dependent advisors you can now extract external context to get necessary parameters:

public class FooAdvisor implements BaseAdvisor {

	@Override
	public int getOrder() {
		return 0;
	}

	@Override
	public ChatClientRequest before(ChatClientRequest request, AdvisorChain chain) {

		Map<String, Object> externalContext = (Map<String, Object>) request.context()
				.get("external-context");
				
		Foo foo = (Foo) externalContext.get("foo");

		// Do something with foo

		return request;
	}

	@Override
	public ChatClientResponse after(ChatClientResponse response, AdvisorChain chain) {
		return response;
	}
}

With the above approach you can also place some results to the external context that will be available from outside

Start at the ChatClient.prompt(...).toolContext entry point and trace how request context reaches ChatClientRequest and the BaseAdvisor before/after methods. Compare that flow with default advisors and determine how a per-call advisorContext would be exposed, including how completion would be verified for context-dependent advisors.