spring-projects / spring-projects/spring-ai

I hope the Advisor of ChatClient can support graceful interruption.

Open
#5,148 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Please do a quick search on GitHub issues first, the feature you are about to request might have already been requested.

Expected Behavior

The before method should allow gracefully interrupting the request to the large language model. For example:

    public @NonNull ChatClientRequest before(@NonNull ChatClientRequest chatClientRequest, @NonNull AdvisorChain advisorChain) {
        if (denySearch) {
            throw new DatasetQueryPermissionException("用户没有查询权限,不能进行查询操作");
        }
        if (CollectionUtils.isEmpty(datasetIds)) {
            throw new DatasetNotFondException("没有任何数据集,不能进行查询操作");
        }
}

Current Behavior

Currently, we can terminate the operation by throwing an exception, but this generates exception messages in the logs, which may mislead monitoring systems into thinking the application has encountered an error (though this could be acceptable if log output could be disabled).

Image

Context

When performing knowledge base retrieval, I need to avoid sending a request to the large language model altogether if no relevant content is found, thereby saving computational resources.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The entry point described is ChatClient Advisor’s before(...) method, using ChatClientRequest and AdvisorChain; begin by tracing how an exception currently terminates the request and is logged. Done means an advisor can stop the LLM request without producing misleading error logs, including for the permission and empty-dataset cases described.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.