spring-projects / spring-projects/spring-ai

Add first-class document reranking (RerankModel + RAG DocumentPostProcessor implementation)

Open
#6,524 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Problem
The modular RAG pipeline in RetrievalAugmentationAdvisor already wires a post-retrieval reranking step that loops over documentPostProcessors:

// 5. Post-process the documents.
for (var documentPostProcessor : this.documentPostProcessors) {
documents = documentPostProcessor.process(originalQuery, documents);
}
But Spring AI ships no implementation of this hook (implements DocumentPostProcessor → 0 matches), and there is no RerankModel abstraction in spring-ai-model. So out of the box, retrieved documents can only be ordered by first-stage vector similarity. Anyone wanting reranking — the primary lever for RAG precision@k — must hand-roll the plumbing with no portable API, auto-config, or observability.

Proposed solution
Add a portable, provider-agnostic RerankModel extends Model<RerankRequest, RerankResponse> in spring-ai-model (mirroring EmbeddingModel), plus a RerankingDocumentPostProcessor in spring-ai-rag that adapts it into the existing advisor — no change to the DocumentPostProcessor interface or the advisor contract. Ship a local ONNX cross-encoder backend first (runs in CI, no API key), then hosted backends (Cohere / Bedrock / Jina) as separate opt-in modules with auto-config + Micrometer observation.

Goals
Portable RerankModel abstraction consistent with existing Model<Req,Resp> design
RerankingDocumentPostProcessor with top-N truncation and score-threshold filtering
Self-contained local backend (no credentials), hosted backends incrementally

Anti-goals
No change to DocumentPostProcessor / RetrievalAugmentationAdvisor (the hook already exists)
No new retrieval/index engine; reranking only reorders an existing candidate set
Not shipping every hosted provider at once
Backwards compatibility
Purely additive and opt-in. No existing public API changes; no behavior change unless a reranker is explicitly added.

I'm happy to implement this. I've already prototyped the RerankModel abstraction in spring-ai-model and a RerankingDocumentPostProcessor in spring-ai-rag with unit tests, and can open a single feat/document-reranking PR (abstraction + RAG adapter + a local ONNX cross-encoder backend so it's CI-testable without API keys) once the direction is confirmed. Happy to start with a design/*.adoc if the team prefers the design-doc process.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing Model abstraction in spring-ai-model and the documentPostProcessors flow in RetrievalAugmentationAdvisor and DocumentPostProcessor in spring-ai-rag. The proposed work is done when the RerankModel abstraction, RerankingDocumentPostProcessor, local ONNX backend, and unit tests are agreed and implemented; use design/*.adoc first if the team requires a design document.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai, backend-api-design, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.