[enhancement] Implement multi-channel retrieval, graph expansion, and fusion execution
- Dominant language
- Java
- Stars
- 808
- Forks
- 188
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 2
Description
**Phase 2 of Parent epic:** [#863](https://github.com/apache/geaflow/issues/863)
## Summary
Build the retrieval core over the versioned graph and indexes delivered by Phase 1. The core must expose BM25, vector, and entity-anchor channels through one Java API, expand selected anchors by at most two hops under explicit budgets, and return candidates through a common evidence protocol.
This phase also adds deterministic deduplication, fusion, truncation, and sequential/parallel/ cascaded execution. Adaptive policy selection is added in Phase 3, but this phase must provide the executor and plan interfaces that the policy can call later.
## Motivation
The existing memory search path is session-oriented and loses channel scores during execution. A shared retrieval core is needed before different modes can be compared fairly or called by a production service.
## Scope
- Implement `TextRetriever`/Lucene BM25 retrieval with real scores, stable ranks, field mappings, `topK`, and candidate limits.
- Implement a replaceable `VectorRetriever` that accepts precomputed vectors and does not depend on a particular embedding service.
- Implement `EntityAnchorResolver` using normalized names, aliases, and entities attached to highly ranked chunks. Graph-only mode must return a structured empty result when no reliable anchor exists.
- Implement `GraphRetriever` with one- and two-hop expansion, lazy edge scanning, edge-scan caps, neighbor sampling, relation-type priority, deterministic sampling, and diffusion stopping rules.
- Convert every channel result to the unified `Evidence` model while preserving raw channel scores, ranks, source/chunk identity, and graph paths.
- Deduplicate first by stable evidence ID and then by normalized source/chunk identity.
- Implement weighted reciprocal-rank fusion by default, an optional reranker interface, and final `topK`/token-budget truncation.
- Implement sequential, parallel, and cascaded executors with per-stage budget accounting.
- Support BM25-only, Vector-only, Graph-only, and Fixed Hybrid modes. Expose a validated plan interface for Adaptive Hybrid without embedding routing rules in channel implementations.
## Suggested implementation locations
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/channel/` - BM25, vector, and anchor retrievers.
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/graph/` - neighbor provider, frontier, path, sampling, and stopping logic.
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/fusion/` - evidence merger, RRF, reranker SPI, and token truncation.
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/execution/` - stage executor and execution modes.
- `geaflow-ai/src/test/java/org/apache/geaflow/ai/retrieval/` - channel, graph, fusion, and budget tests using the fixed fixture from Phase 1.
## Testing requirements
- Unit tests for Lucene score preservation, topK, candidate caps, vector ordering, anchor confidence, path scoring, deterministic sampling, and diffusion stopping.
- Fusion tests for duplicate removal, score/rank preservation, RRF ordering, and token truncation.
- Concurrency tests for parallel execution and isolation of per-request budget/trace state.
- End-to-end tests against the Phase 1 in-memory fixture for all four fixed modes and all execution modes.
## Dependencies and follow-up
- Keep channel and executor interfaces independent of Solon so they can also be used by offline benchmark tooling.
Contributor guide
Research direction
Start by reading the Phase 1 fixture and the suggested retrieval packages: channel, graph, fusion, and execution. Run the retrieval tests under geaflow-ai/src/test/java/org/apache/geaflow/ai/retrieval/; done means the four fixed modes and execution modes satisfy the listed scoring, fusion, budget, concurrency, and end-to-end tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100