[geaflow/ai-memory] Graph Memory Phase1 Roadmap
- Dominant language
- Java
- Stars
- 808
- Forks
- 188
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 2
Description
## 1. Working Position
Phase 1 should turn the current geaflow-ai prototype into a reproducible Graph Memory reference implementation. The goal is not to copy HugeGraph-AI component names, Gremlin, Faiss, or any specific Python stack. The goal is to provide a user-visible loop:
1. Import raw or pre-structured knowledge.
2. Split documents into chunks with stable provenance.
3. Extract schema-first property graph facts.
4. Validate, normalize, quarantine, and write graph facts.
5. Build chunk, entity-vector, and keyword indexes.
6. Support Basic, Vector, Graph, and Hybrid retrieval modes.
7. Safely translate natural language into constrained GQL or fall back to bounded traversal.
8. Fuse evidence and synthesize citation-aware answers.
9. Expose REST/Function Tool contracts with trace, metrics, and run state.
The current code already has valuable anchors: `TextFileReader`, `ModelUtils.splitLongText`, `GraphEntity`, `MemoryGraph`, `GraphMemoryServer`, `EmbeddingIndexStore`, `KeywordVector`, `EmbeddingVector`, `ChatService`, REST/CLI entry points, and a separate `geaflow-mcp` query tool surface. The missing work is contract, lifecycle, verification, safety, and composable retrieval behavior.
## 2. Project Architecture
```mermaid
flowchart TB
Client["CLI / HTTP Client"] --> API["Solon REST
GeaFlowMemoryServer :8080"]
API --> Cache["ServerMemoryCache
graph / server / session HashMap"]
API --> Core["GraphMemoryServer"]
subgraph Write["写入路径"]
API --> Mutable["MemoryMutableGraph"]
Mutable --> MemGraph["MemoryGraph
进程内 EntityGroup"]
Mutable --> Consolidate["ConsolidateServer"]
Consolidate --> KeywordRel["KeywordRelationFunction
全图扫描式关系发现"]
Consolidate --> EmptyEmb["EmbeddingRelationFunction
空实现"]
end
subgraph Read["查询路径"]
Core --> Session["SessionManagement
会话子图 HashMap"]
Core --> KeywordOp["SessionOperator
关键词检索 + 邻域扩展"]
Core --> EmbOp["EmbeddingOperator
全量候选线性余弦 TopN"]
KeywordOp --> Lucene["临时 Lucene GraphSearchStore
主干按查询重建"]
EmbOp --> EmbFile["EmbeddingIndexStore
JSONL 追加文件 + 内存 Map"]
Core --> Verb["SubgraphSemanticPromptFunction
子图文本化"]
end
MemGraph --> KeywordOp
MemGraph --> EmbOp
Session --> Verb
CASTS["CASTS Python Plugin
策略缓存 / Gremlin 状态机 / 仿真"] -. "主干无统一服务契约" .-> Core
GeaFlow["Apache GeaFlow 分布式动态图运行时"] -. "GraphComputeEngine 为空;当前未接入" .-> MemGraph
```
Design rule: every issue below must either freeze a contract, add a deterministic fixture, or implement a small replaceable component. Avoid PRs that mix extraction, storage, retrieval, answer generation, and API changes in one branch.
## 3. Labels and Difficulty
Suggested labels:
- `area:ai-memory`
- `type:feature`, `type:test`, `type:contract`, `type:docs`
- `priority:P0`, `priority:P1`, `priority:P2`
- `difficulty:starter`, `difficulty:intermediate`, `difficulty:advanced`
- `phase:graph-memory-p1`
- `good first issue` only when the issue has stable acceptance criteria and does not require architecture decisions.
Definition of Ready:
- The issue has one owner/sponsor and one backup reviewer.
- The issue lists affected paths and non-goals.
- The issue can be verified offline without external model credentials, or it provides a deterministic fake model provider.
- The issue does not change public contracts silently.
- The issue does not commit private text, tokens, large datasets, or unlicensed data.
## 4. Issue Body Template
```markdown
## Context
Describe the current geaflow-ai behavior and why this issue matters for Graph Memory Phase 1.
## Scope
List the files, interfaces, tests, docs, or fixtures this issue may change.
## Non-goals
State what must not be solved here.
## Constraints
- Offline CI must not require remote LLM or embedding credentials.
- Output must be deterministic or have an explicit tolerance/property oracle.
- Public contract changes require maintainer review.
## Acceptance Criteria
- [ ] Contract or behavior is documented.
- [ ] Success, empty, invalid, and boundary cases are covered.
- [ ] Tests or golden fixtures can run from repository root.
- [ ] Error messages are actionable and do not leak secrets or raw private text.
## Suggested Paths
- `geaflow-ai/...`
- `geaflow-mcp/...`
```
## 5. Atomic Issue List
### Track A. Document Ingestion and Chunk Lifecycle
#### GM-AI-P1-001: Define `DocumentSource` and `DocumentRecord` contracts
Priority: P0
Difficulty: Starter
Suggested labels: `area:ai-memory`, `type:contract`, `difficulty:starter`
Context: `TextFileReader` currently reads non-empty lines into strings. Graph Memory Phase 1 needs a stable document boundary before chunking, extraction, indexing, and replay can be deterministic.
Scope:
- Add contract classes for `DocumentSource`, `DocumentRecord`, and `SourceRef`.
- Include fields such as `source_id`, `uri`, `media_type`, `charset`, `content_hash`, `created_at`, `ingested_at`, and `metadata`.
- Add tests for empty files, duplicate source IDs, Unicode content, and stable content hash.
Constraints:
- Do not replace all existing readers in this issue.
- Do not add remote storage support.
- Do not store raw private text in logs.
Acceptance Criteria:
- A document loaded twice with the same content receives the same `content_hash`.
- Invalid or missing source metadata fails with a typed validation error.
- Existing `TextFileReader` tests remain compatible.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/graph/io`
- `geaflow-ai/src/test/java/org/apache/geaflow/ai`
#### GM-AI-P1-002: Implement deterministic `Chunker` SPI
Priority: P0
Difficulty: Intermediate
Context: `ModelUtils.splitLongText` can split long strings, but it does not create stable chunk IDs, offsets, source references, or replay semantics. Chunk identity is required for chunk vector retrieval and citation-aware answers.
Scope:
- Add `Chunker` interface and a default text chunker.
- Add `ChunkRecord` with `chunk_id`, `source_ref`, `ordinal`, `start_offset`, `end_offset`, `text_hash`, `policy_version`.
- Add golden tests for Chinese, English, mixed punctuation, CRLF, blank lines, and long paragraphs.
Constraints:
- Do not call embedding services.
- Do not decide extraction schema in this issue.
- Chunk IDs must not depend on wall-clock time.
Acceptance Criteria:
- Same document + same chunk policy produces identical chunk IDs across runs.
- Changing chunk policy changes `policy_version` and expected chunk IDs.
- Empty input produces an explicit empty result, not null.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/chunk`
- `geaflow-ai/src/test/resources/chunk`
#### GM-AI-P1-003: Add document replay and deduplication fixtures
Priority: P1
Difficulty: Starter
Context: Graph Memory needs replayable ingestion. If the same document is imported twice, downstream graph facts and indexes should not duplicate silently.
Scope:
- Add fixtures for duplicate documents, updated documents, deleted documents, and reordered files.
- Define expected replay behavior for `source_id + content_hash`.
- Add tests that only verify replay metadata, not full extraction.
Constraints:
- No graph backend implementation in this issue.
- No deletion propagation implementation yet; only define fixture and expected state.
Acceptance Criteria:
- Fixture clearly distinguishes duplicate, update, and delete events.
- Replay metadata can be used by later Event Log and projector issues.
Suggested paths:
- `geaflow-ai/src/test/resources/document-replay`
- `docs/graphmemory-design-doc.md` or a new `docs/geaflow-ai-graph-memory-contracts.md`
### Track B. Schema-First Extraction and KBQA Triple Strategy
#### GM-AI-P1-004: Define `ExtractionSchema` for property graph and KBQA triples
Priority: P0
Difficulty: Intermediate
Context: HugeGraph-AI-style import uses schema/prompt-driven extraction. geaflow-ai currently supports pre-structured graph import, but lacks a workflow that converts text paragraphs into typed graph facts or traditional KBQA triples.
Scope:
- Add `ExtractionSchema` contract for vertex types, edge types, properties, aliases, required fields, and allowed relation predicates.
- Include a KBQA triple projection: `subject`, `predicate`, `object`, `source_span`, `confidence`.
- Add JSON examples and parser tests.
Constraints:
- Do not implement LLM extraction in this issue.
- Do not hard-code a single business ontology.
- Schema parsing must fail closed for unknown required fields.
Acceptance Criteria:
- Valid schema examples round-trip through JSON.
- Invalid relation predicate, missing required property, and duplicate type definitions fail with typed errors.
- Schema examples include both property graph and triple-style facts.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/extract/schema`
- `geaflow-ai/src/test/resources/extraction/schema`
#### GM-AI-P1-005: Add `Extraction SPI` with deterministic fake extractor
Priority: P0
Difficulty: Intermediate
Context: Phase 1 needs a replaceable extraction layer. Contributors should be able to test the workflow without online LLM credentials.
Scope:
- Add `Extractor` interface.
- Add `ExtractionRequest`, `ExtractionResult`, `ExtractedVertex`, `ExtractedEdge`, `ExtractedTriple`.
- Add deterministic fake extractor driven by fixtures.
Constraints:
- Do not integrate remote LLM calls yet.
- Do not write directly to `MemoryGraph`.
- The extractor must preserve original chunk IDs and source spans.
Acceptance Criteria:
- Fake extractor converts fixture chunks into typed graph facts.
- Error, empty, and partial extraction cases are covered.
- Results include schema version, extractor version, confidence, and provenance.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/extract`
- `geaflow-ai/src/test/resources/extraction/results`
#### GM-AI-P1-006: Implement schema-first property graph extraction prompt builder
Priority: P1
Difficulty: Intermediate
Context: Later LLM extraction needs consistent prompts, but prompt construction should be isolated from model clients. This issue provides prompt generation, not model execution.
Scope:
- Add `ExtractionPromptBuilder`.
- Generate prompt sections for schema, allowed entity types, allowed relation types, examples, and output JSON format.
- Add golden prompt tests.
Constraints:
- No remote model call.
- Prompt must include strict JSON output requirement and source span requirement.
- Prompt must not include unrelated schemas.
Acceptance Criteria:
- Prompt output is deterministic for the same schema and examples.
- Golden tests cover English, Chinese, and mixed text.
- Prompt builder rejects schema with no vertex or edge definitions.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/extract/prompt`
#### GM-AI-P1-007: Preserve pre-structured graph import as a bypass path
Priority: P1
Difficulty: Starter
Context: geaflow-ai already supports CSV/JSON-like pre-structured graph loading. Phase 1 should not force every user through LLM extraction.
Scope:
- Document and test a bypass path where structured vertices/edges skip extraction but still produce provenance and idempotency metadata.
- Add a small fixture that imports pre-structured graph facts and raw document chunks in the same run.
Constraints:
- Do not rewrite existing CSV reader.
- Do not allow bypassed facts to skip schema validation.
Acceptance Criteria:
- Structured import and extracted facts share the same validation contract.
- Bypass facts include source reference and import mode.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/graph/io`
- `geaflow-ai/src/test/resources/import`
### Track C. Validation, Entity Resolution, and Quarantine
#### GM-AI-P1-008: Implement `SchemaValidator` for extracted graph facts
Priority: P0
Difficulty: Intermediate
Context: `GraphEntity` can be written today, but extracted facts need schema validation before entering graph storage or indexes.
Scope:
- Validate vertex labels, edge labels, required properties, property types, and relation endpoints.
- Return structured validation errors with fact ID and source span.
- Add tests for invalid label, missing property, wrong type, dangling edge, and unsupported predicate.
Constraints:
- Do not mutate the graph.
- Do not silently drop invalid facts.
Acceptance Criteria:
- Valid facts pass unchanged.
- Invalid facts are returned with stable error codes.
- Error messages do not include full raw document text.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/validate`
#### GM-AI-P1-009: Add `EntityResolver` for aliases and canonical IDs
Priority: P0
Difficulty: Intermediate
Context: Extracted entities can mention the same real-world object under different names. Phase 1 needs deterministic canonicalization before graph writes.
Scope:
- Add resolver interface and a rule-based resolver.
- Support exact alias, normalized text alias, and schema-scoped entity type matching.
- Produce `canonical_id`, `alias_used`, `confidence`, and `resolution_reason`.
Constraints:
- No embedding-based resolver in this issue.
- Do not merge entities across tenants or schemas.
- Low-confidence matches must stay unresolved.
Acceptance Criteria:
- Same alias maps to same canonical ID.
- Ambiguous alias returns an explicit ambiguous result.
- Cross-type alias collision is rejected or scoped correctly.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/resolve`
#### GM-AI-P1-010: Define provenance model for facts, chunks, and evidence
Priority: P0
Difficulty: Starter
Context: Citation-aware answers and deletion propagation require source lineage. Existing subgraph verbalization does not provide claim-level provenance.
Scope:
- Add `ProvenanceRef` and `SourceSpan` contracts.
- Link extracted facts to document, chunk, extractor version, and schema version.
- Add JSON examples.
Constraints:
- Do not implement answer generation.
- Do not store full raw private text inside provenance objects.
Acceptance Criteria:
- Every extracted fact fixture can point back to source document and chunk.
- Missing provenance fails validation for extracted facts.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/provenance`
#### GM-AI-P1-011: Add idempotency key generation for extracted facts
Priority: P0
Difficulty: Starter
Context: Replaying document ingestion must not duplicate vertices, edges, triples, chunks, or evidence records.
Scope:
- Define idempotency key inputs for vertex facts, edge facts, triple facts, and chunk records.
- Add deterministic tests for repeated import, reordered properties, and schema version change.
Constraints:
- Do not rely on Java object identity or insertion order.
- Do not include wall-clock time in keys.
Acceptance Criteria:
- Equivalent facts produce same key.
- Different schema version or source span changes key when expected.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/common/id`
#### GM-AI-P1-012: Add quarantine and dead-letter queue contracts
Priority: P1
Difficulty: Intermediate
Context: Invalid or low-confidence extraction should not be silently discarded or written as facts. Phase 1 needs quarantine visibility.
Scope:
- Add `QuarantineRecord` and `DeadLetterSink` interfaces.
- Add local JSONL dead-letter sink for tests.
- Add tests for invalid schema, low confidence, resolver ambiguity, and model malformed output.
Constraints:
- No production message queue integration.
- Dead-letter logs must redact raw private text unless explicitly configured for local tests.
Acceptance Criteria:
- Invalid facts are routed to quarantine with reason and provenance.
- Reprocessing a dead-letter record is possible using fixture data.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/quarantine`
### Track D. Graph Backend and Persistence
#### GM-AI-P1-013: Define `GraphBackend` SPI
Priority: P0
Difficulty: Advanced
Context: Current main path uses in-process `MemoryGraph`; `GraphComputeEngine` is an empty interface. Phase 1 needs a backend boundary so local persistence and a GeaFlow vertical slice can share contracts.
Scope:
- Define `GraphBackend` interface for upsert vertex, upsert edge, delete, scan, get schema, and transaction boundary.
- Define capability flags: local, persistent, distributed, supports checkpoint, supports tenant scope.
- Add contract tests using an in-memory fake backend.
Constraints:
- Do not replace `MemoryGraph` everywhere in this issue.
- Do not claim distributed production capability.
- Public SPI needs maintainer review.
Acceptance Criteria:
- A fake backend passes contract tests.
- Existing `GraphAccessor` integration path can be adapted later without breaking tests.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/backend`
#### GM-AI-P1-014: Implement local persistent backend prototype
Priority: P1
Difficulty: Intermediate
Context: HugeGraph Server provides persistence in the reference capability. geaflow-ai needs at least a restartable local backend for Phase 1 reference implementation.
Scope:
- Implement a simple local backend using append-only JSONL or another existing lightweight local format.
- Support restart reload, idempotent upsert, delete marker, and checksum.
- Add corruption and partial-write tests.
Constraints:
- This is not a distributed backend.
- Do not introduce large external dependencies without discussion.
- Writes must be recoverable or fail closed.
Acceptance Criteria:
- Data survives process restart in tests.
- Truncated or malformed local store fails with actionable error or quarantine.
- Idempotent replay does not duplicate graph facts.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/backend/local`
#### GM-AI-P1-015: Add minimal GeaFlow backend vertical slice design doc
Priority: P1
Difficulty: Starter
Context: GeaFlow is a streaming graph engine, but geaflow-ai currently cannot treat the main engine as an implemented Graph Memory backend. A vertical slice must be scoped before coding.
Scope:
- Write a design document for a minimal `GeaFlowBackend` slice.
- Define supported operations, non-goals, checkpoint behavior, and test strategy.
- Map to existing GeaFlow DSL/runtime components.
Constraints:
- No production HA claim.
- No implementation in this issue.
- Must identify exact missing APIs or blockers.
Acceptance Criteria:
- Maintainers can decide whether the first slice is read-only, write-through, or projector-based.
- Design includes rollback plan and compatibility constraints.
Suggested paths:
- `docs/geaflow-ai-graph-memory-geaflow-backend-slice.md`
#### GM-AI-P1-016: Add graph backend conformance tests
Priority: P0
Difficulty: Intermediate
Context: Multiple backends must behave consistently for Graph Memory retrieval, deletion, and replay.
Scope:
- Add backend contract test suite.
- Cover duplicate vertex, parallel edge, self-loop, dangling edge rejection, delete, restart, scan order normalization.
Constraints:
- Tests must run against fake/local backend without external services.
- Do not encode implementation-specific ordering unless explicitly sorted.
Acceptance Criteria:
- Fake backend and local backend can both run the same test suite.
- Failures identify contract name and backend name.
Suggested paths:
- `geaflow-ai/src/test/java/org/apache/geaflow/ai/backend`
### Track E. Vector, Entity, and Keyword Indexing
#### GM-AI-P1-017: Define `VectorStore` SPI with model metadata
Priority: P0
Difficulty: Intermediate
Context: `EmbeddingIndexStore` stores embeddings for `GraphEntity` in JSONL and memory map. Phase 1 needs a generic vector store contract for both chunks and graph entities.
Scope:
- Add `VectorStore` interface.
- Define metadata fields: `model_name`, `dimension`, `distance`, `index_version`, `created_at`, `format_version`.
- Add tests for dimension mismatch, missing metadata, and model mismatch.
Constraints:
- Do not implement ANN in this issue.
- Do not break existing `EmbeddingIndexStore` behavior.
Acceptance Criteria:
- Loading vectors with wrong dimension fails closed.
- Metadata is persisted in local fixture format.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/index/vectorstore`
#### GM-AI-P1-018: Implement `ChunkVectorIndex`
Priority: P0
Difficulty: Intermediate
Context: HugeGraph-AI-style RAG retrieves document chunks. geaflow-ai currently indexes `GraphEntity`, not document chunks.
Scope:
- Add chunk vector index built on `VectorStore`.
- Store `chunk_id`, source span, embedding, model metadata, and text hash.
- Add tests for insert, query, delete marker, restart, and source span return.
Constraints:
- Do not call remote embedding services in tests.
- Use deterministic fake embeddings.
- Do not return raw full document when only source span is needed.
Acceptance Criteria:
- Query vector returns chunk hits with score and `source_ref`.
- Deleted chunks do not appear after searchable watermark.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/index/chunk`
#### GM-AI-P1-019: Add `EntityVectorIndex` wrapper around graph entity embeddings
Priority: P0
Difficulty: Intermediate
Context: Graph entity embedding exists, but Phase 1 needs explicit entity-vector semantics for fuzzy entity anchoring.
Scope:
- Wrap current `EmbeddingIndexStore` behavior behind `EntityVectorIndex`.
- Add entity ID, entity type, model metadata, and vector version.
- Add compatibility tests with existing `EmbeddingOperator`.
Constraints:
- Do not change retrieval ranking yet.
- Link existing embedding correctness fix issue/PR where applicable.
Acceptance Criteria:
- Entity vector lookup can distinguish vertex and edge entities.
- Dimension/model mismatch is detected before search.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/index/entity`
#### GM-AI-P1-020: Add minimal ANN adapter interface
Priority: P1
Difficulty: Intermediate
Context: Current vector search is linear TopN. Phase 1 can ship with linear search, but the contract should allow an ANN implementation later.
Scope:
- Add `NearestNeighborIndex` interface.
- Provide linear adapter as default implementation.
- Add tests for deterministic scoring and tie-break order.
Constraints:
- Do not add heavy ANN dependency in this issue.
- Do not claim performance improvement.
Acceptance Criteria:
- Linear adapter passes the same contract that an ANN adapter will use.
- Tie-break behavior is documented.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/index/ann`
#### GM-AI-P1-021: Harden keyword index serialization golden tests
Priority: P0
Difficulty: Starter
Context: Keyword retrieval exists through Lucene/keyword vectors, but the main branch behavior has been described as rebuilding during query and vulnerable to noisy common tokens.
Scope:
- Add golden tests for keyword tokenization, serialization, reload, Unicode, stopwords, and common-token pollution.
- Ensure incremental add/delete expected behavior is documented.
Constraints:
- Do not implement a new keyword engine.
- Coordinate with any existing resident keyword index PR.
Acceptance Criteria:
- Golden tests fail if tokenization or serialization changes unexpectedly.
- Common tokens do not dominate exact graph anchoring fixtures.
Suggested paths:
- `geaflow-ai/src/test/java/org/apache/geaflow/ai/index`
- `geaflow-ai/src/test/resources/index/keyword`
#### GM-AI-P1-022: Add searchable watermark for graph, keyword, and vector indexes
Priority: P1
Difficulty: Intermediate
Context: Graph facts and indexes can diverge. Retrieval must know whether graph, keyword, chunk vector, and entity vector projections are searchable for a given import run.
Scope:
- Define `SearchableWatermark`.
- Add fields for accepted, validated, graphed, keyword_indexed, chunk_vector_indexed, entity_vector_indexed.
- Add tests for partial projection and lag reporting.
Constraints:
- Do not implement distributed consistency.
- Do not hide partial state; expose it in trace.
Acceptance Criteria:
- Retrieval trace can show which projections were available.
- Partial index state is visible and testable.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/workflow`
### Track F. Retrieval Modes, Router, Analyzer, and Anchoring
#### GM-AI-P1-023: Define `RetrievalMode` contract
Priority: P0
Difficulty: Starter
Context: `GraphMemoryServer` currently iterates over configured index stores. There is no explicit Basic, Vector, Graph, or Hybrid mode contract.
Scope:
- Add enum or contract for `BASIC`, `VECTOR`, `GRAPH`, `HYBRID`.
- Define required inputs, allowed indexes, and expected outputs for each mode.
- Add tests that each mode can be selected explicitly.
Constraints:
- Do not implement router ranking in this issue.
- Do not change answer synthesis.
Acceptance Criteria:
- Calling unsupported mode fails with typed error.
- Mode selection is visible in retrieval trace.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval`
#### GM-AI-P1-024: Implement deterministic `RuleRouter`
Priority: P0
Difficulty: Intermediate
Context: Phase 1 needs reproducible routing before any learned router. The router should select retrieval mode based on request flags, query shape, and available indexes.
Scope:
- Add `RuleRouter`.
- Preserve user-forced mode.
- Add fallback rules for missing chunk index, missing entity index, or graph backend unavailable.
Constraints:
- No LLM router.
- No learned ranking.
- Original query must be preserved in trace.
Acceptance Criteria:
- Router decisions are deterministic.
- Trace includes reason, selected mode, skipped modes, and fallback reason.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/router`
#### GM-AI-P1-025: Define `QueryAnalyzer` SPI
Priority: P0
Difficulty: Intermediate
Context: geaflow-ai can accept `KeywordVector` and `EmbeddingVector`, but it lacks a task-specific rewrite/NER pipeline.
Scope:
- Add `QueryAnalyzer` interface.
- Define `AnalyzedQuery` with original text, normalized text, keywords, candidate entities, language, and analyzer version.
- Add deterministic rule analyzer.
Constraints:
- Do not overwrite original query.
- No remote model call in default tests.
- Analyzer output must be optional; retrieval can still run with original query.
Acceptance Criteria:
- Rule analyzer handles blank, Chinese, English, and mixed queries.
- Trace includes analyzer version and output.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/query`
#### GM-AI-P1-026: Add optional LLM analyzer adapter contract
Priority: P2
Difficulty: Intermediate
Context: Later Graph Memory may use LLM NER/rewrite, but Phase 1 should isolate this behind an adapter and keep deterministic CI.
Scope:
- Add adapter contract for LLM analyzer.
- Add fake model-backed analyzer for tests.
- Define timeout, malformed response, and fallback behavior.
Constraints:
- Do not require online model credentials.
- Do not make LLM analyzer default.
Acceptance Criteria:
- Fake adapter can return NER/rewrite fixtures.
- Timeout and malformed output fall back to rule analyzer with trace.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/query/model`
#### GM-AI-P1-027: Implement `EntityAnchorService` exact-to-fuzzy state machine
Priority: P0
Difficulty: Intermediate
Context: Keyword hits and embedding candidates exist separately. Phase 1 needs an explicit exact match to fuzzy match state machine for graph anchoring.
Scope:
- Add `EntityAnchorService`.
- Try exact keyword/entity alias match first.
- Fall back to entity vector fuzzy search.
- Return source, score, model version, index version, and failure reason.
Constraints:
- Do not execute graph traversal in this issue.
- Do not merge entities here; use resolver output.
Acceptance Criteria:
- Exact hit, fuzzy hit, no anchor, ambiguous anchor, stale index cases are covered.
- Anchor trace is stable and serializable.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/anchor`
#### GM-AI-P1-028: Add anchor negative corpus
Priority: P1
Difficulty: Starter
Context: Fuzzy anchoring can produce confident wrong graph queries. New contributors can help by building a negative test corpus.
Scope:
- Add fixtures for same name across types, same name across tenants, common keyword-only queries, typo, and no entity.
- Define expected anchor failure or ambiguity.
Constraints:
- No code implementation required unless tests need small harness glue.
- Do not include private or copyrighted text.
Acceptance Criteria:
- Corpus contains at least 30 negative cases.
- Each case has expected reason code.
Suggested paths:
- `geaflow-ai/src/test/resources/retrieval/anchor-negative`
### Track G. Text2GQL, Query Safety, and Fallback
#### GM-AI-P1-029: Define `Text2GQL` SPI and Query IR
Priority: P0
Difficulty: Advanced
Context: `geaflow-mcp` can execute caller-provided GQL, but Graph Memory needs natural language to constrained GQL. GeaFlow should use GQL/Query IR, not copy a Gremlin-specific design.
Scope:
- Add `Text2GqlService` interface.
- Define `GraphQueryIR` for read-only graph query intents.
- Add example-pair index contract for schema-aware generation.
Constraints:
- Do not execute generated GQL.
- Do not permit write operations in IR.
- Public IR requires maintainer review.
Acceptance Criteria:
- Fake Text2GQL returns deterministic IR/GQL for fixtures.
- Invalid or unsafe generation returns typed error.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/text2gql`
#### GM-AI-P1-030: Add schema and example-pair index for Text2GQL
Priority: P1
Difficulty: Intermediate
Context: Text2GQL needs schema and examples. This issue builds retrieval of examples, not generation.
Scope:
- Store schema examples and query examples with version and domain tags.
- Add simple keyword/vector-free selection first.
- Add golden tests for example selection.
Constraints:
- No remote model call.
- Do not include production user queries.
Acceptance Criteria:
- Given a schema and query category, relevant examples are selected deterministically.
- Missing examples are visible in trace.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/text2gql/examples`
#### GM-AI-P1-031: Implement read-only AST/IR guard
Priority: P0
Difficulty: Advanced
Context: `GeaFlowMcpServerTools.executeQuery` accepts arbitrary query text. Graph Memory generated queries must be guarded before execution.
Scope:
- Add read-only guard for `GraphQueryIR` and/or parsed GQL.
- Block DDL, DML, mutation, unbounded traversal, unsupported functions, and missing tenant scope.
- Add negative tests.
Constraints:
- Fail closed on unknown statement type.
- Do not use regex-only validation for complex query safety.
- Generated query execution must call guard first.
Acceptance Criteria:
- At least 50 negative cases are rejected with reason codes.
- Safe read queries pass with budget metadata.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/security`
- `geaflow-mcp/src/main/java/org/apache/geaflow/mcp/server/util`
#### GM-AI-P1-032: Add traversal budget contract
Priority: P0
Difficulty: Intermediate
Context: Text2GQL and fallback traversal can explode on high-degree graphs. Phase 1 needs hop, result, time, and memory budgets.
Scope:
- Define `TraversalBudget`.
- Include max hops, max visited vertices, max visited edges, max result rows, timeout, and cancellation flag.
- Add tests for budget exhaustion.
Constraints:
- Do not implement full BFS fallback here.
- Budget exhaustion must return partial/empty result with explicit status, not silent truncation.
Acceptance Criteria:
- Budget object serializes into retrieval trace.
- Exhausted budget has stable reason code.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/traversal`
#### GM-AI-P1-033: Implement bounded traversal fallback
Priority: P0
Difficulty: Intermediate
Context: Current Graph Memory can expand a one-hop subgraph, but it lacks failure-triggered BFS fallback with explicit reasons.
Scope:
- Add `BoundedTraversalFallback`.
- Trigger on no-anchor, invalid generated query, timeout, or empty result.
- Use `TraversalBudget`.
Constraints:
- Do not bypass tenant or schema guards.
- Do not return results without marking fallback in trace.
Acceptance Criteria:
- no-anchor, invalid, timeout, and empty-result fixtures trigger distinct fallback reasons.
- Budget exhaustion is reported.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/traversal`
### Track H. Chunk Retrieval, Fusion, and Answer Synthesis
#### GM-AI-P1-034: Implement `ChunkRetriever`
Priority: P0
Difficulty: Intermediate
Context: Pure vector RAG requires query vector to chunk vector retrieval. geaflow-ai currently has entity vector search, not chunk retrieval.
Scope:
- Add `ChunkRetriever`.
- Accept analyzed query and query embedding.
- Return chunk hits with score, source span, model version, and index version.
Constraints:
- Tests must use fake embeddings.
- Do not synthesize answers.
Acceptance Criteria:
- Query returns deterministic top-k chunk hits.
- Empty index and dimension mismatch are covered.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/retrieval/chunk`
#### GM-AI-P1-035: Define normalized `Evidence` contract
Priority: P0
Difficulty: Starter
Context: Graph/vector/operator results currently update session subgraphs, but there is no unified evidence object for fusion, rerank, answer citation, or abstention.
Scope:
- Add `Evidence` contract.
- Fields: `evidence_id`, `kind`, `source_ref`, `source_span`, `raw_score`, `normalized_score`, `retrieval_path`, `lineage`, `tenant_id`.
- Add JSON examples.
Constraints:
- Evidence must be immutable after creation.
- Evidence must not store secrets.
Acceptance Criteria:
- Chunk, entity, path, and graph fact evidence examples round-trip.
- Missing source ref fails validation for answerable evidence.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/evidence`
#### GM-AI-P1-036: Implement RRF and weighted fusion
Priority: P0
Difficulty: Intermediate
Context: Hybrid retrieval needs graph/vector merge with score normalization and dedupe. Current operators mutate session state sequentially without a unified fusion contract.
Scope:
- Implement Reciprocal Rank Fusion and weighted fusion.
- Deduplicate by evidence ID, canonical entity ID, and source span.
- Add tests for score ties, duplicate evidence, and missing modality.
Constraints:
- Do not add learned reranker.
- Fusion must be deterministic.
Acceptance Criteria:
- Same inputs always produce same order.
- Missing vector or graph modality still returns valid fused evidence with trace.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/evidence/fusion`
#### GM-AI-P1-037: Add optional reranker SPI
Priority: P1
Difficulty: Intermediate
Context: HugeGraph-AI-style architecture may include rerankers, but Phase 1 should not depend on a remote reranker.
Scope:
- Define `Reranker` interface.
- Add no-op and fixture reranker.
- Add timeout/fallback behavior.
Constraints:
- No online model requirement.
- Reranker cannot remove provenance.
Acceptance Criteria:
- Reranker timeout falls back to fused order and records trace.
- Fixture reranker can reorder evidence deterministically.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/evidence/rerank`
#### GM-AI-P1-038: Implement `AnswerSynthesizer` contract
Priority: P0
Difficulty: Intermediate
Context: geaflow-ai can verbalize a subgraph as context and has `ChatService`, but it lacks citation-aware answer workflow.
Scope:
- Add `AnswerSynthesizer`.
- Input: original query, evidence list, answer policy, model provider.
- Output: answer text, cited evidence IDs, abstention reason, model trace.
Constraints:
- Tests use fake chat provider.
- No evidence means abstain, not hallucinate.
- Claims must reference evidence IDs.
Acceptance Criteria:
- Answer with evidence includes citations.
- Empty or low-confidence evidence returns abstention.
- Prompt does not include unrelated tenant data.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/answer`
#### GM-AI-P1-039: Add claim-to-evidence citation verifier
Priority: P1
Difficulty: Starter
Context: Citation-aware answers are only useful if every claim can be traced to evidence.
Scope:
- Add verifier that checks cited evidence IDs exist and are allowed.
- Add tests for missing citation, stale evidence, and cross-tenant evidence.
Constraints:
- Do not solve natural language entailment.
- This is structural citation verification.
Acceptance Criteria:
- Answer with unknown evidence ID fails verification.
- Cross-tenant evidence citation is rejected.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/answer`
### Track I. API, Function Tool, MCP Adapter, and Model Providers
#### GM-AI-P1-040: Define REST v1 Graph Memory contracts
Priority: P0
Difficulty: Intermediate
Context: geaflow-ai has REST/CLI surfaces, but Phase 1 needs stable contracts shared by import, retrieval, answer, and trace.
Scope:
- Define REST v1 DTOs for import, query, answer, run status, and trace.
- Add OpenAPI-like documentation or JSON examples.
- Add backward compatibility notes for existing CLI.
Constraints:
- Do not implement all endpoints in this issue.
- Public DTOs require maintainer review.
Acceptance Criteria:
- DTO examples cover success, validation error, no evidence, fallback, and partial index state.
- Contract docs are understandable by a new contributor.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/service`
- `docs/geaflow-ai-graph-memory-rest-v1.md`
#### GM-AI-P1-041: Add Function Tool facade contract
Priority: P1
Difficulty: Starter
Context: Graph Memory should expose a function-tool-like facade for RAG/agent callers without coupling those callers to internal classes.
Scope:
- Define minimal function tool operations: import document, query memory, get trace, list sessions.
- Add JSON examples.
Constraints:
- No MCP dependency in this issue.
- Do not expose unsafe generated query execution directly.
Acceptance Criteria:
- Function tool facade maps to REST v1 DTOs.
- Unsafe operations are absent or explicitly marked unsupported.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/tool`
#### GM-AI-P1-042: Document MCP thin adapter boundary
Priority: P1
Difficulty: Starter
Context: `geaflow-mcp` is currently a separate graph query tool surface. It can execute caller-provided GQL, but should not become the core Graph Memory workflow.
Scope:
- Write adapter boundary doc.
- Explain how MCP should call Graph Memory REST/Function Tool after core contracts stabilize.
- List guardrails for `executeQuery`.
Constraints:
- No MCP implementation change.
- Do not encourage arbitrary generated query execution.
Acceptance Criteria:
- Document clearly states MCP is a thin adapter, not the source of Graph Memory truth.
- Lists future adapter endpoints and safety requirements.
Suggested paths:
- `docs/geaflow-ai-graph-memory-mcp-adapter.md`
#### GM-AI-P1-043: Implement `ModelProviderRegistry`
Priority: P0
Difficulty: Intermediate
Context: `ChatService` and `EmbeddingService` exist, but model usage is not separated by task such as chat, extract, text2gql, embedding, and rerank.
Scope:
- Add registry for task-scoped providers.
- Support capability flags: chat, embedding, extraction, text2gql, rerank.
- Add fake providers for CI.
Constraints:
- Do not remove existing services.
- Do not log secrets.
- Missing provider must produce typed error with task name.
Acceptance Criteria:
- Different tasks can resolve different providers.
- Fake providers cover offline tests.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/common/model`
#### GM-AI-P1-044: Add model timeout, circuit breaker, and cost metadata contract
Priority: P1
Difficulty: Intermediate
Context: Remote model calls affect cost, latency, and reliability. Phase 1 needs observability before production use.
Scope:
- Add task timeout metadata, retryability, circuit breaker status, and optional cost counters.
- Add tests for timeout, retryable error, permanent error, and circuit open.
Constraints:
- No vendor-specific billing integration.
- No online model call in CI.
Acceptance Criteria:
- Model failures appear in run trace with task and retryability.
- Secrets and prompts are redacted according to policy.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/common/model`
### Track J. Workflow, Trace, Metrics, and Exclusions
#### GM-AI-P1-045: Define Workflow SPI and `RunContext`
Priority: P0
Difficulty: Advanced
Context: `GraphMemoryServer` directly loops over operators. Phase 1 needs run state, node trace, retry, cancellation, and metrics boundaries.
Scope:
- Add `Workflow`, `WorkflowNode`, `RunContext`, `NodeResult`.
- Support node status: pending, running, succeeded, failed, skipped, cancelled.
- Add in-memory run store for tests.
Constraints:
- Do not implement distributed scheduler.
- Do not rewrite all operators in this issue.
Acceptance Criteria:
- A simple import-retrieve-answer workflow can be represented as nodes.
- Node failures include error code, retryability, and trace ID.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/workflow`
#### GM-AI-P1-046: Add retrieval trace contract
Priority: P0
Difficulty: Starter
Context: Router, analyzer, anchoring, Text2GQL, fallback, fusion, rerank, and answer synthesis must be inspectable.
Scope:
- Define `RetrievalTrace`.
- Include original query, analyzer output, router decision, anchors, budgets, generated query status, fallback reason, evidence summary.
- Add JSON examples.
Constraints:
- Redact sensitive raw text fields where needed.
- Trace must be stable enough for golden tests.
Acceptance Criteria:
- Trace examples cover Basic, Vector, Graph, and Hybrid modes.
- Failure modes include no-anchor, invalid query, timeout, empty evidence.
Suggested paths:
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/trace`
#### GM-AI-P1-047: Add Graph Memory metric names and low-cardinality labels
Priority: P1
Difficulty: Starter
Context: Phase 1 needs metrics for ingestion, projection, retrieval, fallback, model calls, and answer behavior.
Scope:
- Define metric names and label rules.
- Include counters, timers, gauges, and watermark lag.
- Add documentation and unit tests for name formatting if needed.
Constraints:
- No tenant ID, entity ID, raw query, or source text as metric labels.
- Keep labels low-cardinality.
Acceptance Criteria:
- Metrics cover import accepted/validated/quarantined, index lag, retrieval mode count, fallback count, model latency/error, abstention count.
Suggested paths:
- `docs/geaflow-ai-graph-memory-metrics.md`
- `geaflow-ai/src/main/java/org/apache/geaflow/ai/metrics`
#### GM-AI-P1-048: Add Phase 1 exclusion guardrail tests
Priority: P0
Difficulty: Starter
Context: The capability matrix includes training and graph ML, but Phase 1 explicitly excludes pre-training, SFT, PPO, DPO, and large GNN stacks.
Scope:
- Add documentation and tests/config validation that reject training-mode flags in Phase 1 profiles.
- Add FAQ explaining why CASTS simulation is not a training stack.
Constraints:
- Do not remove CASTS.
- Do not block future Phase 2+ design discussions.
Acceptance Criteria:
- Phase 1 config rejects unsupported training modes with clear messages.
- Docs list excluded capabilities and revisit conditions.
Suggested paths:
- `docs/geaflow-ai-graph-memory-phase1-scope.md`
- `geaflow-ai/src/test/java/org/apache/geaflow/ai`
#### GM-AI-P1-049: Add contributor guide for Graph Memory atomic issues
Priority: P1
Difficulty: Starter
Context: These issues are intended for new contributors. They need local commands, fixture guidance, and boundaries.
Scope:
- Write contributor guide for Graph Memory Phase 1 issues.
- Include module map, test commands, fake model provider usage, and PR checklist.
Constraints:
- Do not require external model credentials.
- Do not ask newcomers to run the full repository if a module-level command exists.
Acceptance Criteria:
- A new contributor can pick a starter issue and run the relevant tests locally.
- Guide explains how to avoid secret leakage and over-sized PRs.
Suggested paths:
- `docs/geaflow-ai-graph-memory-contributor-guide.md`
Contributor guide
Research direction
Treat this as a roadmap and select one atomic GM-AI-P1 issue before starting. Read its listed paths under geaflow-ai, geaflow-mcp, or docs, then run the named or newly added deterministic tests and fixtures from the repository root. Done means that issue's stated contract, acceptance criteria, boundary cases, and offline verification are complete without combining tracks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ai, backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100