apache / apache/geaflow

[geaflow/ai-memory] Implement `ChunkVectorIndex`

Open
#852 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
808
Forks
188
Avg merge
3d 22h
Merged PRs (30d)
2

Description

Priority: P0
Difficulty: Intermediate

Context: HugeGraph-AI-style RAG retrieves document chunks. geaflow-ai currently indexes `GraphEntity`, not document chunks.

Scope:

- Add chunk vector index built on `VectorStore`.
- Store `chunk_id`, source span, embedding, model metadata, and text hash.
- Add tests for insert, query, delete marker, restart, and source span return.

Constraints:

- Do not call remote embedding services in tests.
- Use deterministic fake embeddings.
- Do not return raw full document when only source span is needed.

Acceptance Criteria:

- Query vector returns chunk hits with score and `source_ref`.
- Deleted chunks do not appear after searchable watermark.

Suggested paths:

- `geaflow-ai/src/main/java/org/apache/geaflow/ai/index/chunk`

Contributor guide

Open the contributing guide

Research direction

Start in geaflow-ai/src/main/java/org/apache/geaflow/ai/index/chunk and trace the existing GraphEntity indexing flow and VectorStore interface. Add tests for insert, query, delete markers, restart behavior, deterministic fake embeddings, and source-span returns. Done means queries return chunk hits with scores and source_ref, while deleted chunks disappear after the searchable watermark.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.