cloudwego / cloudwego/eino-ext
feat(valkey): add Valkey indexer and retriever components
- Dominant language
- Go
- Stars
- 811
- Forks
- 368
- Avg merge
- 16h 22m
- Merged PRs (30d)
- 13
Description
**Is your feature request related to a problem? Please describe.**
Eino currently has no native Valkey retriever or indexer. The existing Redis implementations (components/retriever/redis,
components/indexer/redis) use go-redis/v9, which may incidentally work against a Valkey server but isn't purpose-built for it — it doesn't
leverage the Valkey GLIDE client, can't adopt Valkey Search-specific features cleanly, and introduces go-redis as a transitive dependency
for users who have standardized on Valkey.
Users building RAG pipelines with Valkey as their vector store have no supported path today.
**Describe the solution you'd like**
Add native Valkey Retriever and Indexer components using the valkey-glide-go client:
- components/retriever/valkey/ — implements retriever.Retriever
- components/indexer/valkey/ — implements indexer.Indexer
Retriever capabilities:
- KNN and vector range queries via Valkey Search (FT.SEARCH)
- Hybrid search with filter expressions
- Configurable distance metric, vector field, return fields, dialect
- Embedding integration for automatic query vectorization
- Full Eino callback integration (OnStart, OnEnd, OnError)
Indexer capabilities:
- Batch document storage using pipeline operations (HSET)
- Automatic embedding generation with configurable batch size
- Support for both Hash and JSON document types
- Configurable key prefix and field mapping via DocumentToHashes function
- Full Eino callback integration
Module structure (separate go.mod per package per eino-ext conventions):
components/retriever/valkey/
├── go.mod, retriever.go, retriever_test.go, options.go, consts.go, README.md, examples/
components/indexer/valkey/
├── go.mod, indexer.go, indexer_test.go, consts.go, README.md, examples/
**Describe alternatives you've considered**
1. Adapting existing Redis components — rejected because valkey-glide-go has a different API surface from go-redis/v9. A clean
implementation avoids go-redis as a dependency and can adopt Valkey Search-specific features without backward-compatibility constraints.
2. Separate PRs for retriever and indexer — rejected because upstream convention is to submit both together (qdrant #389, milvus2 #628,
opensearch3 #614). They share client patterns and are useless without each other for a complete RAG pipeline.
**Additional context**
- Client: valkey-glide-go (official Valkey client with native Search command support)
- Backend: Valkey 9.1+ with Search module (valkey/valkey-bundle)
- Reference: components/retriever/redis/ and components/indexer/redis/
- Precedent PRs: qdrant #389, milvus2 #628, opensearch3 #614
Contributor guide
Research direction
Start by reading components/retriever/redis/ and components/indexer/redis/ to understand the existing component patterns, then review valkey-glide-go's Search and pipeline APIs. Implement both components under the specified valkey directories with separate go.mod files, tests, READMEs, and examples. Done means the requested search, indexing, embedding, callback, and document-type capabilities are covered by the tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100