llmware-ai / llmware-ai/llmware

feat: Add Valkey as a vector database option (using Valkey GLIDE)

Open
#1,300 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.8k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

Summary

Add Valkey as a supported vector database in llmware, using the official Valkey GLIDE sync client (valkey-glide-sync).

Related discussion: #1296

Motivation

  • Valkey is the open-source (BSD-3-Clause) successor to Redis, backed by the Linux Foundation
  • The valkey-search module provides vector similarity search with HNSW/FLAT indexing, plus full-text, tag, and numeric range queries
  • Cloud-native: AWS ElastiCache 9.0+ and Google Cloud Memorystore both support Valkey with vector search
  • The existing EmbeddingRedis class uses redis-py which targets the SSPL-licensed Redis server

Implementation

Following the existing pattern (EmbeddingRedis, EmbeddingMilvus, etc.):

  • ValkeyConfig in configs.py — host, port, credentials, TLS (all env-var backed)
  • EmbeddingValkey in embeddings.py — standalone class implementing create_new_embedding, search_index, delete_index
  • Registry"valkey" added to VectorDBRegistry and _supported["vector_db"]
  • Dependencyvalkey-glide-sync (lazy-imported, optional)
  • Tests — unit tests for config/registry + integration tests for create/search/delete

Requirements

  • Valkey server >= 9.1 with valkey-search module >= 1.2.0
  • Python: pip install valkey-glide-sync (v2.3.1+, supports Python 3.9-3.13)
  • Docker for testing: docker run -d -p 6379:6379 valkey/valkey-bundle:9.1.0-rc2

Design Decisions

  1. Standalone class — API differences with GLIDE vs redis-py make a shared base impractical
  2. Sync client only — matches the synchronous pattern used by all other embedding classes
  3. No cluster mode — standalone only for initial implementation
  4. Individual hset writes — HNSW indexing processes writes synchronously, causing Batch/pipeline timeouts; individual calls are reliable and match EmbeddingRedis pattern

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with configs.py, embeddings.py, and the VectorDBRegistry or _supported["vector_db"] definitions, following EmbeddingRedis and EmbeddingMilvus. Review the Valkey GLIDE sync client and the stated server and module requirements before adding unit and integration tests. Done means the valkey option supports configuration, create/search/delete operations, registry discovery, and passes tests against the specified Docker server.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
database, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.