apache / apache/geaflow

Add a Redis connector (source + sink)

Open
#799 1 comment 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Java
Stars
808
Forks
188
Avg merge
3d 22h
Merged PRs (30d)
2

Description

### What & why

The project already has a Redis **store** (a state backend) but no Redis
**connector** (treating Redis as a data source/sink). They serve different roles:
the store backs the engine's internal state, while a connector backs DSL table
reads/writes. Redis as a dimension table or result cache is very common.

项目已有 Redis **store**(状态后端),但没有 Redis **connector**(把 Redis 当数据源/汇)。
两者职责不同:store 服务于引擎内部状态,connector 服务于 DSL 建表读写。Redis 作维表/结果缓存很常见。

### The task

Create `geaflow-dsl-connector-redis`: the sink supports writing string/hash, the
source supports scanning by key pattern.

新建 `geaflow-dsl-connector-redis`,sink 支持写 string/hash,source 支持按 key 模式扫描读取。

### Where to look / 怎么做

1. References: the minimal `geaflow-dsl-connector-console` (sink only) and the
connection-managing `geaflow-dsl-connector-elasticsearch`.
2. **Reuse the Redis client already in the project** — check which client
`geaflow-store-redis` uses and stick with it to avoid pulling in a new dependency.
3. Implement connector / source / sink + SPI registration + pom + docs + tests.

### Done when

- [ ] Both source and sink work, supporting string and hash (at least one, with explanation)
- [ ] Reuses the existing Redis client dependency, no duplicate dependency introduced
- [ ] Testcontainers integration test
- [ ] CN + EN docs + examples
- [ ] checkstyle / RAT pass

Contributor guide

Open the contributing guide

Research direction

Start by comparing geaflow-dsl-connector-console and geaflow-dsl-connector-elasticsearch, then inspect the Redis client used by geaflow-store-redis. Add the new connector module with source, sink, SPI registration, pom, tests, documentation, and examples. Done means source and sink support the specified Redis values, reuse the existing client, and checkstyle, RAT, and Testcontainers integration checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, redis
Domain
backend, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.