worktools / worktools/unionid

[设计 / Design] CDC、订阅与物化视图评估 / evaluate CDC, subscriptions, and materialized views

Open
#253 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

deferred design priority/P2
Dominant language
Rust
Stars
0
Forks
0
Avg merge
57m
Merged PRs (30d)
209

Description

中文说明

版本、顺序与当前进度

版本待定 / P2 / deferred,不再将探索池命名为 v1.0 发布承诺。有限 NDJSON stream 不等于 CDC;须用真实增量订阅场景定义日志、重放、积压和预算后另排版本。

问题

数据库不负责 CDC、订阅或物化视图(APPLICATION_DATA.md 明确边界);有限 NDJSON stream 是单次查询结果,不是实时更新流。中等业务常需要“数据变更后驱动缓存失效/事件总线/派生视图”。当前只能轮询或应用双写,容易产生不一致。

范围
  • 评估最小形态:按 sequence/RowId 的有界变更日志读取、订阅游标、以及物化/派生视图的候选方案;明确与幂等 receipt、cursor、migration、compact 的交互。
  • 明确投递语义(at-least-once/游标消费)、保留窗口与清理,不承诺 exactly-once 推送。
  • 评估在单机单写者下的可行性,以及未来复制场景的衔接。
  • 先 RFC 与证据;不把实时流作为当前版本范围。
验收条件
  • RFC 冻结变更日志/订阅/物化视图的最小语义、游标与保留策略。
  • 一个原型证明消费者可从某个 sequence 恢复并观察到一致变更。
  • 明确与备份、migration、compact 及现有 stream 的关系。
  • 若不采纳,说明替代方案(轮询/应用事件表)。

English Description

Version, sequence, and current progress

Version TBD / P2 / deferred, not a v1.0 release promise. Finite NDJSON streams are not CDC; require real incremental-subscription needs and log/replay/backlog/budget contracts before scheduling a release.

Problem

The database owns no CDC, subscriptions, or materialized views (explicitly out of scope in APPLICATION_DATA.md); the finite NDJSON stream is a single query result, not a live update stream. Medium businesses often need "on change, invalidate cache / drive an event bus / maintain a derived view". Today this requires polling or dual writes and drifts.

Scope
  • Evaluate a minimal shape: a bounded change-log read by sequence/RowId, subscription cursors, and candidate materialized/derived views; define interaction with idempotency receipts, cursors, migration, and compaction.
  • Define delivery semantics (at-least-once/cursor consumption), retention windows, and cleanup; do not promise exactly-once push.
  • Evaluate feasibility under single-node single-writer and the path to future replication.
  • RFC and evidence first; live streaming is not in the current scope.
Acceptance Criteria
  • An RFC freezes minimal change-log/subscription/materialized-view semantics, cursors, and retention.
  • A prototype shows a consumer can resume from a sequence and observe consistent changes.
  • The relationship with backup, migration, compaction, and the existing stream is stated.
  • If rejected, alternatives (polling/event table) are documented.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading APPLICATION_DATA.md to confirm the current database boundary, then inspect the existing finite NDJSON stream and related concepts named in the issue: receipts, cursors, migration, and compaction. Done means an RFC defines the minimal semantics and retention policy, a prototype demonstrates sequence-based recovery, and the relationships to backup, migration, compaction, and alternatives are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.