Altinity / Altinity/altinity-sql-browser
GS: ClickHouse 26.8+ commit-cursor append refresh
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 8
- Forks
- 2
- Avg merge
- 1h 34m
- Merged PRs (30d)
- 6
Description
Parent: #398
Depends on: #400, #401, #402
Design:
docs/design/google-sheets/append-refresh.mddocs/design/google-sheets/decisions.md
Scope
Implement capability-gated append-only refresh for eligible direct MergeTree/ReplicatedMergeTree sources using per-partition persistent _block_number and _block_offset checkpoints and finite bounded SELECT queries.
Acceptance criteria
- Append UI requires configured minimum version (initially 26.8) and successful runtime capability probes.
- Eligible query validation permits one deterministic direct source and rejects joins, views, FINAL, aggregation, DISTINCT, window functions, UNION, table functions, and business LIMIT.
- Binding pins table UUID, query hash, schema hash, source-subset parameters, and initialization mode.
- Initial modes support history export and future-only checkpoints.
- Refresh captures finite per-partition upper boundaries.
- Pages are ordered and filtered by
_block_number/_block_offsetand use decimal-string UInt64 values. - Each Google commit copies exact staged rows and advances only the corresponding partition checkpoint in the same publication batch.
- A closed tab resumes from the last committed page without duplicates.
- Invalid capability, UUID, query, schema, parameter, row-count, or checkpoint state requires explicit full refresh.
-
SELECT ... STREAM, updates, deletes, and silent fallback are absent.
Concurrency
Use advisory lease metadata and generation rechecks. Strict distributed locking is out of scope.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading docs/design/google-sheets/append-refresh.md and docs/design/google-sheets/decisions.md, then review dependencies #400, #401, and #402. Use the listed acceptance criteria to define completion, including capability-gated validation, persistent partition checkpoints, bounded refresh pages, exact Google commits, resume behavior, and explicit full-refresh handling for invalid state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, typescript
- Domain
- backend, database, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100