Altinity / Altinity/altinity-sql-browser
GS: Failure-safe full refresh and large-result transfer
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 8
- Forks
- 2
- Avg merge
- 1h 34m
- Merged PRs (30d)
- 6
Description
Parent: #398
Depends on: #399, #401
Design:
docs/design/google-sheets/full-refresh.mddocs/design/google-sheets/large-results.mddocs/design/google-sheets/failure-recovery.md
Scope
Implement manual full refresh with streaming ClickHouse reads, canonical value conversion, hidden staging, stable-sheet publication, limits, progress, cancellation, and failure recovery.
Acceptance criteria
- The complete query result is never required in browser memory.
- Google writes use byte-bounded explicit ranges and RAW semantics.
- Existing live data remains unchanged until all query and staging work succeeds.
- Final publication preserves the managed numeric sheet ID and atomically updates state.
- Old trailing rows/columns are cleared and formatting/metadata are refreshed.
- Ambiguous publication is reconciled by generation/refresh ID before retry.
- Row, cell, byte, duration, and spreadsheet-capacity limits are enforced without truncation.
- Cancellation and closed-tab recovery leave committed data intact and clean orphan staging later.
- Successful full refresh can initialize/reset an append checkpoint when eligible.
Non-goals
Append page selection, automatic scheduling, or in-place partial replacement when staging cannot fit.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read docs/design/google-sheets/full-refresh.md, docs/design/google-sheets/large-results.md, and docs/design/google-sheets/failure-recovery.md, then review prerequisites #399 and #401. Done means implementing the listed refresh, staging, publication, limits, cancellation, recovery, and checkpoint acceptance criteria without truncation or changing committed live data prematurely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, typescript
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100