Altinity / Altinity/altinity-sql-browser
Support safe live-tail results for SELECT … STREAM
まだ誰も着手していません。
- 主要言語
- TypeScript
- スター
- 8
- フォーク
- 2
- 平均マージ
- 1時間 34分
- マージ済み PR(30日)
- 6
説明
Context
ClickHouse 26.6 adds experimental continuous queries with SELECT … STREAM.
The browser already has useful building blocks for this:
- HTTP response streaming via
resp.body.getReader() - line-by-line
JSONStringsEachRowWithProgressprocessing - no
wait_end_of_queryin the streaming result path - Stop already aborts the fetch and kills the query by
query_id - mid-stream exceptions are already surfaced from in-band
{"exception"}rows
This issue is about making SELECT … STREAM safe and understandable in the browser.
It is not about enabling ClickHouse streaming settings. The browser should not inject
enable_streaming_queries or table settings. Those remain user/session/server choices.
Problem
A streaming query is open-ended. The current UI/result model mostly assumes a bounded query that eventually finishes.
Main gaps:
result.rows.push(...)grows memory without bound.- Every chunk can trigger a full table rebuild, which becomes janky over long streams.
- The lifecycle reads like a normal query: “Running…”, then success/history. A stream should read as live, and Stop is the normal terminus.
- The user needs a tailing experience: newest rows visible by default, with a way to pause auto-follow when scrolling up.
Proposed work
- Add
detectStreaming(sql)as a pure core helper.- Detect
STREAMas a table-expression modifier. - Avoid naive matching inside strings/comments where practical.
- Prefer conservative detection; false negatives are safer than false positives.
- Detect
- Add a bounded row buffer.
- Keep last N rows, for example 10k.
- Track total received rows.
- Show “showing last N of M rows” when older rows are dropped.
- Throttle rendering.
- Batch incoming chunks.
- Render at a fixed cadence, for example around 10 fps.
- Keep the throttle decision testable in core; keep rAF wiring in UI.
- Add table tail behavior.
- Auto-scroll to newest rows while the user is already at the bottom.
- Pause auto-follow when the user scrolls up.
- Show “Jump to latest” when paused.
- Add live lifecycle UI.
- Label state as “Streaming…” or “Live”.
- Make Stop the primary action.
- Show received row count and rows/sec.
- Treat user Stop as a normal stream termination, not an error.
- Improve capability errors.
- If the server rejects streaming as disabled/unsupported, show a friendly hint:
ClickHouse streaming queries require a compatible server and user-enabled
enable_streaming_queries.
- If the server rejects streaming as disabled/unsupported, show a friendly hint:
Acceptance criteria
- A
SELECT … STREAMquery can run without unbounded memory growth. - Long-running streams do not rebuild the full table on every chunk.
- The UI clearly distinguishes live streams from finite queries.
- Stop cleanly terminates the stream and does not look like a failure.
- The table follows the newest rows by default and supports pause/jump-to-latest.
- A finite
STREAM LIMITquery can complete normally. - Open streams are not automatically recorded as successful finite history entries.
- Core behavior has unit coverage: streaming detection, ring buffer, received/dropped counters, and render-throttle decisions.
Files likely touched
| File | Change |
|---|---|
src/core/format.js |
detectStreaming(sql) |
src/core/stream.js |
bounded buffer, received counter, dropped counter, flush decision |
src/ui/app.js |
live-mode branch, Stop lifecycle, throttled chunk handling |
src/ui/results.js |
tail auto-scroll, pause, jump-to-latest, live readout |
src/styles.css |
live state styling |
tests/unit/* |
detection, buffer, throttle tests |
README |
document user-controlled ClickHouse streaming setup |
Open questions
- Should live mode be auto-detected only, manually toggled, or both?
- What should the default tail window be: 5k, 10k, or configurable?
- Should
STREAM LIMITbe recorded to history once it completes? - Should Stop on a stream be recorded anywhere, or treated as ephemeral?
- How strict should
detectStreaming(sql)be before we need a real parser?
Non-goals
- Enabling ClickHouse settings on behalf of the user.
- Table DDL changes for cursor/minmax optimization.
- Live chart animation or aggregation. That is tracked separately.
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
src/core/format.js と src/core/stream.js から始め、次に src/ui/app.js と src/ui/results.js を通して結果のライフサイクルを追跡します。detection、buffering、counters、throttling のカバレッジを拡張する前に、tests/unit/* 配下の関連するテストを実行します。完了の条件は、ライブ結果が上限付きであること、明確なストリーミングライフサイクルと tail controls があること、Stop の動作が適切であること、有限ストリームが完了すること、そして README のガイダンスが更新されていることです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- sql, typescript
- 領域
- documentation, frontend, testing
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 静か
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 42/100