Altinity / Altinity/altinity-sql-browser

Live visualizations for streaming queries: sliding-window aggregates and animated updates

Offen
#61 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

enhancement
Vorherrschende Sprache
TypeScript
Sterne
8
Forks
2
Ø Merge
1 Std. 34 Min.
Gemergte PRs (30 T.)
6

Beschreibung

Context

#59 covers the safe baseline for ClickHouse SELECT … STREAM: bounded memory,
throttled rendering, live lifecycle, Stop behavior, and table tailing.

This issue covers the next layer: making streaming results visually useful.

The goal is not just an appending table. The goal is live values that visibly move:
bars grow and reorder, pie slices resize, stat cells tick, and changed values are easy
to notice.

Problem

The ClickHouse stream is append-only raw rows. For live charts, the browser cannot
assume the server sends updated aggregate rows.

The browser needs a client-side rollup over a bounded window, then it needs to update
existing visualizations in place instead of rebuilding them for every chunk.

Proposed work

  • Add a pure client-side aggregation window in src/core/.
    • count-based window, for example last N rows;
    • optional time-based window, for example last 60 seconds when a timestamp column exists;
    • group-by accumulators for count/sum/avg/min/max where feasible;
    • deterministic eviction so memory stays bounded.
  • Reuse existing chart selection/pivot logic where possible, especially src/core/chart-data.js.
  • Update Chart.js instances in place with chart.update() instead of destroying/recreating the canvas per chunk.
  • Batch chart updates on the same render cadence as the streaming table.
  • Add live stat/table-cell affordances.
    • changed numeric cells briefly highlight or count up;
    • support prefers-reduced-motion;
    • show the active window, for example last 10k rows or last 60s.
  • Keep the raw tail table and aggregate chart views as two views over the same feed state.

Suggested architecture

Area Proposed location Notes
Window policy src/core/stream.js or src/core/stream-window.js Count/time window, eviction, received/dropped counters
Aggregation src/core/stream-aggregate.js Pure fold/evict logic
Chart model src/core/chart-data.js Reuse/extend existing chart preparation
Render scheduling src/core/stream.js + UI rAF glue Pure throttle decision; UI owns timers
Animation/UI src/ui/results.js, src/styles.css In-place chart updates, cell transitions, reduced-motion support

Acceptance criteria

  • A streaming query can feed a chart without unbounded memory growth.
  • Chart updates are batched and smooth under sustained row arrival.
  • Existing chart instances are updated in place where possible.
  • The UI makes the aggregation window explicit.
  • Aggregation, eviction, null handling, and update batching decisions have unit coverage.
  • Motion can be reduced through standard reduced-motion preferences.

Open questions

  1. Should the default chart window be last-N-rows, time-based, or user-selectable per query?
  2. Should dimension/measure selection reuse the current Chart view controls, or should live mode expose a simplified picker?
  3. Which aggregations are first scope: count only, count/sum/avg, or broader numeric aggregations?
  4. How should high-cardinality dimensions work: top-K with Other, hard cap, or explicit user choice?
  5. Should a STREAM LIMIT query use the live chart path while running and then settle into a normal finite chart when complete?

Non-goals

  • Enabling ClickHouse streaming settings on behalf of the user.
  • Server-side incremental aggregation.
  • A new charting dependency unless Chart.js proves insufficient.

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne damit, src/core/stream.js, src/core/chart-data.js, src/ui/results.js und src/styles.css zu lesen, um die bestehenden Streaming- und Diagrammpfade zu verstehen. Kläre die aufgeführten Fragen zu Fenster, Aggregation, Diagrammsteuerung und hoher Kardinalität, bevor du mit der Implementierung beginnst. Als abgeschlossen gilt die Arbeit, wenn eine begrenzte Aggregation, gebündelte Diagrammaktualisierungen an Ort und Stelle, ein expliziter Fensterstatus, Unterstützung für reduzierte Bewegungen und eine Unit-Test-Abdeckung für Aggregations- und Eviction-Entscheidungen vorhanden sind.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
chart.js, typescript
Bereich
data-visualization, frontend, stream-processing, testing-qa
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.