Heatmaps ship 4-16 bytes/cell for what the client uploads as 8-bit textures

Open
#164 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Quiet
Tech stack
javascript, python
Domain
data-visualization, performance

Research direction

Read python/xy/_payload.py:529-554 and python/xy/marks.py:2395-2401 to trace the heatmap payloads, then inspect channels.py:540-541 and js/src/50_chartview.ts:2742-2759. Compare the truecolor and scalar paths with the density-grid precedent at _payload.py:988-1000. Done means quantized heatmap textures use an appropriate client texture branch without the conversion loop, while hover behavior follows the chosen precision decision.

Written by the indexing model from the issue text.

Description

needs investigate performance

From a full data-transport audit at b0f8780.

Today:

  • Truecolor heatmaps ship RGBA as four separate f32 planes (rgba_bufs, python/xy/_payload.py:529-547) = 16 B/cell on the wire; the client immediately clamps/rounds into a Uint8Array(w*h*4) (js/src/50_chartview.ts:2742-2759) with a per-texel JS loop. Canonical storage is four f64 planes (32 B/cell, strided-plane copies at python/xy/marks.py:2395-2401).
  • Scalar heatmaps ship the normalized grid as f32 (_payload.py:553-554) for an R8 texture — 4× the needed bytes.
  • Contrast: density grids already ship log-u8 at 1 B/cell with exact-max restoration (_payload.py:988-1000) — the precedent exists.

Cost: a 2000×2000 imshow ships 64 MB instead of 16 MB (or ~4 MB as u8 RGBA); 1000×1000 scalar heatmap ships 4 MB vs 1 MB.

Proposed fix: quantize kernel-side — interleaved RGBA8 via ship_u8 for truecolor (the direct_rgba scatter path already packs u8 RGBA, channels.py:540-541), u8 (or u16) + domain for scalar. Client gets a texture-format branch and drops the conversion loop. Live hover precision is unaffected (pick reads canonical f64); standalone hover readout (_cpuHeatmap) would quantize ~0.4% — decide whether to accept that or keep f32 for standalone only.

Note: a related correctness bug (in-place mutation of user 0-255 RGBA input at marks.py:2349-2351) is tracked separately.

Dominant language
Python
Stars
1.8k
Forks
76
Avg merge
1h 7m
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from reflex-dev/xy

All issues in reflex-dev/xy

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.