developmentseed / developmentseed/cng-sandbox

feat: H3 hexagonal aggregation layer for vector datasets and GeoParquet connections

Open
#401 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3
Forks
0
Avg merge
2h 21m
Merged PRs (30d)
3

Description

## Summary

Add H3 hexagonal binning as a server-side indexing step for vector data, with a corresponding `H3HexagonLayer` rendering option in the frontend. deck.gl already ships `H3HexagonLayer`, so the rendering cost is low — the main work is in the ingestion pipeline and layer config UI.

## Why H3 and not just polygon fill?

H3 aggregation is useful when a vector dataset has many point or polygon features and the user wants to visualize density or aggregate statistics (count, average value) rather than raw geometries. Hexagonal binning produces smoother visual patterns than square grids and is well-suited to geospatial professional audiences.

## Scope

### 1. Ingestion pipeline (uploaded datasets)

After a GeoJSON/Shapefile is converted to GeoParquet, optionally run a DuckDB H3 indexing step:

- Use DuckDB's [`h3` extension](https://duckdb.org/docs/extensions/h3.html) to compute H3 cell indices from the geometry centroid
- Add an `h3_index` column at a default resolution (e.g. res 5 or 6 — to be decided based on typical dataset extents)
- Persist a flag on the dataset (`has_h3_index: bool`, `h3_resolution: int`) so the frontend knows the column is available
- This step is opt-in or auto-applied only to point/polygon datasets — skip for purely linear geometries

### 2. GeoParquet connections

For connections registered via `POST /api/connections` with a GeoParquet URL:

- **Server path** (`render_path="server"`): run the DuckDB H3 step as part of the existing tippecanoe → PMTiles conversion job; store `h3_resolution` on the connection row
- **Client path** (`render_path="client"`): DuckDB-WASM has the H3 extension available; H3 indexing can run in-browser on the streamed parquet before passing data to `H3HexagonLayer`

This means H3 is available for both uploaded datasets and remote connections — the only difference is where the indexing runs.

### 3. Frontend layer config

- Add `H3HexagonLayer` as a selectable layer type in the layer config panel, visible only when the dataset/connection has `has_h3_index: true`
- Controls needed: aggregation function (count / sum / mean), color scale, opacity, resolution override (optional advanced toggle)
- This is a rendering mode toggle alongside the existing point/polygon renderers — not a separate viewer

## Open questions

- [ ] What H3 resolution(s) to pre-compute? Single fixed resolution, or multiple (e.g. res 4–7)? Multiple resolutions are more flexible but multiply storage/compute cost.
- [ ] Should H3 indexing be automatic (applied to all eligible vector datasets on ingest) or opt-in (user-triggered after upload)?
- [ ] For the client render path, is the DuckDB-WASM H3 extension already loaded in the frontend bundle, or does it need to be added?
- [ ] How should aggregation value be chosen when features have no numeric attribute? (Default to count.)

## Out of scope

- Raster datasets (COGs, NetCDF, Zarr) — H3 binning is not meaningful for pre-gridded raster data
- H3 export (saving the indexed GeoParquet back to the user) — rendering only for now

## References

- deck.gl `H3HexagonLayer`: https://deck.gl/docs/api-reference/geo-layers/h3-hexagon-layer
- DuckDB H3 extension: https://duckdb.org/docs/extensions/h3.html
- h3-js (client-side fallback): https://github.com/uber/h3-js

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.