developmentseed / developmentseed/cng-sandbox

feat: multidimensional dataset support via titiler-multidim

Open
#113 1 comment 0 reactions 0 assignees View on GitHub
feat needs-design
Dominant language
TypeScript
Stars
3
Forks
0
Avg merge
2h 21m
Merged PRs (30d)
3

Description

## Summary

Add support for interactive visualization of multidimensional datasets (NetCDF/HDF5 with dimensions beyond x, y, and time). Instead of forcing users to pick a single 2D slice at upload time, keep the original file intact and let users select variables and dimension values in the map view.

## Motivation

User feedback from a researcher working with a 5D dataset (x, y, time, forest pest, driver variable):

> "I've been working on making maps from a 5-dimensional dataset (x, y, time, forest pest, driver variable). I was hoping I would be able to pick a combination of pest, variable, and time and see a map since my collaborators seem to be very interested in that."

The current pipeline flattens NetCDF/HDF5 to COGs during ingestion, losing the multidimensional structure entirely. Users working with scientific datasets that have extra dimensions (depth, species, scenario, etc.) can't explore their data interactively.

## Approach

Add a second tile-serving path using [titiler-multidim](https://github.com/developmentseed/titiler-multidim) (built on `titiler.xarray`). This serves tiles directly from NetCDF files, slicing dimensions on-the-fly via query parameters.

```
Simple raster → COG pipeline → titiler-pgstac (unchanged)
Multidim file → raw to R2 → titiler-multidim (new)
```

The detection heuristic: any NetCDF/HDF5 variable with dimensions beyond (x, y) and (x, y, time) — i.e., extra dimensions like "pest", "driver", "depth" — routes to the multidim path. The existing COG pipeline remains untouched for simple rasters.

## Implementation Issues

1. #114 — Infrastructure: titiler-multidim container and proxy routes
2. #115 — Backend: extend scanner to detect extra dimensions
3. #116 — Backend: multidim ingestion pipeline (skip COG conversion)
4. #117 — Frontend: dimension controls UI in map view
5. #118 — Upload UX for multidim datasets

Issues should be worked in order (1→5), though 1 and 2 can be done in parallel.

## Risks

- **R2 + s3fs compatibility**: `titiler.xarray` uses `s3fs` which needs `AWS_ENDPOINT_URL` for R2. Should work but needs testing.
- **Performance**: Serving tiles from raw NetCDF over S3 without chunking could be slow for large files. Zarr conversion is a future optimization.
- **CRS**: NetCDF files with non-standard CRS metadata may not tile correctly via `rioxarray`.
- **titiler.xarray API surface**: Need to verify exact endpoint paths and query param names during implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.