developmentseed / developmentseed/cng-sandbox
feat: multidimensional dataset support via titiler-multidim
- Dominant language
- TypeScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 2h 21m
- Merged PRs (30d)
- 3
Description
## Summary
Add support for interactive visualization of multidimensional datasets (NetCDF/HDF5 with dimensions beyond x, y, and time). Instead of forcing users to pick a single 2D slice at upload time, keep the original file intact and let users select variables and dimension values in the map view.
## Motivation
User feedback from a researcher working with a 5D dataset (x, y, time, forest pest, driver variable):
> "I've been working on making maps from a 5-dimensional dataset (x, y, time, forest pest, driver variable). I was hoping I would be able to pick a combination of pest, variable, and time and see a map since my collaborators seem to be very interested in that."
The current pipeline flattens NetCDF/HDF5 to COGs during ingestion, losing the multidimensional structure entirely. Users working with scientific datasets that have extra dimensions (depth, species, scenario, etc.) can't explore their data interactively.
## Approach
Add a second tile-serving path using [titiler-multidim](https://github.com/developmentseed/titiler-multidim) (built on `titiler.xarray`). This serves tiles directly from NetCDF files, slicing dimensions on-the-fly via query parameters.
```
Simple raster → COG pipeline → titiler-pgstac (unchanged)
Multidim file → raw to R2 → titiler-multidim (new)
```
The detection heuristic: any NetCDF/HDF5 variable with dimensions beyond (x, y) and (x, y, time) — i.e., extra dimensions like "pest", "driver", "depth" — routes to the multidim path. The existing COG pipeline remains untouched for simple rasters.
## Implementation Issues
1. #114 — Infrastructure: titiler-multidim container and proxy routes
2. #115 — Backend: extend scanner to detect extra dimensions
3. #116 — Backend: multidim ingestion pipeline (skip COG conversion)
4. #117 — Frontend: dimension controls UI in map view
5. #118 — Upload UX for multidim datasets
Issues should be worked in order (1→5), though 1 and 2 can be done in parallel.
## Risks
- **R2 + s3fs compatibility**: `titiler.xarray` uses `s3fs` which needs `AWS_ENDPOINT_URL` for R2. Should work but needs testing.
- **Performance**: Serving tiles from raw NetCDF over S3 without chunking could be slow for large files. Zarr conversion is a future optimization.
- **CRS**: NetCDF files with non-standard CRS metadata may not tile correctly via `rioxarray`.
- **titiler.xarray API surface**: Need to verify exact endpoint paths and query param names during implementation.
Contributor guide
Assessment
This issue has not been assessed yet.