NASA-IMPACT / NASA-IMPACT/veda-odd

Publish Zarr + Virtual Zarr datasets to STAC

Open
#392 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
5
Forks
0
Avg merge
4d 19h
Merged PRs (30d)
3

Description

Just an idea, but what if we were to have a small STAC-GeoParquet catalog for these datasets? I think that could provide a few nice benefits:

  1. showing how to visualize the data from a STAC catalog matches typical use cases in the VEDA ecosystem better than from a URL as far as I can tell
  2. it would serve as an example of how to use STAC well with Zarr/Icechunk, following on from the guidance that Julia put together last year
  3. we could use the catalog for other, non-visualization use-cases

I guess the downsides are yet another STAC catalog and a bit more work relative to just testing URLs

Originally posted by @maxrjones in #384

We now have a set of virtual datasets that should be published to STAC so that they are easily discoverable for demonstration purposes and to prototype integration for future VEDA instances and services to utilize them.

Here is the guidance Julia wrote about how to publish Zarr datasets to STAC: https://guide.cloudnativegeo.org/cookbooks/zarr-stac-report/data-producers. I am curious @maxrjones if you think we should publish all of the underlying files as STAC items (ref: https://guide.cloudnativegeo.org/cookbooks/zarr-stac-report/data-producers/#virtual-dataset-in-an-external-file). At least, in the case of collections already published in CMR, we could point to a CMR query which returns all the associated underlying files.

To facilitate access, I would assume we would want to use the datacube and providers extension. I was also curious if there is an extension to store information about how to authenticate + authorize for access. A quick google search led me to https://github.com/stac-extensions/authentication whose most recent contributor is our very own @alukach 🎉

Datasets:

Any others I am missing? @siddharth0248

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked Cloud Native Geo guidance on publishing Zarr datasets to STAC, then review the NLDAS, RASI, MUR SST, GEOS-CF, and GPM IMERG dataset links. Determine the catalog scope, item coverage, and relevant datacube, providers, and authentication extensions; done should mean an agreed, discoverable STAC catalog for the demonstration datasets.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.