NASA-IMPACT / NASA-IMPACT/veda-odd
Publish Zarr + Virtual Zarr datasets to STAC
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 5
- Forks
- 0
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 3
Description
Just an idea, but what if we were to have a small STAC-GeoParquet catalog for these datasets? I think that could provide a few nice benefits:
- showing how to visualize the data from a STAC catalog matches typical use cases in the VEDA ecosystem better than from a URL as far as I can tell
- it would serve as an example of how to use STAC well with Zarr/Icechunk, following on from the guidance that Julia put together last year
- we could use the catalog for other, non-visualization use-cases
I guess the downsides are yet another STAC catalog and a bit more work relative to just testing URLs
Originally posted by @maxrjones in #384
We now have a set of virtual datasets that should be published to STAC so that they are easily discoverable for demonstration purposes and to prototype integration for future VEDA instances and services to utilize them.
Here is the guidance Julia wrote about how to publish Zarr datasets to STAC: https://guide.cloudnativegeo.org/cookbooks/zarr-stac-report/data-producers. I am curious @maxrjones if you think we should publish all of the underlying files as STAC items (ref: https://guide.cloudnativegeo.org/cookbooks/zarr-stac-report/data-producers/#virtual-dataset-in-an-external-file). At least, in the case of collections already published in CMR, we could point to a CMR query which returns all the associated underlying files.
To facilitate access, I would assume we would want to use the datacube and providers extension. I was also curious if there is an extension to store information about how to authenticate + authorize for access. A quick google search led me to https://github.com/stac-extensions/authentication whose most recent contributor is our very own @alukach 🎉
Datasets:
- NLDAS
- RASI
- MUR SST p1 + p2, see also concatenate_stores.ipynb
- Others from MAAP? Like the GEOS-CF Climatologies dataset?
- In-progress
Any others I am missing? @siddharth0248
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked Cloud Native Geo guidance on publishing Zarr datasets to STAC, then review the NLDAS, RASI, MUR SST, GEOS-CF, and GPM IMERG dataset links. Determine the catalog scope, item coverage, and relevant datacube, providers, and authentication extensions; done should mean an agreed, discoverable STAC catalog for the demonstration datasets.
Written by the indexing model from the issue text.
Assessment
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100