NASA-IMPACT / NASA-IMPACT/veda-data

Mirror demo landsat-c212 collections to mcp-test and mcp-prod

Open
#134 5 comments 0 reactions 1 assignee View on GitHub

@anayeaye is already working on this.

Since May 31, 2024.

Dominant language
Jupyter Notebook
Stars
9
Forks
1
Avg merge
1d 9m
Merged PRs (30d)
1

Description

What

Seven demo lansdat spotlight collections were published to the staging catalog that need to be mirrored in the production account. Because these collections refer to externally hosted data (LPDAAC) and contain custom provider metadata we need to create (or re-use a one-off script to mirror this metadata in the production STAC catalog.

Note this is NOT a transfer or s3 discovery task, all we want to do is mirror the metadata from staging in the production catalog

Suggested steps

For each collection

  1. correct or remove the item_assets (there is no cog_default but there are many band-specific assets)
  2. Keep the summaries object (we can manually create it but there isn't an automated way to generate this for items we are not ingesting via airflow)
  3. publish collection and add to veda-data/production/collections
  4. use a stac client to select all items for this collection in the staging stac catalog and for each:
  • remove any self-referential links, i.e. self, parent, collection, root
  • publish the item using the ingestions/ endpoint
collections

'landsat-c2l2-sr-antarctic-glaciers-pine-island',
'landsat-c2l2-sr-antarctic-glaciers-thwaites',
'landsat-c2l2-sr-lakes-aral-sea',
'landsat-c2l2-sr-lakes-lake-balaton',
'landsat-c2l2-sr-lakes-lake-biwa',
'landsat-c2l2-sr-lakes-tonle-sap',
'landsat-c2l2-sr-lakes-vanern'

AC

  • collection metadata restored/added to ingestion-data/production/collections with existing summaries preserved and with corrected/removed item_assets
  • new or updated notebook for publishing a mirror of these collections to the production STAC catalog added to transformation-scripts/

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.