Add CCI Biomass product from 2010, 2017 and 2018 to the VEDA STAC API (high-level steps)

NOTE: The dataset ingest + publication workflows are currently undergoing a refactor in this branch: https://github.com/NASA-IMPACT/cloud-optimized-data-pipelines/tree/refactor

Identify the dataset and what the processing needs are

1. Identify dataset and where it will be accessed from. Check it's a good source with science team. Ask about specific variables and required spatial and temporal extent. Note most datasets will require back processing (e.g. generating cloud-optimized data for historical data).

   Looks like we can access the geotiffs here: https://data.ceda.ac.uk/neodc/esacci/biomass/data/agb/maps/v2.0/geotiff

   I'm asking Heather Kay if this is the right source and if anyone has capacity to do the transfer to our S3 bucket.

   There are a lot of files however seems self evident we would publish one item per file, but should verify they are valid COGs before doing so.

Design the metadata and publish to the Dev API

1. Review conventions for generating STAC collection and item metadata:
   • Collections: https://github.com/NASA-IMPACT/delta-backend/issues/29 and STAC version 1.0 specification for collections
   • Items: https://github.com/NASA-IMPACT/delta-backend/issues/28 and STAC version 1.0 specification for items
   • NOTE: The delta-backend instructions are specific to datasets for the climate dashboard, however not all datasets are going to be a part of the visual layers for the dashboard so I believe you can ignore the instructions that are specific to "dashboard" extension, "item_assets" in the collection and "cog_default" asset type in the item.

A collection will need the following fields, some of which may be self-evident through the filename or an about page for the product, however there are many cases in which we may need to reach out to product owners to define the right values for these fields:

• temporal interval
• license
• id
• title
• description
• whether it is periodic or not on the dashboard
• the dashboard time density

2. After reviewing the STAC documentation for collections and items and reviewing existing scripts for generating collection metadata (generally with SQL) and item metadata, generate or reuse scripts for your collection and a few items to publish to the testing API.

Review documentation on how to do ingest and publish here: https://github.com/NASA-IMPACT/cloud-optimized-data-pipelines/blob/769fb44272b3613ad781ec6814d1c49e5e729e36/OPERATING.md

Note this documentation is a WIP and that this dataset is the first to be accessed from an HTTP so probably needs to be transferred to s3://veda-data-store-staging first.

4. If necessary, request access and credentials to the dev database and ingest and publish to the Dev API. Submit a PR with the any changes used to run the workflow to publish to the Dev API and include links to the published datasets in the Dev API

Publish to the Staging API

Once the PR is approved, we can merge and publish those datasets to the Staging API

Start by reviewing the refactor branch and OPERATING.md, then inspect the CEDA GeoTIFF source and the collection and item conventions in delta-backend issues 29 and 28. Determine the required metadata and transfer or validation steps before preparing scripts for a test collection and sample items. Done means the dataset is published to the Dev API, with workflow changes in a PR and links to the published data.