NASA-IMPACT / NASA-IMPACT/veda-data
Add a OCO-3 SAM Data data layer (high-level steps)
Open
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 9
- Forks
- 1
- Avg merge
- 1d 9m
- Merged PRs (30d)
- 1
Description
For each dataset, we will follow the following steps:
- Identify dataset and where it will be accessed from. Check it's a good source with science team. Ask about specific variables and required spatial and temporal extent. Note most datasets will require back processing (e.g. generating cloud-optimized data for historical data).
- If the dataset is ongoing (i.e. new files are continuously added and should be included in the dashboard), design and construct the forward-processing workflow.
- Each collection will have a workflow which includes discovering data files from the source, generating the cloud-optimized versions of the data and writing STAC metadata.
- Each collection will have different requirements for both the generation and scheduling of these steps, so a design step much be included for each new collection / data layer.
- Verify the COG output with the science team by sharing in a visual interface.
- Verify the metadata output with STAC API developers and any systems which may be depending on this STAC metadata (e.g. the front-end dev team).
- If the dataset should be backfilled, create and monitor the backward-processing workflow.
- Engage the science team to add any required background information on the methodology used to derive the dataset.
- Add the dataset to the production dashboard.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by identifying the existing dataset-layer and workflow conventions, then confirm the OCO-3 SAM source, variables, spatial and temporal scope with the science team. Done means the processing, COG and STAC metadata verification, required backfill, background information, and production dashboard integration are complete.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100