NASA-IMPACT / NASA-IMPACT/veda-data
Add new LIS-based Western US Datasets
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 9
- Forks
- 1
- Avg merge
- 1d 9m
- Merged PRs (30d)
- 1
Description
-
Identify the point of contact and ensure someone is providing them updates: @TimLahmers
-
Data provider has read guidelines on data preparation
-
Identify data location:
NASA-LIS Variables:
s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE GPP_ANOM/
s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE GPP_PCTL/
s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE PRECIP/
s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE RZSM/
LIS-Derived FWI Quantities:
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE BUI/
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE DC/
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE DMC/
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE FFMC/
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE FWI/
s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE ISI/ -
Number of items: 10
-
Verify that files are valid COGs (e.g. with
rio cogeo validate) -
Gather STAC collection metadata
- id:
- title: LIS-Derived Fire Variables
- description: NASA-LIS (with multivariate snow, soil moisture, TWS, and LAI assimilation) Surface Anomalies, including Monthly Precipitation Anomalies, Daily Root Zone Soil Moisture Percentiles, Daily GPP Percentiles, and Daily GPP Anomalies and components of Fire Weather Index (FWI) from LIS forcing variables. FWI quantities include Fine Fuel Moisture Code (FFMC), Duff Moisture Code (DMC), Drought Code (DC), Buildup Index (BUI), Initial Spread Index (ISI), and Fire Weather Index (FWI). FWI and LIS surface variables are for the Western US with 0.01 degree (~1-km) resolution.
- license:
- temporal interval: 24-Hours
- whether it is periodic on the dashboard (periodic = regular time series of layers without gaps): FWI 01-2002 - 10-2021; LIS 07-2002 - 10-2021
- the dashboard time density:
-
Review and follow https://github.com/NASA-IMPACT/cloud-optimized-data-pipelines/blob/main/OPERATING.md
-
Open PR for publishing those datasets to the Staging API:
-
Notify QA / move ticket to QA state
-
Once approved, merge and close.
Resources on metadata
If not already familiar with these conventions for generating STAC collection and item metadata:
- Collections: https://github.com/NASA-IMPACT/delta-backend/issues/29 and STAC version 1.0 specification for collections
- Items: https://github.com/NASA-IMPACT/delta-backend/issues/28 and STAC version 1.0 specification for items
- NOTE: The delta-backend instructions are specific to datasets for the climate dashboard, however not all datasets are going to be a part of the visual layers for the dashboard so you can ignore the instructions that are specific to "dashboard" extension, "item_assets" in the collection and "cog_default" asset type in the item.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with how-to.md's data-preparation guidance and the cloud-optimized-data-pipelines OPERATING.md instructions. Review the linked STAC collection and item metadata references, then prepare metadata for the ten listed S3 datasets and open a PR for the Staging API. Done means the metadata is complete, the PR is opened, QA is notified, and approval is obtained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100