DOI-USGS / DOI-USGS/pgmtl-data-release

metadata review from Alison

Open
#15 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
2
Forks
3
PR merge metrics
No merged PRs in 30d

Description

### [ ] Overall

- ~I’m confused, based on titles, about how lake metadata is distributed between 1 and 3~ updated the titles to make it clear 3_ is for PB configs and not metadata.
- ~2332 lakes = 1882 + 305 + 145. Where is this partitioning specified? I see type in lake_metadata.csv, but it only gives “source” or “target” and doesn’t distinguish between 305 and extended targets~ added a "ext_target" to the `type` column, which is what is used for the 1882. There are now 305 with "target", 145 with "source", and 1882 with "ext_target".
- Why include FIPS/NIST in the Places? _not sure...I think this is in meddle or the way the metadata is rendered by SB?_
- ~Process date of 7/6/2019 seems unlikely, unless we’re counting from nearer the start of the project on purpose~ updated

### [ ] 1 lake info

- In a lot of the Range Domain Max/Min values are NA _not sure I can update all of these in time for the pre-print deadline, but would like to_
- ~Obs_depth_mean_frac is a fraction but has a maximum of 2.12~ this is explained in the metadata.
- ~I’d say “even though…” instead of “even while appears in the data release”~ updated

### [ ] 2 temp obs

- ~Prmnn_I is quite cryptic~ added text that explains this is an NHD attribute

### [ ] 3 model configs

- ~Consider specifying in the title that this child item contains configs for the process-based models only~ updated
- ~Entity type definition for pb0: what is “expert-chosen default”? they weren’t just defaults for pb0, were they? They were the final parameters, right?~ updated to "lake-specific parameterization"

### [ ] 4 model inputs

- ~Title of item (“meteorological inputs and ice flags”) does not suggest that metamodel metafeatures will be provided in this item, but they are.~ changed to just "Model inputs"
- Range domain Mins/Maxes are NA for several variables _would like to update this too, but not much time before pre-print_
- ~Prmnn_I is still cryptic~ updated same as other
- Diff_lathrop_strat: does 1 mean source=TRUE, target=FALSE? Or the other way around? Looking for clear statement of conversion from binary to integer
- Why wouldn’t mtl_input_metafeatures.csv contain source-target info for all source-target pairs (including those for the extra 1882)? _saving this as a question for Jared_

### [ ] 5 model prediction data

- ~Can you title this “model predictions” instead of “model prediction data”? I think calling it data is misleading, even though this is a “data” release and all~ updated
- ~Abstract seems to be for a different project – Minnesota and Wisconsin only, DL models as well as PB and PGDL and PB0.~ fixed
- ~Native Data Set Environment: nice that we’ve at least nodded to the UM supercomputing system in this metadata file, but it mentions DL predictions and doesn’t mention MTL metamodel predictions.~ Added python env
- ~I think here you could have one entity type for all of the model predictions and then template the model. But it’s OK as-is.~ keeping
- ~Pbmtl_predictions_09 is missing~ known issue, since none of the 305 targets are in group 09.

### [ ] 6 model evaluation
- Could all_MTL_RMSE_predictions.csv also report predictions for the additional 1882 lakes? _saving as Jared question too_
- I’d suggest only reporting the top 9 (predicted) sources for each extended target, and populating the predicted and actual RMSEs and predicted rank for PGDL-MTL and PGDL-MTL9 sources, and using NAs for actual/pred_pb_mtl_rmse/rank and actual_pgdl_mtl_rank _saving this as a question for later due to limited time_
- ~The pgmtl*_evaluation.csv files report targets and RMSEs for all 2188, so that’s good.~ all good
- ~It also appears that preditions matched to obs for PGDL-MTL[9] include the extended targets, rigth?~ yes
- ~It also appears that the raw predictions for PGDL-MTL[9] include extended targets?~ yes
- ~Entity Type Definition – could note that the file contains not just MTL (metamodel?) predictions but also actual results for performance of source models applied to targets~ updated this text
- ~Metadata file doesn’t yet describe the xx_matched_to_observations.zip files or xx_evaluation.csv files or sparse_PGDL_vs_PGDL-MTL_rmse.csv file~ updated now
- ~In sparse_PGDL_vs_PGDL-MTL_rmse.csv, it’s weird that RMSE cells for which no model was possible are filled with 0 instead of NA~ filtered out these rows instead of using NA (or 0). Thanks for pointing that out

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.