DOI-USGS / DOI-USGS/pgmtl-data-release
metadata review from Alison
- Dominant language
- R
- Stars
- 2
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
### [ ] Overall
- ~I’m confused, based on titles, about how lake metadata is distributed between 1 and 3~ updated the titles to make it clear 3_ is for PB configs and not metadata.
- ~2332 lakes = 1882 + 305 + 145. Where is this partitioning specified? I see type in lake_metadata.csv, but it only gives “source” or “target” and doesn’t distinguish between 305 and extended targets~ added a "ext_target" to the `type` column, which is what is used for the 1882. There are now 305 with "target", 145 with "source", and 1882 with "ext_target".
- Why include FIPS/NIST in the Places? _not sure...I think this is in meddle or the way the metadata is rendered by SB?_
- ~Process date of 7/6/2019 seems unlikely, unless we’re counting from nearer the start of the project on purpose~ updated
### [ ] 1 lake info
- In a lot of the Range Domain Max/Min values are NA _not sure I can update all of these in time for the pre-print deadline, but would like to_
- ~Obs_depth_mean_frac is a fraction but has a maximum of 2.12~ this is explained in the metadata.
- ~I’d say “even though…” instead of “even while appears in the data release”~ updated
### [ ] 2 temp obs
- ~Prmnn_I is quite cryptic~ added text that explains this is an NHD attribute
### [ ] 3 model configs
- ~Consider specifying in the title that this child item contains configs for the process-based models only~ updated
- ~Entity type definition for pb0: what is “expert-chosen default”? they weren’t just defaults for pb0, were they? They were the final parameters, right?~ updated to "lake-specific parameterization"
### [ ] 4 model inputs
- ~Title of item (“meteorological inputs and ice flags”) does not suggest that metamodel metafeatures will be provided in this item, but they are.~ changed to just "Model inputs"
- Range domain Mins/Maxes are NA for several variables _would like to update this too, but not much time before pre-print_
- ~Prmnn_I is still cryptic~ updated same as other
- Diff_lathrop_strat: does 1 mean source=TRUE, target=FALSE? Or the other way around? Looking for clear statement of conversion from binary to integer
- Why wouldn’t mtl_input_metafeatures.csv contain source-target info for all source-target pairs (including those for the extra 1882)? _saving this as a question for Jared_
### [ ] 5 model prediction data
- ~Can you title this “model predictions” instead of “model prediction data”? I think calling it data is misleading, even though this is a “data” release and all~ updated
- ~Abstract seems to be for a different project – Minnesota and Wisconsin only, DL models as well as PB and PGDL and PB0.~ fixed
- ~Native Data Set Environment: nice that we’ve at least nodded to the UM supercomputing system in this metadata file, but it mentions DL predictions and doesn’t mention MTL metamodel predictions.~ Added python env
- ~I think here you could have one entity type for all of the model predictions and then template the model. But it’s OK as-is.~ keeping
- ~Pbmtl_predictions_09 is missing~ known issue, since none of the 305 targets are in group 09.
### [ ] 6 model evaluation
- Could all_MTL_RMSE_predictions.csv also report predictions for the additional 1882 lakes? _saving as Jared question too_
- I’d suggest only reporting the top 9 (predicted) sources for each extended target, and populating the predicted and actual RMSEs and predicted rank for PGDL-MTL and PGDL-MTL9 sources, and using NAs for actual/pred_pb_mtl_rmse/rank and actual_pgdl_mtl_rank _saving this as a question for later due to limited time_
- ~The pgmtl*_evaluation.csv files report targets and RMSEs for all 2188, so that’s good.~ all good
- ~It also appears that preditions matched to obs for PGDL-MTL[9] include the extended targets, rigth?~ yes
- ~It also appears that the raw predictions for PGDL-MTL[9] include extended targets?~ yes
- ~Entity Type Definition – could note that the file contains not just MTL (metamodel?) predictions but also actual results for performance of source models applied to targets~ updated this text
- ~Metadata file doesn’t yet describe the xx_matched_to_observations.zip files or xx_evaluation.csv files or sparse_PGDL_vs_PGDL-MTL_rmse.csv file~ updated now
- ~In sparse_PGDL_vs_PGDL-MTL_rmse.csv, it’s weird that RMSE cells for which no model was possible are filled with 0 instead of NA~ filtered out these rows instead of using NA (or 0). Thanks for pointing that out
Contributor guide
Assessment
This issue has not been assessed yet.