DOI-USGS / DOI-USGS/streamMetabolizer

How To Test The Goodness of The Model

Open
#389 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Stan
Stars
47
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Hello everyone,
I created a Bayesian model in StreamMetabolizer and this is how my input data looks like:
![Depth_Temp_PAR](https://user-images.githubusercontent.com/36547773/81456633-adf6be80-9158-11ea-90ec-8e67301a7bf7.png)

These are the model specifications I used:

Model specifications:
model_name b_Kn_oipi_tr_plrckm.stan
engine stan
split_dates FALSE
keep_mcmcs TRUE
keep_mcmc_data TRUE
day_start 4
day_end 28
day_tests full_day, even_timesteps, complete_data, ...
required_timestep NA
GPP_daily_mu 3
GPP_daily_lower 0
GPP_daily_sigma 2
ER_daily_mu -7.1
ER_daily_upper 0
ER_daily_sigma 7.1
K600_daily_meanlog_meanlog 2.484906649788
K600_daily_meanlog_sdlog 1.32
K600_daily_sdlog_sigma 0.05
err_obs_iid_sigma_scale 0.03
err_proc_iid_sigma_scale 5
params_in GPP_daily_mu, GPP_daily_lower, GPP_daily_...
params_out GPP, ER, DO_R2, GPP_daily, ER_daily, K600...
n_chains 4
n_cores 1
burnin_steps 100
saved_steps 200
thin_steps 1
verbose FALSE

This is how the DO predictions look like:
![DO](https://user-images.githubusercontent.com/36547773/81456751-37a68c00-9159-11ea-879f-b2f80b48e147.png)

I am not sure how to test the goodness of the model? I need to improve the predictions and see when the model results are acceptable?
The tutorials you have does not have an explanation (might missed it) of how to test the model performance.

What do I need to do next? how to further optimize the model and improve the results.

Please let me know if there is another area where I should post such questions. I am new hear.
Thank you very much.

### Session information
> sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] rstan_2.19.3 StanHeaders_2.21.0-1
[3] unitted_0.2.9 ggplot2_3.3.0
[5] tidyr_1.0.2 dplyr_0.8.5
[7] streamMetabolizer_0.11.4

loaded via a namespace (and not attached):
[1] deSolve_1.28 tidyselect_1.0.0 purrr_0.3.4
[4] colorspace_1.4-1 vctrs_0.2.4 generics_0.0.2
[7] stats4_3.6.0 loo_2.2.0 utf8_1.1.4
[10] rlang_0.4.6 pkgbuild_1.0.7 pillar_1.4.3
[13] glue_1.4.0 withr_2.2.0 LakeMetabolizer_1.5.0
[16] readxl_1.3.1 matrixStats_0.56.0 lifecycle_0.2.0
[19] plyr_1.8.6 munsell_0.5.0 gtable_0.3.0
[22] cellranger_1.1.0 codetools_0.2-16 labeling_0.3
[25] inline_0.3.15 callr_3.4.3 ps_1.3.2
[28] parallel_3.6.0 fansi_0.4.1 Rcpp_1.0.4.6
[31] scales_1.1.0 farver_2.0.3 gridExtra_2.3
[34] rLakeAnalyzer_1.11.4.1 digest_0.6.25 processx_3.4.2
[37] grid_3.6.0 cli_2.0.2 tools_3.6.0
[40] magrittr_1.5 lazyeval_0.2.2 tibble_3.0.1
[43] crayon_1.3.4 pkgconfig_2.0.3 ellipsis_0.3.0
[46] prettyunits_1.1.1 lubridate_1.7.8 assertthat_0.2.1
[49] rstudioapi_0.11 R6_2.4.1 compiler_3.6.0

Contributor guide

Open the contributing guide

Research direction

No source file or test is named. Start with the existing tutorials and the supplied model specifications, then review the DO prediction output and session information. Done would be a clear, project-supported explanation of how to assess model performance and decide what improvement steps are appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.