Organize non-default options into for-testing physics options
- Dominant language
- Fortran
- Stars
- 352
- Forks
- 361
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 7
Description
Thinking about how to test multi-parameterization parts of the code (i.e., where there are a number of different options), I'm coming to feel that we should try to cover most or all of the options via system tests – in particular, via ERP tests, which cover exact restart and changing processor count, because these can catch bugs that are hard or impossible to catch via unit tests. (Changing processor count can catch things like accidentally assigning to a whole array when you mean to assign to a single element, or having an index variable in one loop accidentally using its value from the last iteration from the previous loop because you forgot to set it in the current loop.) (We could have a long-ish ERP test combined with a short SMS_D test, or just a short-ish ERP_D test.)
Currently, we have a ton of testmods, most of which just change one thing. To make it feasible to test each option of each parameterization, I'd like to consider introducing new "physics options" that are used only for testing.
For example, we could have test_option1, test_option2, etc. This could be done with testmods, but it's probably better to do it in namelist_defaults: for a given variable that controls different options, we'd make sure that all possible values fall under some physics option that's tested – if not one of the "real" options, then one of these test options. Then we'd have system tests covering each of the physics options, including these test options.
So physics "test_option1" would choose a suite of options that are all the defaults for test_option1, etc. These would be arbitrarily mixed and matched; some care may be needed to ensure that we don't try to combine incompatible options.
For this to work, I think we'd need a default in namelist_defaults if there's no match – e.g., what happens for test_option3 for something that only has two options.
Contributor guide
Assessment
This issue has not been assessed yet.