Need CAM snapshot post-processing script
- Dominant language
- No language data
- Stars
- 9
- Forks
- 22
- Avg merge
- 8d 5h
- Merged PRs (30d)
- 4
Description
### What is the feature/what would you like to discuss?
Issue #230 requires holding onto many CAM snapshot files for an extended (possibly indefinite) period of time. This could potentially introduce a significant storage burden, and make it difficult to run these tests outside of the NCAR computing ecosystem.
To remedy this problem, the snapshots could be post-processed with a script to make them as small as possible while still providing all of the data needed for the physics scheme tests to run properly. This could include the following modifications:
1. Remove all timesteps except the last two (eventually this may be reduced to just one timestep).
2. Remove all variables that aren't explicitly required by the physics scheme(s) being tested. This should be doable by using the CCPP-framework scripts directly to determine which variables are needed from the host model, and thus the snapshot file.
Other possibilities may include file compression (although for these tests it may need to be lossless), and removing un-necessary grid cells or vertical levels. However, the first two listed items are likely the only two post-processing steps that will definitely be required, at least for this particular issue.
### Is there anyone in particular you want to be part of this conversation?
_No response_
### Will this change (regression test) answers?
No
### Will you be implementing this enhancement yourself?
Any Software Engineer can do this
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading issue #230 and the CCPP-framework scripts used to determine host-model variables for the physics scheme tests. Identify how CAM snapshot files are produced and which inputs the tests require. Done means a script retains the final two timesteps and only required variables, while preserving the data needed by the tests.
Written by the indexing model from the issue text.
Assessment
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100