Usability: Add postprocessing instructions and tools
- Dominant language
- Fortran
- Stars
- 352
- Forks
- 361
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 7
Description
CTSM history variables can be confusing and processing the history files is nontrivial, meaning that there's a lot of room for mistakes. A significant volume of our support requests are questions like, "What variables should I save to get aboveground biomass?" and "How do I make maps with this PFT-level output?" We should support documentation and tools to help with such common questions.
I'm starting this issue as a parent for eventual children. As we decide to work on things listed here, we can convert them from check boxes to sub-issues.
## What?
### "Which outputs should I save to calculate X?"
Make sure to include information like what variables to save and at what subgrid level, if applicable. These would be mostly just documentation updates, but it would be good to also test somehow that whatever variable list we give at least doesn't contain bad names.
- [ ] Aboveground biomass
- [ ] Crop yield/production. See also #2494, which I'm closing as superseded by this issue.
- See also Adrianna's [spreadsheet for TRENDY](https://github.com/user-attachments/files/28158280/trendy_listofvariables_CLM_2025.csv)
### "How do I do Y with the history files?"
These would require both (a) development of tooling and/or testing and (b) additions to our docs.
- [ ] Mapping ungridded outputs: Regular grids. Already in my [ctsm_postprocessing](https://github.com/samsrabin/ctsm_postprocessing) repo, but we'd want to make sure it's fully tested, wrapped, and of course documented.
- [ ] Mapping ungridded outputs: Irregular grids
- [ ] Interpreting `*1d_ityp*` variable values
- [ ] Calculate crop yield, including C to biomass, harvest efficiency, and moisture. (Lots of work on this already in my [ctsm_postprocessing](https://github.com/samsrabin/ctsm_postprocessing) repo.) See also #2494, which I'm closing as superseded by this issue.
## Where?
The [NCAR/ctsm_python_gallery](https://github.com/NCAR/ctsm_python_gallery) repo had some notebooks and functions to illustrate and/or help with common tasks. However, it hasn't been touched in years, it has nothing in the way of testing, and it's seemingly run out of space. I would be in favor of deprecating it in favor of a new repo focused mainly around Python tools/modules. We could include some notebooks as illustrative examples, I guess, but I would prefer to instead have proper documentation. (Although apparently there are Sphinx extensions to embed Jupyter Notebooks...)
I've already done some work in this direction in my [ctsm_postprocessing](https://github.com/samsrabin/ctsm_postprocessing) repo. This contains the Python modules from the other repo and has added testing for some of them. It's currently used as an external in my crop CUPiD branch, so once that comes into main CUPiD and CUPiD comes into CTSM we'll get it for "free." In the meantime, I think we'd want to bring it in as a submodule to CTSM at `python/ctsm/ctsm_postprocessing/`.
The postprocessing code there would be available for anyone to import, copy into their own stuff, whatever. We would also add wrapper scripts to `tools/` as necessary for any command-line utilities we want to support.
## How?
The most basic version of these tools would just be functions that users can import into their existing scripts. For some, it might make sense to also have a command-line tool. However, that can add a lot of complexity—handling existing outputs, handling missing inputs, etc. For now, I'm going to lean towards no command-line tools, then we can add those as needed.
Contributor guide
Assessment
This issue has not been assessed yet.