CROCODILE-CESM / CROCODILE-CESM/workshop25-hack-projects
CESM Hybrid multi-instance run
- Dominant language
- No language data
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## CESM / MOM_interface — Hybrid Multi-Instance Restart Precheck Fix
**Summary**
CESM can be initialized in one of three ways: startup, hybrid and branch.
https://ncar.github.io/CESM-Tutorial/notebooks/modifications/xml/modify_run_type/hybrid_branch_restart.html
Hybrid is a common way to use and existing ensemble of CESM in a data assimilation experiment. However, currently the setup for hybrid for the MOM6 interface for CESM is incorrect.
This project aims to correct the logic in the MOM_interface’s `cime_config/buildnml` module that verifies the presence of restart files so that it correctly handles *multi-instance* cases.
Currently, the precheck only looks for a single-instance restart file (e.g.,
`${CASE}.mom6.r.${DATE}.nc`), and fails when instance-specific filenames (e.g.,
`${CASE}.mom6_${INST}.r.${DATE}.nc` or `${CASE}.mom6.r.${DATE}._${INST}.nc`) should instead be validated.
See [Issue #275](https://github.com/ESCOMP/MOM_interface/issues/275).
---
### Motivation
- The current bug prevents proper checking of restart files in multi-instance (ensemble) runs, leading to errors when users setup the case
- Hybrid or ensemble CESM / MOM workflows often rely on multiple instances of the ocean model (MOM6), so this precheck is a critical component for correct initialization.
---
### Goals
* Analyze the existing `buildnml` precheck logic (around lines 403–405).
*. Add or restructure conditional logic to detect restart files in multi-instance naming conventions (old and new formats), e.g.:
- `${CASE}.mom6.r.${DATE}._${INST}.nc`
- `${CASE}.mom6_${INST}.r.${DATE}.nc`
For restarts, thickness files, and any other instance specific files.
* Implement regression or unit tests to verify correct behavior for both single- and multi-instance runs.
---
### Approach / Methodology
- **Inspect and refactor:** Review the restart-check logic in `buildnml` and identify where it assumes single-instance behavior.
- **Add multi-instance support:** When `NINST_OCN > 1` (or similar), loop over instance indices and check for multiple naming patterns.
- **Provide error handling:** Report clearly if instance(s) are missing restart files.
- **Testing and validation:** Run both single- and multi-instance cases to confirm correct file detection and failure handling.
---
### Skills Needed or To be Gained with this Project
- Familiarity with **CESM / CIME / MOM_interface** build and run systems.
- Proficiency in **Python**, **shell scripting**, and **Git**.
- Understanding of **multi-instance / ensemble modeling** concepts.
- Ability to interpret CESM restart and configuration files.
---
### Possible Challenges & Open Questions
- Handling both single and new restart naming conventions consistently.
- Interactions with `continue_run`.
- Multi instance branch mode may also have the same bug.
---
### References
- [Issue #275 — “precheck test of restart existence doesn’t handle multi-instance”](https://github.com/ESCOMP/MOM_interface/issues/275)
- [MOM_interface source code (buildnml)](https://github.com/ESCOMP/MOM_interface/)
- [CESM / CIME User Guide](https://esmci.github.io/cime/versions/master/html/users_guide/)
- CESM and MOM6 multi-instance documentation
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.