ESMCI / ESMCI/cime

Having trouble resubmitting a job that crashed

Open
#4,764 6 comments 0 reactions 0 assignees View on GitHub
Low Priority
Dominant language
Python
Stars
174
Forks
225
Avg merge
1d 16h
Merged PRs (30d)
14

Description

I ran a CESM job for 4 years with annual restarts and `RESUBMIT=4` to try to get 20 years of output. The first 4 years ran fine, and the job restarted from `0005-01-01-00000`, but the second job crashed in year 6 (well after writing the `0006-01-01-00000` restart files). I was able to run the short-term archiver by hand with `./case.submit --job case.st_archive`, but it failed when trying to resubmit `case.run` with

```
ERROR: File ${RUNDIR}/${CASE}.cpl.r.0005-01-01-00000.nc not present or does not match case ${CASE}
```

It looks like `case.submit` checks `DRV_RESTART_POINTER` and won't restart without it, but this variable was pointing to the `0005-01-01` restart (which the archiver was happy to archive, since `0006-01-01` was the latest). I'm not sure if this is simply a known issue, and you need to update `DRV_RESTART_POINTER` before re-submitting, or if there's a software fix for this

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.