E3SM-Project / E3SM-Project/E3SM
SIGTERM error when running ultra-low resolution on Cori
- Dominant language
- Fortran
- Stars
- 440
- Forks
- 481
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 33
Description
I am trying to run the E3SM at an ultra-low resolution on cori but keep getting the following error:
> 631: forrtl: error (78): process killed (SIGTERM)
631: Image PC Routine Line Source
631: e3sm.exe 0000000004428CEE Unknown Unknown Unknown
631: e3sm.exe 0000000003C9A300 Unknown Unknown Unknown
631: e3sm.exe 0000000004388761 Unknown Unknown Unknown
631: e3sm.exe 0000000004382948 Unknown Unknown Unknown
631: e3sm.exe 0000000003E7FF3F Unknown Unknown Unknown
631: e3sm.exe 0000000003E8099A Unknown Unknown Unknown
631: e3sm.exe 0000000003E68CE6 Unknown Unknown Unknown
631: e3sm.exe 0000000003E3057E Unknown Unknown Unknown
631: e3sm.exe 0000000003E30807 Unknown Unknown Unknown
631: e3sm.exe 0000000003F22E8C Unknown Unknown Unknown
631: e3sm.exe 0000000003F24D0F Unknown Unknown Unknown
631: e3sm.exe 0000000003E1C66B Unknown Unknown Unknown
631: e3sm.exe 0000000003E1CEBA Unknown Unknown Unknown
631: e3sm.exe 0000000003E2E68A Unknown Unknown Unknown
631: e3sm.exe 00000000026E79CA shr_mpi_mod_mp_sh 676 shr_mpi_mod.F90
631: e3sm.exe 00000000025A33DA seq_infodata_mod_ 2522 seq_infodata_mod.F90
631: e3sm.exe 000000000042500B component_mod_mp_ 906 component_mod.F90
631: e3sm.exe 000000000040C0AA cime_comp_mod_mp_ 3629 cime_comp_mod.F90
631: e3sm.exe 00000000004041AD cime_comp_mod_mp_ 2762 cime_comp_mod.F90
631: e3sm.exe 000000000042406D MAIN__ 133 cime_driver.F90
631: e3sm.exe 0000000000401ADE Unknown Unknown Unknown
631: e3sm.exe 000000000453BE4F Unknown Unknown Unknown
631: e3sm.exe 00000000004019CA Unknown Unknown Unknown
srun: error: nid03374: tasks 202-267: Exited with exit code 1
srun: error: nid03378: tasks 470-535: Exited with exit code 1
srun: error: nid03382: tasks 738-803: Exited with exit code 1
srun: error: nid03375: tasks 269-334: Exited with exit code 1
srun: error: nid03381: tasks 671-736: Exited with exit code 1
srun: error: nid03376: tasks 336-401: Exited with exit code 1
srun: error: nid03372: tasks 84,95-96,98,114,117,121,128-133: Exited with exit code 1
srun: error: nid03373: tasks 135-200: Exited with exit code 1
srun: error: nid03371: tasks 29-30,43,59: Exited with exit code 1
srun: error: nid03380: tasks 604-669: Exited with exit code 1
I have edited the radlw.f90 for a project I am working on so I am unsure if this error is because of that or something else. Any help debugging this error would be much appreciated!
Contributor guide
Research direction
Start by reproducing the ultra-low-resolution run on Cori and reviewing the changes in radlw.f90. Then inspect the reported Fortran stack frames, including shr_mpi_mod.F90, seq_infodata_mod.F90, component_mod.F90, cime_comp_mod.F90, and cime_driver.F90. Done means identifying whether the SIGTERM comes from the edit or the run configuration and documenting a reproducible resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fortran
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100