ESCOMP / ESCOMP/SimpleLand

Inconsistent problem running longer tests with SOM on cheyenne with gnu compiler segfault

Open
#20 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Fortran
Stars
15
Forks
9
PR merge metrics
No merged PRs in 30d

Description

I'm getting an inconsistent problem with SMS_Lm3_D.f19_g16.H_MML_2000_CAM5_BGC.cheyenne_gnu.clm-global_uniform_g16_SOM. It has segfaulted after running for a short period of time, but at an inconsistent time. The first time was 9-steps and the second was after 2105 timesteps.

I also ran debug and short tests without debug for both intel and gnu that all worked. Including a DEBUG 3-month test.

Here's the error from cesm.log

```
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x2b115b7a494f in ???
#1 0x2b116cc1781d in ???
#2 0x2b115c027af3 in ???
#3 0x2b115ae31bcc in ???
#4 0x2b115ae7078e in ???
#5 0x2b115abd4100 in ???
#6 0xcbe0a9 in ???
#7 0xd4d312 in ???
#8 0xd42762 in ???
#9 0xd31728 in ???
#10 0xca0f76 in ???
#11 0xca18ce in ???
#12 0xc2bdff in ???
#13 0x420f2b in ???
#14 0x40a7aa in ???
#15 0x42014b in ???
#16 0x2b115b7906e4 in ???
#17 0x407268 in ???
at ../sysdeps/x86_64/start.S:118
#18 0xffffffffffffffff in ???
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 101 with PID 37244 on node r13i7n32 exited on signal 11 (Segmentation fault).
```

This is running with d1ed49c24fd6b9e0cc55cb60465ca037cc9838ec which I haven't pushed yet, but will. But, I also don't think it's necessarily anything in the new code which is just removing unused code.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.