E3SM-Project / E3SM-Project/scream
ne120 F2010-SCREAMv1 negative dp3d near surface after 5months
- Dominant language
- No language data
- Stars
- 79
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
ne120 F2010-SCREAMv1 has negative pressure thickness at the bottom 2 layers, causing the simulation to crash on month 2010-06-02. I'm running master + 3 pending PRs that make changes to nudging process (not used), online coarsening, time interpolation, fill value handling in output files. Per discussion with @AaronDonahue, we don't believe any of the changes from master would've altered the actual state of the simulation, since they are all related to the output files. This branch is pushed as `elynnw/combine-required-PRs` and there's a corresponding draft PR.
Just to add, I did a similar run back in May and was able to run 1+ year w/o errors with the same setup, just with different output files.
Here's the error log:
```
label: DIRK Newton loop np1
time-level 2
lat -5.838607460181858e-01 lon 5.075294102294816e+00
ie 16 igll 3 jgll 2 lev 0: bad dphi
level dphi dp3d vtheta_dp
0 nan 8.116123105946262e+00 1.050345307305943e+04
1 nan 8.120904420979381e+00 1.010679274528319e+04
2 nan 8.125528693199154e+00 9.500985545542608e+03
3 nan 9.796196222305301e+00 1.083750375219071e+04
4 nan 1.208848953247071e+01 1.254710764804474e+04
5 nan 1.438079476356506e+01 1.404252762954407e+04
6 nan 1.500243544578553e+01 1.387472445551333e+04
7 nan 1.646275520324706e+01 1.448872978559699e+04
8 nan 1.965382099151612e+01 1.651990363584305e+04
9 nan 2.284488677978514e+01 1.816815723837834e+04
10 nan 2.457562685012817e+01 1.854742764227186e+04
11 nan 2.525157928466798e+01 1.805810401730368e+04
12 nan 2.887706756591795e+01 1.965482097178846e+04
13 nan 3.250255584716800e+01 2.123443339245032e+04
14 nan 3.545210361480711e+01 2.220192132661842e+04
15 nan 3.545210361480715e+01 2.140784215547827e+04
16 nan 3.820617198944093e+01 2.249280056707786e+04
17 nan 4.121522903442381e+01 2.353517166588493e+04
18 nan 4.422430992126469e+01 2.464876337741630e+04
19 nan 4.447932243347164e+01 2.427589670884702e+04
20 nan 4.589781761169436e+01 2.449876552440042e+04
21 nan 4.810023307800287e+01 2.502406931000192e+04
22 nan 5.030260086059572e+01 2.519572475651374e+04
23 nan 5.102406943804709e+01 2.465705270128446e+04
24 nan 5.097065707692524e+01 2.409917288434942e+04
25 nan 5.098199844360361e+01 2.380614567922096e+04
26 nan 5.090579986572266e+01 2.364709858023474e+04
27 nan 5.085802078247069e+01 2.362062186156419e+04
28 nan 5.054674148559577e+01 2.354052326236139e+04
29 nan 4.748516082763669e+01 2.202332157197097e+04
30 nan 4.442367553710927e+01 2.014586222085208e+04
31 nan 4.167346954345712e+01 1.824648655417920e+04
32 nan 4.167356491088879e+01 1.761311463617185e+04
33 nan 4.370908737182603e+01 1.793200040218905e+04
34 nan 4.614294150815238e+01 1.846309526559711e+04
35 nan 4.833503459143489e+01 1.891206335880019e+04
36 nan 4.858948937119167e+01 1.863626073360711e+04
37 nan 5.018349959901707e+01 1.893286629421374e+04
38 nan 5.305277394919178e+01 1.974997213896985e+04
39 nan 5.587793524227193e+01 2.055863030876184e+04
40 nan 5.713331034782026e+01 2.075226841404954e+04
41 nan 5.846080096017586e+01 2.088010420496588e+04
42 nan 6.180956293580269e+01 2.171199852054599e+04
43 nan 6.500889762059212e+01 2.262329378816136e+04
44 nan 6.764710466178465e+01 2.340217344177961e+04
45 nan 6.792201995849609e+01 2.337805061051268e+04
46 nan 7.191314697265653e+01 2.461901631896333e+04
47 nan 7.590389251708949e+01 2.584555611478603e+04
48 nan 7.982730865478516e+01 2.707485126462504e+04
49 nan 7.982711791992183e+01 2.701395864204060e+04
50 nan 8.290150707358771e+01 2.802923384963514e+04
51 nan 8.721302903624898e+01 2.947908103469017e+04
52 nan 9.175375843242695e+01 3.101090009952010e+04
53 nan 9.292092417475349e+01 3.138387778600415e+04
54 nan 9.557546789074796e+01 3.223650094159671e+04
55 nan 1.011441931928490e+02 3.405112027269451e+04
56 nan 1.067034577490085e+02 3.585297501157598e+04
57 nan 1.096032438411808e+02 3.676446850702176e+04
58 nan 1.108841732287328e+02 3.715005551294310e+04
59 nan 1.172518524932856e+02 3.925896908479661e+04
60 nan 1.235568295746217e+02 4.136483538836808e+04
61 nan 1.284953379460902e+02 4.303062607833301e+04
62 nan 1.282880193895170e+02 4.298994685260793e+04
63 nan 1.351108667525314e+02 4.532078468551709e+04
64 nan 1.424061179319005e+02 4.782901743480825e+04
65 nan 1.496605429708601e+02 5.034382612302559e+04
66 nan 1.499173144772197e+02 5.051165661710460e+04
67 nan 1.555098141349338e+02 5.247659017306905e+04
68 nan 1.641039290572248e+02 5.545609025102726e+04
69 nan 1.726846860624530e+02 5.842797495208837e+04
70 nan 1.754416044563231e+02 5.942004507338264e+04
71 nan 1.792793286348983e+02 6.077077143075327e+04
72 nan 1.894583267838008e+02 6.427907254184449e+04
73 nan 1.996398804617076e+02 6.779166468353639e+04
74 nan 2.058098771206749e+02 6.993039393280397e+04
75 nan 2.071969318209151e+02 7.044843027676581e+04
76 nan 2.194025806896300e+02 7.466028755163917e+04
77 nan 2.318018078411348e+02 7.895001617360850e+04
78 nan 2.428991339039192e+02 8.271529348972446e+04
79 nan 2.434971430137506e+02 8.279007892852287e+04
80 nan 2.444771835044339e+02 8.286407659291617e+04
81 nan 2.455764381158557e+02 8.290077539944055e+04
82 nan 2.468208736332913e+02 8.294323718373575e+04
83 nan 2.480018259911924e+02 8.291805883538902e+04
84 nan 2.450688559942502e+02 8.151242822783257e+04
85 nan 2.383209228515627e+02 7.883033592644926e+04
86 nan 2.314956665039065e+02 7.613481675173927e+04
87 nan 2.286041259765628e+02 7.475824063907583e+04
88 nan 2.249748229980463e+02 7.316129356043968e+04
89 nan 2.130767822265633e+02 6.892376073185453e+04
90 nan 2.011802673339843e+02 6.478236581724232e+04
91 nan 1.929130554199220e+02 6.187929235412407e+04
92 nan 1.923522949218747e+02 6.147479894643279e+04
93 nan 1.758407592773438e+02 5.602132508636494e+04
94 nan 1.593276977539056e+02 5.066189595741256e+04
95 nan 1.433746337890620e+02 4.553368007299292e+04
96 nan 1.433753967285158e+02 4.547803185219193e+04
97 nan 1.280265808105466e+02 4.055564901595342e+04
98 nan 1.079032897949216e+02 3.413923491209590e+04
99 nan 8.777923583984418e+01 2.774497737102141e+04
100 nan 8.300476074218803e+01 2.620549954605494e+04
101 nan 8.216400146484341e+01 2.590258063898866e+04
102 nan 8.045349121093849e+01 2.532303718469900e+04
103 nan 7.874221801757709e+01 2.473925867047321e+04
104 nan 7.787322998046979e+01 2.441671676599310e+04
105 nan 7.744979858398358e+01 2.423314407478811e+04
106 nan 7.552871704101646e+01 2.358148346684966e+04
107 nan 7.360763549804633e+01 2.293798921093508e+04
108 nan 7.211074829101562e+01 2.242922248385438e+04
109 nan 7.211074829101564e+01 2.238929768700521e+04
110 nan 7.009963989257767e+01 2.172928064975278e+04
111 nan 6.798019409179682e+01 2.103537487976322e+04
112 nan 6.586227416992192e+01 2.034918894995621e+04
113 nan 6.575393676757943e+01 2.029535759924825e+04
114 nan 6.419219970703136e+01 1.978401800698152e+04
115 nan 6.188888549804679e+01 1.904174635165801e+04
116 nan 5.958480834960889e+01 1.830636699613458e+04
117 nan 5.884246826171915e+01 1.805428445860181e+04
118 nan 5.783767700195187e+01 1.772628559876334e+04
119 nan 5.536727905273488e+01 1.694237604077612e+04
120 nan 5.289688110351478e+01 1.614288151962728e+04
121 nan 5.143051147461043e+01 1.556071046116983e+04
122 nan 5.107498168945219e+01 1.534433249847294e+04
123 nan 4.845504760742195e+01 1.452831552369905e+04
124 nan 4.583511352539109e+01 1.368393547057439e+04
125 nan 4.753417968750006e+01 1.408091560893521e+04
126 nan -5.475540203827660e+01 -7.870957607817262e+03
127 nan -3.335955044312981e+03 -6.523905985721044e+05
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.