E3SM-Project / E3SM-Project/scream
mpassi picard convergence failed in F-case runs
- Dominant language
- No language data
- Stars
- 79
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
The failure is captured in scream F-case tests on mappy that are not run in debug mode: e.g.,
- ERP_Ln22.ne4pg2_oQU480.F2010-SCREAMv1.mappy_gnu9.atmlndactive-rtm_off
- ERS_Ln22.ne30pg2_EC30to60E2r2.F2010-SCREAMv1.mappy_gnu9.atmlndactive-rtm_off
- PEM_Ln90.ne30pg2_EC30to60E2r2.F2010-SCREAMv1.mappy_gnu9
-
The runs with DEBUG mode are ok (e..g, ERS_D_Ln22.ne4pg2_oQU480.F2010-SCREAMv1.mappy_gnu9.atmlndactive-rtm_off)
The problem appears to be limited to mappy_gnu9. Same tests run ok on cori-knl_intel, ascent_gnugpu (ERP and PEM tests can run, but there are separate issues with ERP and PEM tests)
The first few lines of the error messages are
```
ERROR: -------------------------------------
ERROR:
ERROR: picard convergence failed!
ERROR: ==========================
ERROR:
ERROR: Surface: Tsf0, Tsf
ERROR: 0 -1.5518241246345625E-002 -1.5518241246345625E-002
ERROR:
ERROR: Snow: zTsn0(k), zTsn(k), zqsn0(k), ks(k), Sswabs(k)
ERROR: 1 0.0000000000000000 0.0000000000000000 -110121000.00000000 5.0748471383885514E-315 1.4242374018559045E-004
ERROR: 2 0.0000000000000000 0.0000000000000000 -110121000.00000000 0.0000000000000000 2.8507717591509711E-004
ERROR: 3 0.0000000000000000 0.0000000000000000 -110121000.00000000 0.0000000000000000 2.8506068425131106E-004
ERROR: 4 0.0000000000000000 0.0000000000000000 -110121000.00000000 0.0000000000000000 2.8504419328220386E-004
ERROR: 5 0.0000000000000000 0.0000000000000000 -110121000.00000000 0.0000000000000000 2.8503306846455615E-004
ERROR:
ERROR: Ice: zTin0(k), zTin(k), zSin0(k), zSin(k), phi(k), zqin0(k), km(k), Iswabs(k), dSdt(k)
ERROR: 1 -1.5669014377443815 -1.1483111027943425 0.28669488070650007 0.28669488070650007 1.0187631769471800E-002 -305962105.53206354 2.2820442990063059 39.671971386944676 -0.0000000000000000
ERROR: 2 -1.8969937920754070 -1.8217913690068226 1.3635776715090122 1.3635776715090122 4.0260210251142084E-002 -297531082.94632715 2.2290413794323620 2.6795040427153185 -0.0000000000000000
ERROR: 3 -1.9170322212976172 -1.8998275638189550 2.2729906124362880 2.2729906124362880 6.6433272399213275E-002 -289670991.45763475 2.1829113573963865 1.8657709229851689 -0.0000000000000000
ERROR: 4 -1.9239646293470931 -1.9145641701978748 2.7976235394494950 2.7976235394494950 8.1482310802327998E-002 -285142888.34896457 2.1563874272108969 1.4803309560091980 -0.0000000000000000
ERROR: 5 -1.9300147751069721 -1.9228302713083494 3.0534043034900389 3.0534043034900389 8.8662855449247471E-002 -282988343.21798432 2.1437317172707013 1.1763423061030871 -0.0000000000000000
ERROR: 6 -1.9358092128448650 -1.9284727307531435 3.1617892630245010 3.1617892630245010 9.1544720521249276E-002 -282130884.04962820 2.1386524300812981 0.91333060486698547 -0.0000000000000000
ERROR: 7 -1.9396863755549438 -1.9119262337646825 3.1967991129058810 3.1967991129058810 9.2379755267141359E-002 -281887150.63035828 2.1371806813416634 5.9601763175954110 -0.0000000000000000
ERROR:
ERROR: Ice boundary: q(k)
ERROR: 0 0.0000000000000000
ERROR: 1 0.0000000000000000
ERROR: 2 0.0000000000000000
ERROR: 3 0.0000000000000000
ERROR: 4 0.0000000000000000
ERROR: 5 0.0000000000000000
ERROR: 6 0.0000000000000000
ERROR: 7 0.0000000000000000
ERROR:
ERROR: dt: 1800.0000000000000
ERROR: hilyr: 0.14285714285714285
ERROR: hslyr: 2.2395386215650115E-008
ERROR: Tbot: -1.9452042139289627
ERROR: fswint: 53.748709176081938
ERROR: fswsfc: 0.95745738885343168
ERROR: rhoa: 1.2007377975844544
ERROR: flw: 315.67065769398960
ERROR: potT: 279.08200226855087
ERROR: Qa: 3.7165813980616479E-003
ERROR: shcoef: 39.246328372928367
ERROR: lhcoef: 107565.90315871530
ERROR: qpond: 0.0000000000000000
ERROR: qocn: -7975134.9758704985
ERROR: Spond: 0.0000000000000000
ERROR: sss: 34.700000000000003
ERROR: w: 0.0000000000000000
ERROR: flwoutn: -315.56525722151355
ERROR: fsensn: 233.41834293222894
ERROR: flatn: -37.105662405730875
ERROR: fsurfn: 197.37553838782753
ERROR: fcondtop: 36.191168883109036
ERROR: fcondbot: 0.99569478849638959
ERROR: fadvheat: 0.0000000000000000
ERROR:
ERROR: -------------------------------------
ERROR: temperature_changes_salinity: Picard solver non-convergence (no snow)
ERROR: column_vertical_thermodynamics: ice: Vertical thermo error: picard_solver: Picard solver non-convergence
ERROR: iCell: 210611
ERROR: config_dt: 1800.00000000000
ERROR: nCategories: 5
ERROR: nIceLayers: 7
ERROR: nSnowLayers: 5
ERROR: nAerosols: 0
ERROR: openWaterArea: 0.994771899963043
ERROR: iceAreaCategoryInitial: 0.00000000000000 0.522810003695731E-02 0.00000000000000 0.00000000000000 0.00000000000000
ERROR: iceVolumeCategoryInitial: 0.00000000000000 0.522810003695731E-02 0.00000000000000 0.00000000000000 0.00000000000000
ERROR: snowVolumeCategoryInitial: 0.00000000000000 0.585426597508568E-09 0.00000000000000 0.00000000000000 0.00000000000000
ERROR: iceAreaCell: 0.522810003695731E-02
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.