Menter SST turbulence model outputs NaNs

Description

Working on wind turbine blades with @marcomangano, I tried to use other turbulence models than the Spalart-Allmaras model which I had been using until now. I know that only sa is differentiated, but I just want to understand the sensitivity of the results to the turbulence model, from an analysis perspective. So I went for Menter's k-\omega SST model (see my options below), and I got NaNs and a PETSc error as off iteration 1 (see the log).

I also tried k omega wilcox and k omega modified, using "turbresscale": [1.0e3,1.0e-6]. There is no NaN in that case, however I found it suspicious that the two turbulence residuals displayed for monitoring are exactly equal to 0.0. Looks like it is the same at iteration 0 with Menter's model.

Less important: I noted a typo in the doc: the option to specify the rescaling parameters for turbulence models is turbresscale instead of turbresscalar.

Looking forward to your insights on what could be causing this.

Options

{'adjointl2convergence': 1e-09,
 'adjointmaxiter': 500000,
 'adjointsubspacesize': 500,
 'adpc': True,
 'anklinresmax': 0.1,
 'ankmaxiter': 60,
 'anksecondordswitchtol': 0.01,
 'cfl': 1.5,
 'cflcoarse': 1.5,
 'equationtype': 'RANS',
 'gridfile': 'UAE_1blade_short_3pitch_coarse_5_L1_3D_128.cgns',
 'l2convergence': 1e-11,
 'l2convergencecoarse': 1e-05,
 'lowspeedpreconditioner': True,
 'mgcycle': 'sg',
 'monitorvariables': ['cpu', 'resrho', 'cd', 'resturb', 'yplus'],
 'ncycles': 30000,
 'ncyclescoarse': 250,
 'nkinnerpreconits': 2,
 'nkjacobianlag': 3,
 'nkouterpreconits': 3,
 'nksubspacesize': 100,
 'nkswitchtol': 1e-14,
 'nsubiterturb': 10,
 'restrictionrelaxation': 1.0,
 'smoother': 'dadi',
 'surfacevariables': ['cp', 'mach', 'yplus', 'sepsensor', 'p', 'temp'],
 'turbresscale': [1000.0, 1e-06],
 'turbulencemodel': 'menter sst',
 'useanksolver': True,
 'useblockettes': False,
 'usenksolver': True,
 'useqcr': True,
 'userotationsa': True,
 'volumevariables': ['resrho', 'resturb', 'vort', 'mach']}

Log

#
# Grid 1: Performing 30000 iterations, unless converged earlier. Minimum required iteration before NK switch:      5. Switch to NK at totalR of:   0.43E-07
#
#---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
#  Grid  | Iter | Iter |  Iter  |   CFL   | Step | Lin  |    Wall    |        Res rho         |       Res kturb        |       Res wturb        |        C_drag          |        totalRes        |         Y+_max         |
#  level |      | Tot  |  Type  |         |      | Res  | Clock (s)  |                        |                        |                        |                        |                        |                        |
#---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      1       0      0     None   0.00E+00  1.00   ----  0.23527E+00   0.4008312643096706E+03   0.0000000000000000E+00   0.0000000000000000E+00   0.6116832261115712E-01   0.4296967046625193E+07   0.4770154773966238E+01
 Bad Block:  -58.457080994455325                            NaN                       NaN                       NaN                       NaN  -136.43358162455107                            NaN                       NaN                       NaN                       NaN  -27.057667980828771                            NaN                       NaN                       NaN                       NaN  -21.493860824869575                            NaN                       NaN                       NaN                       NaN   0.0000000000000000                            NaN                       NaN                       NaN                       NaN
 irow:     1963028
 icol     1802688
 nn:           1
 ijk:           6           6           2
 ---------------------------------------------------------------------------
PETSc or MPI Error. Error Code  1. Detected on Proc 51
Error at line:   630 in file: ../adjoint/adjointUtils.F90
 ---------------------------------------------------------------------------
 Bad Block:  -27.121688365402417                            NaN                       NaN                       NaN                       NaN -0.32959771301234786                            NaN                       NaN                       NaN                       NaN   12.760238156574124                            NaN                       NaN                       NaN                       NaN   3.1770896941151587                            NaN                       NaN                       NaN                       NaN   0.0000000000000000                            NaN                       NaN                       NaN                       NaN
 irow:     1320948
 icol      849908
 nn:           1
 ijk:           6           6           1

...

Code versions

• ADflow v2.2.1
• Python 3.7.2

Reproduce the failure with the supplied ADflow v2.2.1 options and log, then inspect adjoint/adjointUtils.F90 around line 630 and compare the Menter SST, k omega Wilcox, and k omega modified runs. Use the reported grid indices and turbulence residuals to narrow the NaN source. Done means the failure is explained with a corrective direction, and the turbresscale spelling is verified against the documentation.