shallow water blow up due to large wind speed or too much cooling?

[Honolulu_Wang]

Hi all,
I met a problem for model blowing-up in the form of outputting NaN in the log file with ROMS3.7. My model is used in temperate shallow-water estuarine systems in East US, the grid resolution is around 400m*400m, mean-depth is less than 9 meters. The model starts in Jan 1st-XXXX, and It blows up in Jan 24th-XXXX.
It is running ok in other years but blows up in this specific year, so I replace the atoms forcing, river discharge, boundary forcing input one by one for self-check then I found the in blowing up year wind is a little large, V direction wind is around 12m/s, but not that large, so I damping that, then the model running ok to pass that blow-up time point. No wet and dry in my model.
Could anyone give some suggestions why this occur? Is it also possible due to cooling, since the temperature is also below -6 in many very shallow region as shown in the plots[? But turning on the cooling-limit CPP flag also did not work. The only work method so far is to damp the wind speed around that blowing-up time points.

Thanks.

logfile:
(197,237,20) 0.000000E+00 8.596251E-03 6.457468E+00 1.407614E+00
133911 2003-01-24 05:57:45.00 3.985038E-02 5.215968E+01 5.219953E+01 8.753632E+09
(307,125,03) 6.757250E-03 8.270931E-03 2.094695E+00 1.408217E+00
133912 2003-01-24 05:58:00.00 3.987643E-02 5.215955E+01 5.219943E+01 8.753486E+09
(307,125,03) 6.748983E-03 8.280034E-03 2.098586E+00 1.408819E+00
133913 2003-01-24 05:58:15.00 3.990253E-02 5.215943E+01 5.219933E+01 8.753340E+09
(307,125,03) 6.740551E-03 8.289057E-03 2.102443E+00 1.409422E+00
133914 2003-01-24 05:58:30.00 3.992866E-02 5.215930E+01 5.219923E+01 8.753194E+09
(307,125,03) 6.731679E-03 8.298338E-03 2.106327E+00 1.410024E+00
133915 2003-01-24 05:58:45.00 3.995484E-02 5.215917E+01 5.219913E+01 8.753048E+09
(307,125,03) 6.722215E-03 8.308228E-03 2.110320E+00 1.410625E+00
133916 2003-01-24 05:59:00.00 3.998106E-02 5.215905E+01 5.219903E+01 8.752903E+09
(307,125,03) 6.712142E-03 8.318626E-03 2.114499E+00 1.411227E+00
133917 2003-01-24 05:59:15.00 4.000731E-02 5.215892E+01 5.219893E+01 8.752758E+09
(307,125,03) 6.701547E-03 8.329611E-03 2.118809E+00 1.411828E+00
133918 2003-01-24 05:59:30.00 4.003361E-02 NaN NaN 8.752613E+09
(307,125,03) 6.690628E-03 8.341048E-03 2.123252E+00 1.412428E+00
Found Error: 01 Line: 298 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 298 Source: ROMS/Drivers/nl_ocean.h

Blowing-up: Saving latest model state into RESTART file

WRT_RST - wrote re-start fields (Index=1,1) in record = 0000003

Activated C-preprocessing Options:

ANA_BPFLUX Analytical bottom passive tracers fluxes
ANA_BSFLUX Analytical kinematic bottom salinity flux
ANA_BTFLUX Analytical kinematic bottom temperature flux
ANA_SPFLUX Analytical surface passive tracer fluxes
ANA_SSFLUX Analytical kinematic surface salinity flux
ASSUMED_SHAPE Using assumed-shape arrays
AVERAGES Writing out time-averaged nonlinear model fields
!BOUNDARY_ALLGATHER Using mpi_allreduce in mp_boundary routine
BULK_FLUXES Surface bulk fluxes parameterization
!COLLECT_ALL... Using mpi_isend/mpi_recv in mp_collect routine
CURVGRID Orthogonal curvilinear grid
DJ_GRADPS Parabolic Splines density Jacobian (Shchepetkin, 2002)
DOUBLE_PRECISION Double precision arithmetic numerical kernel.
GLS_MIXING Generic Length-Scale turbulence closure
KANTHA_CLAYSON Kantha and Clayson stability function formulation
MASKING Land/Sea masking
MPI MPI distributed-memory configuration
NONLINEAR Nonlinear Model
NONLIN_EOS Nonlinear Equation of State for seawater
N2S2_HORAVG Horizontal smoothing of buoyancy and shear
POWER_LAW Power-law shape time-averaging barotropic filter
PROFILE Time profiling activated
K_GSCHEME Third-order upstream advection of TKE fields
REDUCE_ALLGATHER Using mpi_allgather in mp_reduce routine
!RST_SINGLE Double precision fields in restart NetCDF file
SALINITY Using salinity
SOLAR_SOURCE Solar Radiation Source Term
SOLVE3D Solving 3D Primitive Equations
T_PASSIVE Advecting and diffusing inert passive tracer
TS_MPDATA Recursive flux corrected MPDATA 3D advection of tracers
UV_ADV Advection of momentum
UV_COR Coriolis term
UV_U3HADVECTION Third-order upstream horizontal advection of 3D momentum
UV_C4VADVECTION Fourth-order centered vertical advection of momentum
UV_QDRAG Quadratic bottom stress
VAR_RHO_2D Variable density barotropic mode

[jcwarner]

difficult to assess why the model would blow up with that limited information.
if you want to run the simulation and have the blow up occur, look at the rst file, his file time steps right before the blow up, etc. that can help to diagnose what may have been going on. look at water levels, currents, etc. it may have gotten too shallow with the strong wind and you needed to activate wet/dry. not sure. hard to tell.
-j

[Honolulu_Wang]

Thanks, John.
Yes. I checked and plotted his and rst file, looking normal.
When I turned on the wet_and dry flag, the model just passes that blow-up points.
May I know the reasons behind that?
Thanks

[wilkin]

Upon blow-up a third record should have been written to the Restart file. So you should examine that closely.

The log is telling you the i,j,k point where maximum speed is occurring so look in that general area.

(307,125,03) 6.701547E-03 8.329611E-03 2.118809E+00 1.411828E+00
133918 2003-01-24 05:59:30.00 4.003361E-02 NaN NaN 8.752613E+09

If the issue goes away when you activate wet/dry, that suggests the sea level got too small in your original run.

Maybe make the bathymetry a bit more gently there, or deeper, or mask that area as land if it is not important to the aspects of circulation you care about.