Whether different intel environments will affect the operation of ROMS

[xiaxx666]

Hello, predecessors. I encountered a strange problem about ROMS recently.

If I use the 2017 intel compiler and intelmpi to compile dependencies and ROMS, the ROMS job will be completed normally.
However, I used the 2018 Intel compiler and intelmpi to compile dependencies and ROMS. And using the same set of input files, the ROMS reported "NaN" and "blow up" errors.

As for the errors encountered in the 2018 environment, if 3600 cores are used for operation, ROMS blows up at 1-20 timesteps in DIAG. sometimes NaN printed, sometimes not.
If 240 cores are used for running, the time for the occurrence of the blows up will be greatly delayed, sometimes at 500 timesteps, and sometimes at at 100 timesteps.
I wonder whether to continue to reduce the number of cores to run successfully. And what factors are related to the range of cores available for ROMS？

I will attach the running log of intel2017 intel2018 roms、the build_roms.sh used for compiling Roms and ocean.in.
Looking forward to your reply. Thank you very much.

[wilkin]

I think you need to take some time to carefully quality control your initial conditions.

Quickly looking at one of your log files 2018/3600/..out2 I see initial condition values of:

Code: Select all

                - potential temperature
                   (Min = -7.18577153E+00 Max =  3.30190636E+01)
                - salinity
                   (Min = -4.23978987E+00 Max =  4.56975751E+01)

Those unphysical minimum values might be making trouble in the equation of state. You probably have some odd extrapolation of HYCOM values near your coastline where the ROMS and HYCOM land/sea masks are incompatible. I suggest cleaning this up before debugging any further.