Whether different intel environments will affect the operation of ROMS

Bug reports, work arounds and fixes

Moderators: arango, robertson

Post Reply
Message
Author
xiaxx666
Posts: 1
Joined: Mon Dec 26, 2022 6:10 pm
Location: High Performance Computing Cloud Company

Whether different intel environments will affect the operation of ROMS

#1 Unread post by xiaxx666 »

Hello, predecessors. I encountered a strange problem about ROMS recently.

If I use the 2017 intel compiler and intelmpi to compile dependencies and ROMS, the ROMS job will be completed normally.
However, I used the 2018 Intel compiler and intelmpi to compile dependencies and ROMS. And using the same set of input files, the ROMS reported "NaN" and "blow up" errors.

As for the errors encountered in the 2018 environment, if 3600 cores are used for operation, ROMS blows up at 1-20 timesteps in DIAG. sometimes NaN printed, sometimes not.
If 240 cores are used for running, the time for the occurrence of the blows up will be greatly delayed, sometimes at 500 timesteps, and sometimes at at 100 timesteps.
I wonder whether to continue to reduce the number of cores to run successfully. And what factors are related to the range of cores available for ROMS?

I will attach the running log of intel2017 intel2018 roms、the build_roms.sh used for compiling Roms and ocean.in.
Looking forward to your reply. Thank you very much.
Attachments
roms 2.zip
(1.81 MiB) Downloaded 87 times

User avatar
wilkin
Posts: 875
Joined: Mon Apr 28, 2003 5:44 pm
Location: Rutgers University
Contact:

Re: Whether different intel environments will affect the operation of ROMS

#2 Unread post by wilkin »

I think you need to take some time to carefully quality control your initial conditions.

Quickly looking at one of your log files 2018/3600/..out2 I see initial condition values of:

Code: Select all

                - potential temperature
                   (Min = -7.18577153E+00 Max =  3.30190636E+01)
                - salinity
                   (Min = -4.23978987E+00 Max =  4.56975751E+01)
Those unphysical minimum values might be making trouble in the equation of state. You probably have some odd extrapolation of HYCOM values near your coastline where the ROMS and HYCOM land/sea masks are incompatible. I suggest cleaning this up before debugging any further.
John Wilkin: DMCS Rutgers University
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu

Post Reply