COAWST problem when using 2way WRF-ROMS

[jivica]

Dear all,

I'm having problem with setting/using COAWST (rev 1169) for 2-way WRF and ROMS.
The problem is with WRF sending unrealistic forcing fields to ROMS which is crashing
at the 1st ROMS time step (can't save his as not ending 1st step). Possibly someone had
similar problem and could shed some light.
I've sent similar post to COAWST sourcerepo, but it is not so active as this group.

My feeling is that mapping btw WRF and ROMS is the problem (scrip/weights), as they are
not defined on the same grid (WRF is on 10km grid and ROMS 2km, so I have to use
#define MCT_INTERP_OC2AT). However, land masks btw grids are quite similar, but not identical.
WRF is using larger area grid, big enough to provide forcing for ROMS/SWAN and have convex hull defined.
I used similar approach as in JOE_TS/Diffgrids (which I tested and is working, to check compiler).

So far I've tried running 2way WRF and ROMS using bulk_flux (just to see if masking is the problem,
and let ROMS to compute flux) as well ATM2OCN_FLUXES (WRF computing and sending fluxes directly to ROMS).
Both cases are not working OK for me. When using bulk_flux I get something like:

Code: Select all

 ## ROMS grid  1 recv data from WRF grid  1
WRFtoROMS Min/Max  GSW     (Wm-2):    -4.120588E+03  1.025860E+05
WRFtoROMS Min/Max  GLW     (Wm-2):    -4.264686E+03  1.026416E+05
WRFtoROMS Min/Max  MSLP    (mb):      -4.332417E+01  1.025978E+03
WRFtoROMS Min/Max  RELH    (-):       -4.359052E+01  1.026255E+03
WRFtoROMS Min/Max  T2      (C):       -4.391962E+03  1.026091E+05
WRFtoROMS Min/Max  U10     (ms-1):    -4.415034E+03  1.025284E+05
WRFtoROMS Min/Max  V10     (ms-1):    -4.392584E+03  1.026202E+05
WRFtoROMS Min/Max  RAIN  (kgm-2s-1):  -4.438196E+06  1.051382E+08
WRFtoROMS Min/Max  EVAP  (kgm-2s-1):  -4.344074E+06  1.051968E+08

To test mapping options btw WRF/ROMS I've checked scrip_coawst and tried using different
options within scrip.f like instead 'conservative' 'bilinear', as not using corners but centers,
or constrain search in 'lonlat' (default is 'latitude'), or increase number of search_bins to 120
(default 90) but nothing helped. Have looked at the grids.f, create_masks.f, scrip.f if there
is something I could try more, but out of ideas.

Then just to test mapping btw models is working (WRF and SWAN) I created SWAN grid from my ROMS grid
and run 2-way WRF-SWAN which ended fine. So, SWAN and ROMS are at the same grid, Mapping btw WRF and
SWAN should look the same as btw WRF and ROMS for say U10/V10 if using bulk_flux? This was my test for
mapping btw atmo and ocean/wave grids. Exchange btw ROMS and SWAN does not use #define MCT_INTERP_OC2WV, as on the same grid.
I am working for southern hemisphere which shouldn't matter (it does when mapping global forcing
as lower left corner can be flipped, which is not the case here).

Then I tried WRF-ROMS-SWAN and it is not working, as expected, the same as WRF-ROMS only run.
This narrows the problem to WRF->ROMS only.

I've run WRF only, ROMS only, SWAN only, WRF-SWAN and they all are OK. For example when running
all 3 models together at the beginning I get:

SWAN->ROMS:

Code: Select all

 ** ROMS grid  1 recv data from SWAN grid  1
 == SWAN grid  1 sent wave data to ROMS grid  1
SWANtoROMS Min/Max DISBOT  (Wm-2):     0.000000E+00  0.000000E+00
SWANtoROMS Min/Max DISSURF (Wm-2):     0.000000E+00  0.000000E+00
SWANtoROMS Min/Max DISWCAP (Wm-2):     0.000000E+00  0.000000E+00
SWANtoROMS Min/Max HSIGN   (m):        0.000000E+00  3.863863E+00
SWANtoROMS Min/Max RTP     (s):        0.000000E+00  1.313264E+01
SWANtoROMS Min/Max TMBOT   (s):        0.000000E+00  2.128347E+01
SWANtoROMS Min/Max DIR     (deg):      6.498113E-04  6.283185E+00
SWANtoROMS Min/Max WLEN    (m):        1.000000E+00  1.827806E+02
SWANtoROMS Min/Max WLENP   (m):        1.000000E+00  2.692732E+02

Which is OK, then ROMS->SWAN:

Code: Select all

 ** ROMS grid  1 sent data to SWAN grid  1
 == SWAN grid  1 recv data from ROMS grid  1
ROMStoSWAN Min/Max DEPTH   (m):        1.905913E+00  5.500000E+03
ROMStoSWAN Min/Max WLEV    (m):        0.000000E+00  1.199123E+00
ROMStoSWAN Min/Max VELX    (ms-1):    -1.497411E+00  1.626195E+00
ROMStoSWAN Min/Max VELY    (ms-1):    -1.332385E+00  1.458454E+00
ROMStoSWAN Min/Max ZO      (m):        5.000000E-02  5.000000E-02

WRF->SWAN is OK (!!!!) :

Code: Select all

  ## WRF grid  1 sent data to SWAN grid  1
 == SWAN grid  1 recv data from WRF grid 1
WRFtoSWAN Min/Max  U10     (ms-1):    -6.140821E+00  1.550999E+01
WRFtoSWAN Min/Max  V10     (ms-1):    -1.504831E+01  1.992949E+01

The problem is when WRF sending info to the ROMS and ROMS to WRF:
ROMS->WRF:

Code: Select all

 ## ROMS grid  1  sent data to WRF grid  1
ROMStoWRF  Min/Max SST     (K):        0.000000E+00  2.984500E+02   

(should not send 0 K?!)

WRF->ROMS:

Code: Select all

 ## ROMS grid  1 recv data from WRF grid  1
WRFtoROMS Min/Max  GSW     (Wm-2):    -4.120588E+03  1.025860E+05
WRFtoROMS Min/Max  GLW     (Wm-2):    -4.264686E+03  1.026416E+05
WRFtoROMS Min/Max  MSLP    (mb):      -4.332417E+01  1.025978E+03
WRFtoROMS Min/Max  RELH    (-):       -4.359052E+01  1.026255E+03
WRFtoROMS Min/Max  T2      (C):       -4.391962E+03  1.026091E+05
WRFtoROMS Min/Max  U10     (ms-1):    -4.415034E+03  1.025284E+05
WRFtoROMS Min/Max  V10     (ms-1):    -4.392584E+03  1.026202E+05
WRFtoROMS Min/Max  RAIN  (kgm-2s-1):  -4.438196E+06  1.051382E+08
WRFtoROMS Min/Max  EVAP  (kgm-2s-1):  -4.344074E+06  1.051968E+08

If any of you have some ideas to share I would be grateful.

Cheers,
Ivica

[mathieu]

One possibility to consider is whether the WRF grid completely covers the ROMS grid.
A variant of that possibility is that the interpolation matrix is not well computed for some reason.

[jivica]

I don't think it is the cause, as I wrote in my initial email WRF is bigger and encompassing ROMS (and SWAN) grids;

My feeling is that mapping btw WRF and ROMS is the problem (scrip/weights), as they are
not defined on the same grid (WRF is on 10km grid and ROMS 2km, so I have to use
#define MCT_INTERP_OC2AT). However, land masks btw grids are quite similar, but not identical.
WRF is using larger area grid, big enough to provide forcing for ROMS/SWAN and have convex hull defined.

And if that is the problem then WRF <-> SWAN wouldn't work either...

What is strange to me is that SWAN is made from ROMS grid (so basically the same rho/psi points) and it is working OK with WRF in 2-way constellation.

Cheers,
Ivica

[Serega.Osipov]

I checked my output in which ROMS domain is completely covered by larger WRF domain and min SST is also zero.

Do you know which model is crashing? I would recommend increasing the debug_level to 100 or 1000 in the WRF namelist.input, that should provide additional information.

A straightforward test would be to run ROMS-WRF on the exactly the same grid. If that works, then it has to do with the scrip. Does you grid cross -180 longitude?

[jivica]

Well it is strange,

ROMS is crashing at the first step, or better to say between init and 1 step while as it didn't saved 1st his record. WRF is happily running till next coupling exchange even after ROMS died.
I've put WRF output every time step and fields looks OK.
So, I am confident that problem is on ROMS side when getting info from WRF via MCT.

What is even more odd is that I've plotted weights (scrip interpolation matrix) as John Warner showed me and they look fine.
source and destination btw models is as it should be.

So, I've done bunch of cases and: WRF with SWAN is working fine, ROMS with SWAN using WRF outputs is fine, WRF alone is fine, SWAN alone and forced with WRF output is fine, ROMS alone with WRF forcing is fine. The only problem is WRF - ROMS when they MCT communicate. Even differences btw weights for atm2ocn and atm2wav are quite similar, and similar for ocn2atm and wav2atm.
I'll make trap and see where it is having problems just to see what is wrong.

My grid is not crossing -180; it is 100E -> 116E and 22S -> 35S

Seems it doesn't like southern hemisphere ;(

If anybody is running Southern grid with 2way WRF-ROMS it would be ultimate test.

Cheers,
Ivica

[prakash22]

If you send or post you cpp header (.h), .in and output log file. than that will more helpful to understand your problem.

[jivica]

Just to update you, I've managed to trap and fix it .. not nice and full fix but working on it.

John's script to plot weights helped in a way to see that wav2atm and ocn2atm differ at the narrow strip along the coast i.e. landmask.
I just plotted diffs btw them and then you can see the region clearly. This is ~1 pixel along the coast. Then I managed to trap ROMS to save all vars of interest before crash and saw that WRF is sending to ROMS (at ROMS wet points(!)) MSLP with value 0.
Actually all vars at the strip were zeroed (probably with mask multiplication when via mct sending). That just killed ROMS.

I've tried to use conservative, bilinear, distweight but all is similar, that strip is there and causing problems.

What I did was, copied WRF wrfinput_d01 into tmp.nc and there extended sea over the land for 1-2 pixels along the coast (for the strip), simple matlab conv2 (see at the end) and then applied scrip_coawst over the mask in tmp.nc.
Using that way weights WRF is sending non-zero values along the coast, which at the end make ROMS physical. WRF is using original mask in wrfinput_d01.

Code: Select all

lm=nc_varget('geo_em.d01_mod.nc','LANDMASK');
k=ones(3,3)./9;new_mask=[];
new_mask=conv2(lm,k,'same');     
new_mask(find(new_mask<1))=0;
tmp(1,:,:)=new_mask;nc_varput('wrfinput_d01_mod','LANDMASK',tmp);

Cheers,
Ivica