MP_EXCHANGE2D - communication buffer too small

Report or discuss software problems and other woes

Moderators: arango, robertson

Post Reply
Message
Author
turuncu
Posts: 128
Joined: Tue Feb 01, 2005 8:21 pm
Location: Istanbul Technical University (ITU)
Contact:

MP_EXCHANGE2D - communication buffer too small

#1 Unread post by turuncu »

Hi,

I try to run ROMS in coupling mode. I implement the ESMF WRF+ROMS coupling code and i try to do some test with that using 2 CPU. Each model component (WRF and ROMS) uses one CPU (or 1 PET) after running coupled code and few time steps, i got following error,

MP_EXCHANGE2D - communication buffer too small, EWsize = 957 501

What does it mean? Can i increase the buffer size? May be these problem triggered because of the running model in single CPU and MPI mode but i don't know. I am using this configuration for testing purpose so if i run it, it would be great.

Any suggestion will be helpful.

Thanks,

turuncu
Posts: 128
Joined: Tue Feb 01, 2005 8:21 pm
Location: Istanbul Technical University (ITU)
Contact:

Re: MP_EXCHANGE2D - communication buffer too small

#2 Unread post by turuncu »

I solve the problem. It is mainly related bug in Master/esmf_roms.F file. To solve it modify following lines in ROMS_GetImportData routine,

old one,
LBj=BOUNDS(ng)%LBi(MyRank)
UBj=BOUNDS(ng)%UBi(MyRank)

new one,
LBj=BOUNDS(ng)%LBj(MyRank)
UBj=BOUNDS(ng)%UBj(MyRank)

The i must be changed to j.

User avatar
arango
Site Admin
Posts: 1347
Joined: Wed Feb 26, 2003 4:41 pm
Location: DMCS, Rutgers University
Contact:

Re: MP_EXCHANGE2D - communication buffer too small

#3 Unread post by arango »

Yes, good catch. Thank you for reporting this bug and finding a solution. This is what we expect in good users. It is very important to become familiar with the code and have a great deal of curiosity.

jadranka_sepic
Posts: 9
Joined: Wed May 21, 2008 11:12 am
Location: Institute of Oceanography and Fisheries

Re: MP_EXCHANGE2D - communication buffer too small

#4 Unread post by jadranka_sepic »

I have a similar problem. I'm running the Adriatic Sea model with mpirun.

Model runs fine, until I try to add biology. Then I get the following error (warning):
MP_EXCHANGE2(3,4)D - communication buffer too small, EWsize = 800 648

My biology options are activated with:
#define BIO_FENNEL
#define ANA_BIOLOGY
#define ANA_SPFLUX
#define ANA_BPFLUX
#define CARBON
#define DENITRIFICATION
#define BIO_SEDIMENT

When I run ROMS with OpenMP and same biology options activated, model runs normally.

Any help is appreciated.

User avatar
arango
Site Admin
Posts: 1347
Joined: Wed Feb 26, 2003 4:41 pm
Location: DMCS, Rutgers University
Contact:

Re: MP_EXCHANGE2D - communication buffer too small

#5 Unread post by arango »

This is not related to earlier posts in this forum thread. Anyway, this is weird. If you added biology, only mp_exchange4d is used in just two routines:

Code: Select all

pre_step3d.F:

#  ifdef DISTRIBUTE
      CALL mp_exchange4d (ng, tile, iNLM, 1,                            &
     &                    LBi, UBi, LBj, UBj, 1, N(ng), 1, NT(ng),      &
     &                    NghostPoints,                                 &
     &                    EWperiodic(ng), NSperiodic(ng),               &
     &                    t(:,:,:,3,:))
#  endif

step3d_t.F:

# ifdef DISTRIBUTE
      CALL mp_exchange4d (ng, tile, iNLM, 1,                            &
     &                    LBi, UBi, LBj, UBj, 1, N(ng), 1, NT(ng),      &
     &                    NghostPoints,                                 &
     &                    EWperiodic(ng), NSperiodic(ng),               &
     &                    t(:,:,:,nnew,:))
# endif
The routine mp_exchange3d is not used in any ecosystem model.

The routine mp_exchange4d allocates the size of the buffers according with calling arguments:

Code: Select all

      SUBROUTINE mp_exchange4d (ng, tile, model, Nvar,                  &
     &                          LBi, UBi, LBj, UBj, LBk, UBk, LBt, UBt, &
     &                          Nghost, EW_periodic, NS_periodic,       &
     &                          A, B, C)
Did you altered the code to change the number of tracers, NT(ng)?

I cannot reproduce your problem. I have been using MPI with biological models for years.

You need to report the error message. The information provided is insufficient.

jadranka_sepic
Posts: 9
Joined: Wed May 21, 2008 11:12 am
Location: Institute of Oceanography and Fisheries

Re: MP_EXCHANGE2D - communication buffer too small

#6 Unread post by jadranka_sepic »

It doesn't actually stop with error message. I'd say it enters an endless loop.

From log file, it starts with:
INITIAL: Configuring and initializing forward nonlinear model ...
*******

MP_EXCHANGE2D - communication buffer too small, EWsize = 388 324

MP_EXCHANGE2D - communication buffer too small, EWsize = 392 324

This repeats itself for a while (let's say 100 times) then it manages to read vertical coordinates, then a lot of these warnings again... and so on, until it writes a first history file... and then more of these warnings (some of them with MP_EXCHANGE3D and some with MP_EXCHANGE_4D) - but without ever advancing to second model time step.

I've made no changes to any of biological files. For the time being, I've just used them as they are.

jadranka_sepic
Posts: 9
Joined: Wed May 21, 2008 11:12 am
Location: Institute of Oceanography and Fisheries

Re: MP_EXCHANGE2D - communication buffer too small

#7 Unread post by jadranka_sepic »

Update...

SO finally, I've checked bio_Fennel.in file, and noticed that I have periodic boundary conditions left at my closed (land) boundaries. I've changed that to closed and now ROMS runs fine.

Post Reply