IMPORTANT: Corrected shared-memory bug

[arango]

A shared-memory bug was corrected for variables ntstart(ng), ntfirst(ng), and ntend(ng). Because of nesting, these variables were declared as THREADPRIVATE. However, this introduced a parallel bug when calling get_state to read initial conditions or restart fields. The routine get_state is only executed by the master thread. As a consequence, the above variables are only initialized in the master thread and their values are unknown to other shared-memory threads. Similar problem occurs for the time(ng) variable.

The solution is to remove the THREADPRIVATE clause for such variables and initialize them by master thread as follows:

!$OMP MASTER
        ntstart(ng)=INT((time(ng)-dstart*day2sec)/dt(ng))+1
        ntend(ng)=ntimes(ng)
        ntfirst(ng)=ntstart(ng)
!$OMP END MASTER
!$OMP BARRIER

and introduce a new scalar variable io_time to update the value of time(ng) for the other shared-memory threads after initialization or restart:

#ifdef INI_FILE
!
!  Read in initial conditions from initial NetCDF file.
!
      DO ng=1,Ngrids
!$OMP MASTER
        CALL get_state (ng, iNLM, 1, INI(ng)%name,                      &
     &                  IniRec(ng), Tindex(ng))
!$OMP END MASTER
# ifdef DISTRIBUTE
        CALL mp_bcasti (ng, iNLM, exit_flag)
# endif
!$OMP BARRIER
        IF (exit_flag.ne.NoError) RETURN
        time(ng)=io_time                     ! needed for shared-memory
      END DO
#else
!
!  If restart, read in initial conditions restart NetCDF file.
!
      DO ng=1,Ngrids
        IF (nrrec(ng).ne.0) THEN
!$OMP MASTER
          CALL get_state (ng, 0, 1, INI(ng)%name,                      &
     &                    IniRec(ng), Tindex(ng))
!$OMP END MASTER
# ifdef DISTRIBUTE
          CALL mp_bcasti (ng, iNLM, exit_flag)
# endif
!$OMP BARRIER
          IF (exit_flag.ne.NoError) RETURN
          time(ng)=io_time                   ! needed for shared-memory
        END IF
      END DO
#endif

Many thanks to Mark Hadfield for ​bringing this to my attention. This was also ​reported by Mitsuhiro Kawase.