Difference between revisions of "NUOPC Cap"

From WikiROMS
Jump to navigationJump to search
 
(25 intermediate revisions by 2 users not shown)
Line 3: Line 3:


==Overview==
==Overview==
This document describes the ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span>, which is a light weight software layer that is required when the ROMS ocean model is used in National Unified Operation Prediction Capability (NUOPC) coupled systems. Detailed information about ESMF/NUOPC and how it is implemented in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.
This document describes the ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span> module <span class="blue">Master/esmf_roms.F</span> used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (<span class="blue">Master/esmf_driver.h</span>), which '''creates''', '''register''', '''initialize''', '''run''', and '''finalize''' the coupling of each Earth System Model ('''ESM''') component ('''data''', '''atmosphere''', '''sea-ice''', '''wave''', and '''ocean'''). It also sets the coupler '''connectors''' and the '''RunSequence''' policy (<span class="blue">Master/esmf_esm.F</span>). The framework provides the '''NUOPC''' <span class="red">cap</span> modules for each coupled component. Detailed information about '''ESMF/NUOPC''' and how it is implemented in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.


A '''NUOPC''' <span class="red">cap</span> is a Fortran module that serves as the interface to a model when used in a '''NUOPC'''-based coupled system. The term <span class="red">cap</span> is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its '''initialize''', '''run''', and '''finalize''' phases and exposing the model data structures to the field exchange ('''import''' and '''export''' states) between coupled components.


A NUOPC <span class="red">cap</span> is a Fortran module that serves as the interface to a model when it's used in a NUOPC-based coupled system. The term <span class="red">cap</span> is used because it is a light weight software layer that sits on top of model code, making calls into it and exposing model data structures in a standard way.
 
[[Image:ROMS_Coupling.png|800px|center]]
 
<center>'''''Figure 1:''''' ROMS native ESMF/NUOPC coupling infrastructure</center>
 
==Implementation==
 
The <span class="blue">Master/mod_esmf_esm.F</span> module declares several derived-type structures to facilitate the management of all internal objects and variables in each '''ESM''' coupled component:
 
* '''ESM''' coupling time managing variables and '''ESMF''' objects. The <span class="red">ClockInfo</span> object is allocated as <span class="red">(0:Nmodels)</span>, where <span class="red">Nmodels</span> is the number of coupled '''ESM''' components. Here, index zero corresponds to the '''driver'''.<div class="code">      <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Clock</span><br /><br />        <span class="darkTurquoise">logical</span> :: <span class="red">Restarted</span><br /><br />        <span class="darkTurquoise">integer</span> (<span class="red">i8b</span>) :: <span class="red">AdvanceCount</span>              <span class="twilightBlue">! advance counter</span><br /><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Current_Time</span>                  <span class="twilightBlue">! seconds</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Reference</span>                <span class="twilightBlue">! seconds</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Restart</span>                  <span class="twilightBlue">! seconds</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Start</span>                    <span class="twilightBlue">! seconds</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Stop</span>                      <span class="twilightBlue">! seconds</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Step</span>                      <span class="twilightBlue">! seconds</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Name</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">CalendarString</span>        <span class="twilightBlue">! 360_day, gregorian</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_ReferenceString</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_RestartString</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StartString</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StopString</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Calendar</span>)      :: <span class="red">Calendar</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Clock</span>)          :: <span class="red">Clock</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Direction_flag</span>) :: <span class="red">Direction</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>)          :: <span class="red">CurrentTime</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>)          :: <span class="red">ReferenceTime</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>)          :: <span class="red">RestartTime</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>)          :: <span class="red">StartTime</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>)          :: <span class="red">StopTime</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_TimeInterval</span>)  :: <span class="red">TimeStep</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Clock</span><br /><br />'''. . .'''<br /><br />      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div>
 
* '''ESM''' coupled state sets. If appropriate, it includes the logic for connecting nested grids. The <span class="red">COUPLED</span> object is of size <span class="red">Nmodels</span>.<div class="code">      <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_CplSet</span><br /><br />        <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">LinkedGrid(:,:)</span>      <span class="twilightBlue">! connected grid</span><br /><br />        <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">DataCoupledSets(:,:)</span>  <span class="twilightBlue">! DATA linked sets</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">SetLabel(:)</span> <span class="twilightBlue">! set label</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExpLabel(:)</span> <span class="twilightBlue">! export label</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImpLabel(:)</span> ! <span class="twilightBlue">import label</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:,:)</span> <span class="twilightBlue">! export set</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_CplSet</span><br /><br />      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_CplSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">COUPLED(:)</span></div>
 
* '''DATA''' model field processing information. The '''DATA''' component is often necessary for applications with incongruent grids to provide complete field melding capabilities or methods for reading and interpolating stream data in space and time from archived NetCDF files.<div class="code">      <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Data</span><br /><br />        <span class="darkTurquoise">logical</span> :: <span class="red">Lcycle</span>                    <span class="twilightBlue">! cycling time coordinate</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">Lcoord</span>                    <span class="twilightBlue">! coordinates attribute</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">Lmask</span>                      <span class="twilightBlue">! land/sea mask</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">Lmulti</span>                    <span class="twilightBlue">! field across multi-files</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">LastRec</span>                    <span class="twilightBlue">! processed last record</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Icomp</span>                      <span class="twilightBlue">! target component index</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">ncid</span>                      <span class="twilightBlue">! NetCDF file ID</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Tid</span>                        <span class="twilightBlue">! time variable ID</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Vid</span>                        <span class="twilightBlue">! field variable ID</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Nvdim</span>                      <span class="twilightBlue">! number spatial dimensions</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Zlevel</span>                    <span class="twilightBlue">! level index to process</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Nrec</span>                      <span class="twilightBlue">! number of time records</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Trec</span>                      <span class="twilightBlue">! latest read time record</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span>                    <span class="twilightBlue">! rolling two-time indices</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">LandValue</span>                  <span class="twilightBlue">! Masking land value</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">SeaValue</span>                  <span class="twilightBlue">! Masking sea value</span><br /><br />        <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Vsize(:)</span>      <span class="twilightBlue">! dimensions size</span><br /><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">add_offset</span>                <span class="twilightBlue">! add_offset attribute</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">FillValue</span>                <span class="twilightBlue">! _FillValue attribute</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">scale_factor</span>              <span class="twilightBlue">! scale_factor attribute</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Clength</span>                  <span class="twilightBlue">! time cycling length</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMin</span>                    <span class="twilightBlue">! grid minimum longitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMax</span>                    <span class="twilightBlue">! grid maximum longitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMin</span>                    <span class="twilightBlue">! grid minimum latitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMax</span>                    <span class="twilightBlue">! grid maximum latitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tscale</span>                    <span class="twilightBlue">! time scale to day</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmono</span>                    <span class="twilightBlue">! monotonic time (days)</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmin</span>                      <span class="twilightBlue">! time minimum value</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmax</span>                      <span class="twilightBlue">! time maximum value</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tstr</span>                      <span class="twilightBlue">! lower time-snapshot</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tend</span>                      <span class="twilightBlue">! upper time-snapshot</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmin</span>                      <span class="twilightBlue">! variable minimum value</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmax</span>                      <span class="twilightBlue">! variable maximum value</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span>                <span class="twilightBlue">! interpolation time (days)</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Vtime(2)</span>                  <span class="twilightBlue">! latest two-time values</span><br />        <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Date(6,2)</span>                <span class="twilightBlue">! time-snapshots dates YYYY,MM,DD hh:mm:ss.ss</span><br /><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30), <span class="darkTurquoise">allocatable</span> :: <span class="red">Dname(:)</span>  <span class="twilightBlue">! variable dimensions names</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Vcoord(:)</span> <span class="twilightBlue">! variable coordinates names</span><br /><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=22 ) :: <span class="red">DateString(2)</span>  <span class="twilightBlue">! date-snapshots string</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30 ) :: <span class="red">SpecialAction</span>  <span class="twilightBlue">! special processing</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tname</span>          <span class="twilightBlue">! time variable name</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tunits</span>          <span class="twilightBlue">! time variable units</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vname</span>          <span class="twilightBlue">! variable name</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vunits</span>          <span class="twilightBlue">! variable units</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vdescriptor</span>    <span class="twilightBlue">! Variable descriptive name</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vlongname</span>      <span class="twilightBlue">! long_name attribute</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span>          <span class="twilightBlue">! NetCDF filename</span><br /><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span>    <span class="twilightBlue">! field longitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span>    <span class="twilightBlue">! field latitude</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span>    <span class="twilightBlue">! field land/sea mask</span><br /><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2d(:,:)</span>    <span class="twilightBlue">! time interpolated 2D data</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3d(:,:,:)</span>  <span class="twilightBlue">! time interpolated 3D data</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2dG(:,:,:)</span>  <span class="twilightBlue">! latest 2D data snapsnots</span><br />        <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3dG(:,:,:,:)</span><span class="twilightBlue">! latest 3D data snapsnots</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Field</span>)      :: <span class="red">field</span>      <span class="twilightBlue">! field object</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>)        :: <span class="red">grid</span>      <span class="twilightBlue">! field grid object</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span>    <span class="twilightBlue">! field RouteHandle</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Data</span></div>
 
* '''DATA''' model high-level structure. <div class="code">      <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_DataSet</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Nfields</span>                    <span class="twilightBlue">! number of fields</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Nfiles</span>                    <span class="twilightBlue">! number of input files</span><br /><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Ctarget(:)</span>  <span class="twilightBlue">! component target</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Field(:)</span>    <span class="twilightBlue">! field short-name</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Data</span>),  <span class="darkTurquoise">allocatable</span> :: <span class="red">Export(:)</span>  <span class="twilightBlue">! Export field structure</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">T_IO</span>),      <span class="darkTurquoise">allocatable</span> :: <span class="red">IFS(:)</span>      <span class="twilightBlue">! Input Files structure</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_DataSet</span></div>
 
* Import and export field metadata information between the source (<span class="twilightBlue">SRC</span>) and destination (<span class="twilightBlue">DST</span>) components. <div class="code">      <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Field</span><br /><br />        <span class="darkTurquoise">logical</span> :: <span class="red">connected</span>                  <span class="twilightBlue">! connected to coupler</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">debug_write</span>                <span class="twilightBlue">! write exchanged field</span><br />        <span class="darkTurquoise">logical</span> :: <span class="red">enable_integral_adj</span>        <span class="twilightBlue">! area integral adjusted</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">fid</span>                        <span class="twilightBlue">! internal field ID</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span>                      <span class="twilightBlue">! field grid mesh type</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">etype</span>                      <span class="twilightBlue">! field extrapolation flag</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">itype</span>                      <span class="twilightBlue">! field interpolation flag</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span>                    <span class="twilightBlue">! rolling two-time indices</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=20) :: <span class="red">Ctarget</span>        <span class="twilightBlue">! component destination</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">DateString(2)</span>  <span class="twilightBlue">! date-snapshots string</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">short_name</span>    <span class="twilightBlue">! short name</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">standard_name</span> <span class="twilightBlue">! standard name</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">long_name</span>    <span class="twilightBlue">! long name</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_gtype</span>    <span class="twilightBlue">! DST grid type</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_units</span>    <span class="twilightBlue">! DST units</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_gtype</span>    <span class="twilightBlue">! SRC grid type</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_units</span>    <span class="twilightBlue">! SRC units</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_vname</span>      <span class="twilightBlue">! DATA Vname</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_tname</span>      <span class="twilightBlue">! DATA Tname</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">RegridMethod</span>  <span class="twilightBlue">! regrid method</span><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExtrapMethod</span>  <span class="twilightBlue">! extrapolate</span><br /><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">scale_factor</span>            <span class="twilightBlue">! field scale factor</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">add_offset</span>              <span class="twilightBlue">! field add offset value</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmin</span>                    <span class="twilightBlue">! DATA time minimum value</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmax</span>                    <span class="twilightBlue">! DATA time maximum value</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tstr</span>                    <span class="twilightBlue">! DATA lower time-snapshot</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tend</span>                    <span class="twilightBlue">! DATA upper time-snapshot</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span>                <span class="twilightBlue">! interpolation time (day)</span><br />        <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Vtime(2)</span>                <span class="twilightBlue">! latest two-time values</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span>    ! <span class="twilightBlue">field RouteHandle</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Field</span></div>
 
* Import and export fields mesh data. <div class="code">      <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Mesh</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">gid</span>                          <span class="twilightBlue">! grid ID</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span>                        <span class="twilightBlue">! grid mesh type</span><br /><br />        <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span>  <span class="twilightBlue">! grid land/sea mask</span><br /><br />        <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span>      <span class="twilightBlue">! grid longitude</span><br />        <span class="red">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span>      <span class="twilightBlue">! grid latitude</span><br />        <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">area(:,:)</span>      <span class="twilightBlue">! grid area</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Mesh</span></div>
 
* Melding coefficients used to combine fields from '''DATA''' and '''ESM''' components. The weight factors are read from the input NetCDF specified in the <span class="blue">WeightsFile(atmos)</span> keyword. The user has  full control of how the merging is done. It is recommended to provide a gradual transition between the two components. <br /><br />  Recall that the '''DATA''' component supplies needed data to a particular '''ESM''' component.  For example, it may export data to the atmosphere model at locations not covered by the other '''ESM''' components because of smaller grid coverage.  If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data: <div class="code">  <span class="red">SST_atm(:,:)</span> = <span class="red">Cesm(:,:)</span> * <span class="red">SST_esm(:,:)</span> + <span class="red">Cdat(:,:)</span> * <span class="red">SST_dat(:,:)</span></div>  where<span class="red"> Cesm(:,:)</span> + <span class="red">Cdat(:,:)</span> = <span class="red">1</span>.<br /><div class="code">      <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">NestedGrid</span>              <span class="twilightBlue">! grid needing merged field</span><br /><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameDATA</span>    <span class="twilightBlue">! DATA weights variable name</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameESM</span>      <span class="twilightBlue">! ESM  weights variable name</span><br />        <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span>        <span class="twilightBlue">! Weights NetCDF filename</span><br /><br />        <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="forestGreen">allocatable</span> :: <span class="red">Cdat(:,:)</span> <span class="twilightBlue">! coefficients for DATA</span><br />        <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cesm(:,:)</span> <span class="twilightBlue">! coefficients for ESM</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br />      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Meld</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">WEIGHTS(:)</span><br /><br />      <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightDAT</span> = <span class="red">0.0_dp</span>      <span class="twilightBlue">! DATA component weight</span><br />      <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightESM</span> = <span class="red">1.0_dp</span>      <span class="twilightBlue">! ESM  component weight</span></div>
 
* Coupled models high-level object, [<span class="red">Nmodels=5</span>]. <div class="code">      <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Model</span><br /><br />        <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span>                      <span class="twilightBlue">! active for coupling</span><br /><br />        <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">LandValue</span>                <span class="twilightBlue">! land mask value</span><br />        <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">SeaValue</span>                <span class="twilightBlue">! sea  mask value</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">Ngrids</span>                        <span class="twilightBlue">! number nested grids</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">ExportCalls</span>                    <span class="twilightBlue">! export CALL counter</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">ImportCalls</span>                    <span class="twilightBlue">! import CALL counter</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span>                          <span class="twilightBlue">! number model PETs</span><br />        <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span>        <span class="twilightBlue">! component PETs list</span><br /><br />        <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">TimeFrac(:,:)</span>    <span class="twilightBlue">! driver time fraction</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">name</span>    <span class="twilightBlue">! component name</span><br /><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>),  <span class="darkTurquoise">allocatable</span> :: <span class="red">grid(:)</span>        <span class="twilightBlue">! grid object</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Mesh</span>),  <span class="darkTurquoise">allocatable</span> :: <span class="red">mesh(:)</span>        <span class="twilightBlue">! mesh</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>),  <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportField(:)</span> <span class="twilightBlue">! import fields</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>),  <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportField(:)</span> <span class="twilightBlue">! export fields</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportState(:)</span> <span class="twilightBlue">! import state</span><br />        <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:)</span> <span class="twilightBlue">! export state</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Model</span><br /><br />      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Model</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">MODELS(:)</span></div>
 
* Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [<span class="red">Nmodels</span>, <span class="red">Nmodels</span>]. <div class="code">      <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br />        <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span>                <span class="twilightBlue">! active connector</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">divDT</span><br />        <span class="darkTurquoise">integer</span> :: <span class="red">MaskInteraction</span>          <span class="twilightBlue">! connector mask interaction</span><br /><br />        <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span>                    <span class="twilightBlue">! number of connector PETs</span><br />        <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span>  <span class="twilightBlue">! connector PETs list</span><br /><br />        <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100) :: <span class="red">name</span>        <span class="twilightBlue">! connector name</span><br /><br />      <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br />      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Conn</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">CONNECTORS(:,:)</span></div>
 
* Define '''DATA''' Model field processing information from input data files, [<span class="red">1</span>:<span class="red">Nmodels</span>]. Currently, the '''DATA''' Model only export fields and it only supports input NetCDF files. <div class="code">      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_DataSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">DataSet(:)</span></div>
 
*  '''ESM''' clock for driver (zeroth element) and coupled components, [<span class="red">0</span>:<span class="red">Nmodels</span>]. <div class="code">      <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div>
 
*  '''Internal module parameters and variables:'''<br />
** Number of coupled '''ESM''' gridded components. Currently, five '''ESM''' components are supported ('''ROMS''', '''DATA''', '''ATMOSPHERE''', '''SEA-ICE''', and '''WAVE''' model types).  <br /> All supported components are accounted here even if we are running an application with less number of models. The <span class="red">IsActive</span> switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Nmodels</span> = <span class="red">5</span></div>
** Number of nested grids.  All are initialized to just one grid; its values are overwritten during processing. An additional variable '''NgridsR''' is created to avoid using ROMS module <span class="blue">mod_param</span> in the generic interface.  Both '''Ngrids''' and '''NgridsR''' will have the same value.<div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsA</span> = <span class="red">1</span>                  <span class="twilightBlue">! Atmosphere Model</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsD</span> = <span class="red">1</span>                  <span class="twilightBlue">! DATA Model</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsI</span> = <span class="red">1</span>                  <span class="twilightBlue">! Sea-ice Model</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsR</span> = <span class="red">1</span>                  <span class="twilightBlue">! ROMS</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsW</span> = <span class="red">1</span>                  <span class="twilightBlue">! Wave Model</span></div>
** Coupled '''ESM''' components identification indices.<div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idriver</span> = <span class="red">0</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iroms</span>  = <span class="red">1</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iatmos</span>  = <span class="red">2</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idata</span>  = <span class="red">3</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseaice</span> = <span class="red">4</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iwave</span>  = <span class="red">5</span></div>
** Generic '''ESM''' component labels used in the <span class="darkTurquoise">CASE</span> constructs.  We cannot use the identification indices because the vector <span class="red">Iroms(:)</span> cannot be defined as a parameter and a non constant expression is illegal: <span class="darkTurquoise">CASE</span> ( <span class="red">Iroms</span>(<span class="red">1</span>) : <span class="red">Iroms</span>(<span class="red">NgridsR</span>) )<div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 3), <span class="darkTurquoise">allocatable</span> :: <span class="red">Clabel(:)</span><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len=</span>10), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cmodel(:)</span></div>
** Number of ROMS export and import fields per component. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nexport(:)</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nimport(:)</span></div>
** Model coupling type: [<span class="red">1</span>] Explicit, [<span class="red">otherwise</span>] Semi-Implicit. In <span class="red">explicit coupling</span>, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In <span class="red">implicit coupling</span>, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In <span class="red">semi-implicit coupling</span>, '''ROMS -> ATM''' is explicit, '''ATM -> ROMS''' is implicit. <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">CouplingType</span> = <span class="red">1</span></div>
** Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent). <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">ESMcomm(:)</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">sumPETs</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">PETrank</span><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=10) :: <span class="red">PETlayoutOption</span></div>
**  Driver clock parameters specified in configuration script. A integer vector with six elements: (<span class="red">1</span>) year including century, like 2017; (<span class="red">2</span>) month of the year, 1 to 12; (<span class="red">3</span>) day of the month; (<span class="red">4</span>) hour of the day, 0 to 23; (<span class="red">5</span>) minutes of the hour, 0 to 59; and (<span class="red">6</span>) seconds of the minute, 0 to 59 <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">ReferenceDate</span>(6)          <span class="twilightBlue">! reference date</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">RestartDate</span>(6)            <span class="twilightBlue">! restarting date</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">StartDate</span>(6)              <span class="twilightBlue">! starting date</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">StopDate</span>(6)              <span class="twilightBlue">! stopping date</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">TimeStep</span>(6)              <span class="twilightBlue">! coupling interval</span></div>
**  Today date string. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=44) :: <span class="red">TodayDateString</span></div>
**  '''ESM''' coupling simulation reference date number, element: (<span class="red">1</span>) seconds, and (<span class="red">2</span>) fractional days<div class="code">      <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">ReferenceDateNumber</span>(2)</div>
** '''DATA''' component parallel distributed-memory domain partions in the '''I'''- and '''J'''-directions (lon,lat).  <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">ItileD</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">JtileD</span></div>
** Coupling debugging flag: [<span class="red">0</span>] no debugging, [<span class="red">1</span>] reports informative messages, or [<span class="red">2</span>] <span class="red">'1'</span> plus writes grid information in '''VTK''' format, [<span class="red">3</span>] <span class="red">'2'</span> plus writes exchage fields into '''NetCDF''' files. <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">DebugLevel</span> = <span class="red">0</span></div>
** Execution tracing level flag: [<span class="red">0</span>] no tracing, [<span class="red">1</span>] reports sequence of coupling subroutine calls, or [<span class="red">2</span>] <<span class="red">1</span>> plus writes voluminous '''ESMF''' library tracing information which slowdown performace, and creates large log file. <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">TraceLevel</span> = <span class="red">0</span></div>
** Switch to trace/track run sequence during debugging.  All information is written to Fortan unit <span class="red">trac</span>. For now, use standard output unit. <div class="code">      <span class="darkTurquoise">logical</span> :: <span class="red">ESM_track</span> = <span class="red">.FALSE.</span>  <span class="twilightBlue">! trace/track CALL sequence switch</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">trac</span> = <span class="red">6</span>            <span class="twilightBlue">! trace/track CALL sequence unit</span></div>
** Coupled model staggered grid-cell type indices. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inan</span>    = <span class="red">0</span>    <span class="twilightBlue">! unstaggered, cell center</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icenter</span> = <span class="red">1</span>    <span class="twilightBlue">! cell center</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icorner</span> = <span class="red">2</span>    <span class="twilightBlue">! cell corners</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iupoint</span> = <span class="red">3</span>    <span class="twilightBlue">! right and left cell faces</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ivpoint</span> = <span class="red">4</span>    <span class="twilightBlue">! upper and lower cell faces</span><br /><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=6), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">GridType</span> =                  &<br />    &                                    (/ '<span class="red">N/A</span>  ',                &<br />    &                                        '<span class="red">Center</span>',                &<br />    &                                        '<span class="red">Corner</span>',                &<br />    &                                        '<span class="red">U</span>    ',                &<br />    &                                        '<span class="red">V</span>    ' /)</div>
** '''REGRID''' interpolation method between source and destination fields. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inone</span>  = <span class="red">0</span>    <span class="twilightBlue">! none</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ibilin</span>  = <span class="red">1</span>    <span class="twilightBlue">! bilinear</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipatch</span>  = <span class="red">2</span>    <span class="twilightBlue">! high-order patch recovery</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv1</span> = <span class="red">3</span>    <span class="twilightBlue">! first-order conservative</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv2</span> = <span class="red">4</span>    <span class="twilightBlue">! second-order conservative</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InStoD</span>  = <span class="red">5</span>    <span class="twilightBlue">! nearest neighbor Src to Dst</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InDtoS</span>  = <span class="red">6</span>    <span class="twilightBlue">! nearest neighbor Dst to Src</span><br /><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:6) :: <span class="red">IntrpType</span> =                  &<br />    &                                    (/ '<span class="red">NONE</span>',                  &<br />    &                                        '<span class="red">BLIN</span>',                  &<br />    &                                        '<span class="red">PTCH</span>',                  &<br />    &                                        '<span class="red">CNS1</span>',                  &<br />    &                                        '<span class="red">CNS2</span>',                  &<br />    &                                        '<span class="red">NS2D</span>',                  &<br />    &                                        '<span class="red">ND2S</span>' /)</div>
** Extrapolation method for unmapped destination points. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Enone</span>  = <span class="red">0</span>    <span class="twilightBlue">! none</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">ExStoD</span>  = <span class="red">1</span>    <span class="twilightBlue">! nearear neighbor Src to Dst</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Eidavg</span>  = <span class="red">2</span>    <span class="twilightBlue">! inverse distance average</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ecreep</span>  = <span class="red">3</span>    <span class="twilightBlue">! creep fill</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">E2steps</span> = <span class="red">4</span>    <span class="twilightBlue">! Turuncoglu two steps</span><br /><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">ExtrpType</span> =                  &<br />    &                                    (/ '<span class="red">NONE</span>',                  &<br />    &                                        '<span class="red">NS2D</span>',                  &<br />    &                                        '<span class="red">IDAV</span>',                  &<br />    &                                        '<span class="red">CREE</span>',                  &<br />    &                                        '<span class="red">2STP</span>' /)</div>
**  The number of levels to output for the extrapolation methods that fill levels, like creep fill (<span class="red">ESMF_EXTRAPMETHOD_CREEP</span>). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels.  For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding ('''ESMF''' Reference Manual, v '''8.0.0'''). <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">extrapNumLevels</span> = <span class="red">1</span></div>
** Interpolation connectors mask interaction flags. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverLand</span>  = <span class="red">1</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverOcean</span> = <span class="red">2</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverAll</span>  = <span class="red">3</span><br /><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=3), <span class="darkTurquoise">dimension</span>(3) :: <span class="red">MaskType</span> =                    &<br />    &                                  (/ '<span class="red">LND</span>',                      &<br />    &                                      '<span class="red">OCN</span>',                      &<br />    &                                      '<span class="red">ALL</span>'/)</div>
** Coupling run mode: sequential or concurrent. <div class="code">      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseq</span> = <span class="red">1</span><br />      <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipar</span> = <span class="red">2</span><br /><br />      <span class="darkTurquoise">character</span> (len=10), <span class="darkTurquoise">dimension</span>(2) :: <span class="red">RunMode</span> =                    &<br />    &                                  (/ '<span class="red">SEQUENTIA</span>L',              &<br />    &                                      '<span class="red">CONCURRENT</span>' /)</div>
**  Compling standard input parameters filename. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CinpName</span></div>
** '''ESM''' free-format run sequence configuration filename. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CONFname</span></div>
** ROMS coupling '''YAML''' configuration filename. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">CPLname</span></div>
**  Standard input filename for each coupled model, [<span class="red">Nmodels</span>]. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">INPname(:)</span></div>
** Standard output units and log messages filename for coupler and '''ESMF''' library. <div class="code">      <span class="darkTurquoise">integer</span> :: <span class="red">cplout</span>  = <span class="red">77</span>        <span class="twilightBlue">! coupling driver</span><br />      <span class="darkTurquoise">integer</span> :: <span class="red">dataout</span> = <span class="red">77</span>        <span class="twilightBlue">! data component</span><br /><br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">ESMnameLog</span> = '<span class="red">log.esmf</span>'<br />      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">CouplerLog</span> = '<span class="red">log.coupler</span>'</div>
**  Output '''NetCDF''' file used to store field snapshot attributes needed for time interpolation by the '''ESM''' component kernel during concurrent coupling. <div class="code">      <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=17), <span class="darkTurquoise">parameter</span> :: <span class="red">AttFileName</span> = '<span class="red">time_intrp_att.nc</span>'</div>
**  '''ESM''' single and double precision constants. <div class="code">      <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MAPPED_MASK</span> = <span class="red">99_i4b</span><br />      <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">UNMAPPED_MASK</span> = <span class="red">98_i4b</span><br /><br />      <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_dp</span> = <span class="red">1.0E20_dp</span><br />      <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r4</span> = <span class="red">1.0E20_r4</span><br />      <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r8</span> = <span class="red">1.0E20_r8</span><br /><br />      <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_dp</span> = <span class="red">0.5E20_dp</span><br />      <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r4</span> = <span class="red">0.5E20_r4</span><br />      <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r8</span> = <span class="red">0.5E20_r8</span></div>


==Capabilities==
==Capabilities==
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by NUOPC. These subroutines are called by the NUOPC infrastructure according to a predefined calling sequence. Some subroutines are called during initialization of the coupled system, some during the run of the coupled system, and some during finalization of the coupled system.
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by '''NUOPC'''. According to a predefined calling sequence, these subroutines are called by the '''NUOPC''' infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.
 
The initialization sequence is the most complex and is governed by the '''NUOPC''' technical rules. Details about the initialization sequence can be found in the '''NUOPC''' Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.


The initialization sequence is the most complex and is governed by the NUOPC technical rules. Details about the initialization sequence can be found in the NUOPC Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.


{| border="1" cellspacing="0" cellpadding="5" width="1000"
:{| border="1" cellspacing="0" cellpadding="5" width="1000"
|valign="top"|<span class="blue">ROMS_SetServices</span>  
|valign="top"|<span class="blue">ROMS_SetServices</span>  
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component chared-object entry points for using NUOPC generic methods for '''initialize''', '''run''', and '''finalize'''.
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component chared-object entry points for using '''NUOPC''' generic methods for '''initialize''', '''run''', and '''finalize'''.
|-
|-
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span>
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span>
Line 27: Line 87:
|-
|-
|valign="top"|<span class="blue">ROMS_SetClock</span>
|valign="top"|<span class="blue">ROMS_SetClock</span>
|Sets ROMS component date calendar, start and stop times, and coupling interval. At initialization, the variable [[Variables#tdays|tdays]] is the initial time measured in fractional days since the reference time.
|Sets ROMS component date calendar, start and stop times, and coupling interval.
|-
|-
|valign="top"|<span class="blue">ROMS_SetRunClock</span>
|valign="top"|<span class="blue">ROMS_SetRunClock</span>
Line 42: Line 102:
|-
|-
|valign="top"|<span class="blue">ROMS_ModelAdvance</span>
|valign="top"|<span class="blue">ROMS_ModelAdvance</span>
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_import</span> and <span class="blue">ROMS_export</span> routines.
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_Import</span> and <span class="blue">ROMS_Export</span> routines.
|-
|-
|valign="top"|<span class="blue">ROMS_SetFinalize</span>
|valign="top"|<span class="blue">ROMS_SetFinalize</span>
Line 48: Line 108:
|-
|-
|valign="top"|<span class="blue">ROMS_Import</span>
|valign="top"|<span class="blue">ROMS_Import</span>
|Imports fields into ROMS. The fields are loaded into the snapshot storage arrays to allow time interpolation elsewhere.
|Loads import fields into ROMS internal kernel arrays.
|-
|-
|valign="top"|<span class="blue">ROMS_Export</span>
|valign="top"|<span class="blue">ROMS_Export</span>
|Exports ROMS fields to other gridded components.
|Exports ROMS fields to other gridded components.
|-
|-
|valign="top"|<span class="blue">ROMS_Rotate</span>
|Rotates vector components from computational grid to geographical '''EAST''' and '''NORTH''' directions or vice versa.
|-
|}
Other <span class="red">NUOPC</span> cap modules are available within the ROMS native framework:
:{| border="1" cellspacing="0" cellpadding="5" align="left"
| align="center" | '''ESMF/NUOPC Modules'''
| align="center" | '''Tested'''
|style="width: 700px;" align="center" | '''Description'''
|-
| <span class="red">Master/esmf_driver.h</span>
| <span class="forestGreen">yes</span>
| '''ESMF/NUOPC''' Master Driver
|-
| <span class="red">Master/esmf_esm.F</span>
| <span class="forestGreen">yes</span>
| Sets '''ESMF/NUOPC''' '''Services''' and '''RunSequence''' for each coupled component
|-
| <span class="red">Master/esmf_coupler.h</span>
| <span class="forestGreen">yes</span>
| '''Computes''', '''Execute''', and '''Release''' the '''Connectors''' between <span class="violet">source</span> and <span class="violet">destination</span> fields
|-
| <span class="red">Master/mod_esmf_esm.h</span>
| <span class="forestGreen">yes</span>
| Defines framework object '''structures''' and '''variables''' and includes support routines
|-
| <span class="blue">Master/esmf_atm_coamps.h</span>
| <span class="forestGreen">yes</span>
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')
|-
| <span class="blue">Master/esmf_atm_coamps.h</span>
| <span class="forestGreen">yes</span>
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')
|-
| <span class="blue">Master/esmf_atm_regcm.h</span>
| <span class="red">no</span>
| ICTP Regional Climate Model ('''RegCM''', Version '''4.5''')
|-
| <span class="blue">Master/esmf_atm_wrf.h</span>
| <span class="forestGreen">yes</span>
| Weather Research and Forecasting model ('''WRF''')
|-
| <span class="blue">Master/esmf_data.F</span>
| <span class="forestGreen">yes</span>
| Generic '''DATA''' component via '''NetCDF''' files
|-
| <span class="blue">Master/esmf_ice_cice.h</span>
| <span class="red">no</span>
| Los Alamos Sea Ice Model ('''CICE''')
|-
| <span class="blue">Master/esmf_roms.F</span>
| <span class="forestGreen">yes</span>
| Regional Ocean Modeling System ('''ROMS''')
|-
| <span class="blue">Master/esmf_wav_wam.h</span>
| <span class="red">no</span>
| ECMWF's Wave Model ('''WAM''')
|-
| <span class="blue">Master/esmf_wav_ww3.h</span>
| <span class="red">no</span>
| WaveWatch III ('''WW3'''), <span class="twilightBlue">under development</span>
|}
<div style="clear: both"></div>
==Import and Export Fields==
The <span class="red">export</span> and <span class="red">import</span> fields are configured and specified in the '''YAML''' file [[coupling_esmf_atm.yaml]]. Check the following [[coupling_esmf_atm.yaml|wikiROMS]] page for detailed information.  The user has full control of the coupling exchange fields metadata.
The '''ESMF/NUOPC''' library version '''8.0.0''' or higher supports native nesting capabilities.  The '''NUOPC''' layer now allows coupling sets in the <span class="red">import</span> and <span class="red">export</span> states of the '''ESM''' components. A nested grid is represented as another '''ESMF_Grid''' object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The '''NUOPC''' connector can recognize the different grids and exchange fields to a connected '''ESM''' component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The <span class="blue">NUOPC_AddNestedState</span> routine is used to advertise the nested couple sets.
For example, the ROMS '''NOUPC''' <span class="red">cap</span> module routine <span class="blue">ROMS_SetInitializeP1</span> illustrates how easily the coupled sets is added to the <span class="red">ImportState</span> and <span class="red">ExportState</span> as shown below in the green blocks of code.
{|cellpadding="3"
|-
|[[Image:ImportCoupledStates.png]]
|[[Image:ExportCoupledStates.png]]
|}
|}

Latest revision as of 19:47, 30 August 2022

ROMS Native ESMF/NUOPC Cap

Overview

This document describes the ROMS ESMF/NUOPC cap module Master/esmf_roms.F used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (Master/esmf_driver.h), which creates, register, initialize, run, and finalize the coupling of each Earth System Model (ESM) component (data, atmosphere, sea-ice, wave, and ocean). It also sets the coupler connectors and the RunSequence policy (Master/esmf_esm.F). The framework provides the NUOPC cap modules for each coupled component. Detailed information about ESMF/NUOPC and how it is implemented in ROMS can be found on the Earth System Modeling Framework WikiROMS page.

A NUOPC cap is a Fortran module that serves as the interface to a model when used in a NUOPC-based coupled system. The term cap is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its initialize, run, and finalize phases and exposing the model data structures to the field exchange (import and export states) between coupled components.


ROMS Coupling.png
Figure 1: ROMS native ESMF/NUOPC coupling infrastructure

Implementation

The Master/mod_esmf_esm.F module declares several derived-type structures to facilitate the management of all internal objects and variables in each ESM coupled component:

  • ESM coupling time managing variables and ESMF objects. The ClockInfo object is allocated as (0:Nmodels), where Nmodels is the number of coupled ESM components. Here, index zero corresponds to the driver.
    TYPE :: ESM_Clock

    logical :: Restarted

    integer (i8b) :: AdvanceCount ! advance counter

    real (dp) :: Current_Time ! seconds
    real (dp) :: Time_Reference ! seconds
    real (dp) :: Time_Restart ! seconds
    real (dp) :: Time_Start ! seconds
    real (dp) :: Time_Stop ! seconds
    real (dp) :: Time_Step ! seconds

    character (len=22) :: Name
    character (len=22) :: CalendarString ! 360_day, gregorian
    character (len=22) :: Time_ReferenceString
    character (len=22) :: Time_RestartString
    character (len=22) :: Time_StartString
    character (len=22) :: Time_StopString

    TYPE (ESMF_Calendar)  :: Calendar
    TYPE (ESMF_Clock)  :: Clock
    TYPE (ESMF_Direction_flag) :: Direction
    TYPE (ESMF_Time)  :: CurrentTime
    TYPE (ESMF_Time)  :: ReferenceTime
    TYPE (ESMF_Time)  :: RestartTime
    TYPE (ESMF_Time)  :: StartTime
    TYPE (ESMF_Time)  :: StopTime
    TYPE (ESMF_TimeInterval)  :: TimeStep

    END TYPE ESM_Clock

    . . .

    TYPE (ESM_Clock), allocatable, target :: ClockInfo(:)
  • ESM coupled state sets. If appropriate, it includes the logic for connecting nested grids. The COUPLED object is of size Nmodels.
    TYPE :: ESM_CplSet

    logical, allocatable :: LinkedGrid(:,:) ! connected grid

    logical, allocatable :: DataCoupledSets(:,:) ! DATA linked sets

    character (len=100), allocatable :: SetLabel(:) ! set label
    character (len=100), allocatable :: ExpLabel(:) ! export label
    character (len=100), allocatable :: ImpLabel(:) ! import label

    TYPE (ESMF_State), allocatable :: ExportState(:,:) ! export set

    END TYPE ESM_CplSet

    TYPE (ESM_CplSet), allocatable, target :: COUPLED(:)
  • DATA model field processing information. The DATA component is often necessary for applications with incongruent grids to provide complete field melding capabilities or methods for reading and interpolating stream data in space and time from archived NetCDF files.
    TYPE ESM_Data

    logical :: Lcycle ! cycling time coordinate
    logical :: Lcoord ! coordinates attribute
    logical :: Lmask ! land/sea mask
    logical :: Lmulti ! field across multi-files
    logical :: LastRec ! processed last record

    integer :: Icomp ! target component index
    integer :: ncid ! NetCDF file ID
    integer :: Tid ! time variable ID
    integer :: Vid ! field variable ID
    integer :: Nvdim ! number spatial dimensions
    integer :: Zlevel ! level index to process
    integer :: Nrec ! number of time records
    integer :: Trec ! latest read time record
    integer :: Tindex ! rolling two-time indices
    integer :: LandValue ! Masking land value
    integer :: SeaValue ! Masking sea value

    integer, allocatable :: Vsize(:) ! dimensions size

    real(dp) :: add_offset ! add_offset attribute
    real(dp) :: FillValue ! _FillValue attribute
    real(dp) :: scale_factor ! scale_factor attribute
    real(dp) :: Clength ! time cycling length
    real(r8) :: LonMin ! grid minimum longitude
    real(r8) :: LonMax ! grid maximum longitude
    real(r8) :: LatMin ! grid minimum latitude
    real(r8) :: LatMax ! grid maximum latitude
    real(dp) :: Tscale ! time scale to day
    real(dp) :: Tmono ! monotonic time (days)
    real(dp) :: Tmin ! time minimum value
    real(dp) :: Tmax ! time maximum value
    real(dp) :: Tstr ! lower time-snapshot
    real(dp) :: Tend ! upper time-snapshot
    real(r8) :: Vmin ! variable minimum value
    real(r8) :: Vmax ! variable maximum value
    real(dp) :: Tintrp(2) ! interpolation time (days)
    real(dp) :: Vtime(2) ! latest two-time values
    real(dp) :: Date(6,2) ! time-snapshots dates YYYY,MM,DD hh:mm:ss.ss

    character(len=30), allocatable :: Dname(:) ! variable dimensions names
    character(len=20), allocatable :: Vcoord(:) ! variable coordinates names

    character(len=22 ) :: DateString(2) ! date-snapshots string
    character(len=30 ) :: SpecialAction ! special processing
    character(len=100) :: Tname ! time variable name
    character(len=100) :: Tunits ! time variable units
    character(len=100) :: Vname ! variable name
    character(len=100) :: Vunits ! variable units
    character(len=256) :: Vdescriptor ! Variable descriptive name
    character(len=256) :: Vlongname ! long_name attribute
    character(len=256) :: ncfile ! NetCDF filename

    real(r8), allocatable :: lon(:,:) ! field longitude
    real(r8), allocatable :: lat(:,:) ! field latitude
    real(r8), allocatable :: mask(:,:) ! field land/sea mask

    real(r8), allocatable :: A2d(:,:) ! time interpolated 2D data
    real(r8), allocatable :: A3d(:,:,:) ! time interpolated 3D data
    real(r8), allocatable :: A2dG(:,:,:) ! latest 2D data snapsnots
    real(r8), allocatable :: A3dG(:,:,:,:)! latest 3D data snapsnots

    TYPE (ESMF_Field)  :: field ! field object
    TYPE (ESMF_Grid)  :: grid ! field grid object
    TYPE (ESMF_RouteHandle) :: rhandle ! field RouteHandle

    END TYPE ESM_Data
  • DATA model high-level structure.
    TYPE ESM_DataSet

    integer :: Nfields ! number of fields
    integer :: Nfiles ! number of input files

    character(len=20), allocatable :: Ctarget(:) ! component target
    character(len=20), allocatable :: Field(:) ! field short-name

    TYPE (ESM_Data), allocatable :: Export(:) ! Export field structure
    TYPE (T_IO), allocatable :: IFS(:) ! Input Files structure

    END TYPE ESM_DataSet
  • Import and export field metadata information between the source (SRC) and destination (DST) components.
    TYPE :: ESM_Field

    logical :: connected ! connected to coupler
    logical :: debug_write ! write exchanged field
    logical :: enable_integral_adj ! area integral adjusted

    integer :: fid ! internal field ID
    integer :: gtype ! field grid mesh type
    integer :: etype ! field extrapolation flag
    integer :: itype ! field interpolation flag
    integer :: Tindex ! rolling two-time indices

    character (len=20) :: Ctarget ! component destination
    character (len=22) :: DateString(2) ! date-snapshots string

    character (len=:), allocatable :: short_name ! short name
    character (len=:), allocatable :: standard_name ! standard name
    character (len=:), allocatable :: long_name ! long name
    character (len=:), allocatable :: dst_gtype ! DST grid type
    character (len=:), allocatable :: dst_units ! DST units
    character (len=:), allocatable :: src_gtype ! SRC grid type
    character (len=:), allocatable :: src_units ! SRC units
    character (len=:), allocatable :: nc_vname ! DATA Vname
    character (len=:), allocatable :: nc_tname ! DATA Tname
    character (len=:), allocatable :: RegridMethod ! regrid method
    character (len=:), allocatable :: ExtrapMethod ! extrapolate

    real (dp) :: scale_factor ! field scale factor
    real (dp) :: add_offset ! field add offset value
    real (dp) :: Tmin ! DATA time minimum value
    real (dp) :: Tmax ! DATA time maximum value
    real (dp) :: Tstr ! DATA lower time-snapshot
    real (dp) :: Tend ! DATA upper time-snapshot
    real (dp) :: Tintrp(2) ! interpolation time (day)
    real (dp) :: Vtime(2) ! latest two-time values

    TYPE (ESMF_RouteHandle) :: rhandle  ! field RouteHandle

    END TYPE ESM_Field
  • Import and export fields mesh data.
    TYPE :: ESM_Mesh

    integer :: gid ! grid ID
    integer :: gtype ! grid mesh type

    integer (i4b), allocatable :: mask(:,:) ! grid land/sea mask

    real (r8), allocatable :: lon(:,:) ! grid longitude
    real (r8), allocatable :: lat(:,:) ! grid latitude
    real (r8), allocatable :: area(:,:) ! grid area

    END TYPE ESM_Mesh
  • Melding coefficients used to combine fields from DATA and ESM components. The weight factors are read from the input NetCDF specified in the WeightsFile(atmos) keyword. The user has full control of how the merging is done. It is recommended to provide a gradual transition between the two components.

    Recall that the DATA component supplies needed data to a particular ESM component. For example, it may export data to the atmosphere model at locations not covered by the other ESM components because of smaller grid coverage. If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data:
    SST_atm(:,:) = Cesm(:,:) * SST_esm(:,:) + Cdat(:,:) * SST_dat(:,:)
    where Cesm(:,:) + Cdat(:,:) = 1.
    TYPE ESM_Meld

    integer :: NestedGrid ! grid needing merged field

    character(len=100) :: VnameDATA ! DATA weights variable name
    character(len=100) :: VnameESM ! ESM weights variable name
    character(len=256) :: ncfile ! Weights NetCDF filename

    real (r8), allocatable :: Cdat(:,:) ! coefficients for DATA
    real (r8), allocatable :: Cesm(:,:) ! coefficients for ESM

    END TYPE ESM_Meld

    TYPE (ESM_Meld), allocatable, target :: WEIGHTS(:)

    real (dp) :: WeightDAT = 0.0_dp ! DATA component weight
    real (dp) :: WeightESM = 1.0_dp ! ESM component weight
  • Coupled models high-level object, [Nmodels=5].
    TYPE :: ESM_Model

    logical :: IsActive ! active for coupling

    integer (i4b) :: LandValue ! land mask value
    integer (i4b) :: SeaValue ! sea mask value

    integer :: Ngrids ! number nested grids

    integer :: ExportCalls ! export CALL counter
    integer :: ImportCalls ! import CALL counter

    integer :: nPETs ! number model PETs
    integer, allocatable :: PETlist(:) ! component PETs list

    integer, allocatable :: TimeFrac(:,:) ! driver time fraction

    character (len=:), allocatable :: name ! component name

    TYPE (ESMF_Grid), allocatable :: grid(:) ! grid object
    TYPE (ESM_Mesh), allocatable :: mesh(:) ! mesh
    TYPE (ESM_Field), allocatable :: ImportField(:) ! import fields
    TYPE (ESM_Field), allocatable :: ExportField(:) ! export fields
    TYPE (ESMF_State), allocatable :: ImportState(:) ! import state
    TYPE (ESMF_State), allocatable :: ExportState(:) ! export state

    END TYPE ESM_Model

    TYPE (ESM_Model), allocatable, target :: MODELS(:)
  • Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [Nmodels, Nmodels].
    TYPE ESM_Conn

    logical :: IsActive ! active connector

    integer :: divDT
    integer :: MaskInteraction ! connector mask interaction

    integer :: nPETs ! number of connector PETs
    integer, allocatable :: PETlist(:) ! connector PETs list

    character (len=100) :: name ! connector name

    END TYPE ESM_Conn

    TYPE (ESM_Conn), allocatable, target :: CONNECTORS(:,:)
  • Define DATA Model field processing information from input data files, [1:Nmodels]. Currently, the DATA Model only export fields and it only supports input NetCDF files.
    TYPE (ESM_DataSet), allocatable, target :: DataSet(:)
  • ESM clock for driver (zeroth element) and coupled components, [0:Nmodels].
    TYPE (ESM_Clock), allocatable, target :: ClockInfo(:)
  • Internal module parameters and variables:
    • Number of coupled ESM gridded components. Currently, five ESM components are supported (ROMS, DATA, ATMOSPHERE, SEA-ICE, and WAVE model types).
      All supported components are accounted here even if we are running an application with less number of models. The IsActive switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active.
      integer, parameter :: Nmodels = 5
    • Number of nested grids. All are initialized to just one grid; its values are overwritten during processing. An additional variable NgridsR is created to avoid using ROMS module mod_param in the generic interface. Both Ngrids and NgridsR will have the same value.
      integer, parameter :: NgridsA = 1 ! Atmosphere Model
      integer, parameter :: NgridsD = 1 ! DATA Model
      integer, parameter :: NgridsI = 1 ! Sea-ice Model
      integer, parameter :: NgridsR = 1 ! ROMS
      integer, parameter :: NgridsW = 1 ! Wave Model
    • Coupled ESM components identification indices.
      integer, parameter :: Idriver = 0
      integer, parameter :: Iroms = 1
      integer, parameter :: Iatmos = 2
      integer, parameter :: Idata = 3
      integer, parameter :: Iseaice = 4
      integer, parameter :: Iwave = 5
    • Generic ESM component labels used in the CASE constructs. We cannot use the identification indices because the vector Iroms(:) cannot be defined as a parameter and a non constant expression is illegal: CASE ( Iroms(1) : Iroms(NgridsR) )
      character (len= 3), allocatable :: Clabel(:)
      character (len=10), allocatable :: Cmodel(:)
    • Number of ROMS export and import fields per component.
      integer, allocatable :: Nexport(:)
      integer, allocatable :: Nimport(:)
    • Model coupling type: [1] Explicit, [otherwise] Semi-Implicit. In explicit coupling, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In implicit coupling, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In semi-implicit coupling, ROMS -> ATM is explicit, ATM -> ROMS is implicit.
      integer :: CouplingType = 1
    • Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent).
      integer, allocatable :: ESMcomm(:)
      integer :: sumPETs
      integer :: PETrank
      character (len=10) :: PETlayoutOption
    • Driver clock parameters specified in configuration script. A integer vector with six elements: (1) year including century, like 2017; (2) month of the year, 1 to 12; (3) day of the month; (4) hour of the day, 0 to 23; (5) minutes of the hour, 0 to 59; and (6) seconds of the minute, 0 to 59
      integer :: ReferenceDate(6) ! reference date
      integer :: RestartDate(6) ! restarting date
      integer :: StartDate(6) ! starting date
      integer :: StopDate(6) ! stopping date
      integer :: TimeStep(6) ! coupling interval
    • Today date string.
      character (len=44) :: TodayDateString
    • ESM coupling simulation reference date number, element: (1) seconds, and (2) fractional days
      real (dp) :: ReferenceDateNumber(2)
    • DATA component parallel distributed-memory domain partions in the I- and J-directions (lon,lat).
      integer :: ItileD
      integer :: JtileD
    • Coupling debugging flag: [0] no debugging, [1] reports informative messages, or [2] '1' plus writes grid information in VTK format, [3] '2' plus writes exchage fields into NetCDF files.
      integer :: DebugLevel = 0
    • Execution tracing level flag: [0] no tracing, [1] reports sequence of coupling subroutine calls, or [2] <1> plus writes voluminous ESMF library tracing information which slowdown performace, and creates large log file.
      integer :: TraceLevel = 0
    • Switch to trace/track run sequence during debugging. All information is written to Fortan unit trac. For now, use standard output unit.
      logical :: ESM_track = .FALSE. ! trace/track CALL sequence switch
      integer :: trac = 6 ! trace/track CALL sequence unit
    • Coupled model staggered grid-cell type indices.
      integer, parameter :: Inan = 0 ! unstaggered, cell center
      integer, parameter :: Icenter = 1 ! cell center
      integer, parameter :: Icorner = 2 ! cell corners
      integer, parameter :: Iupoint = 3 ! right and left cell faces
      integer, parameter :: Ivpoint = 4 ! upper and lower cell faces

      character (len=6), dimension(0:4) :: GridType = &
      & (/ 'N/A ', &
      & 'Center', &
      & 'Corner', &
      & 'U ', &
      & 'V ' /)
    • REGRID interpolation method between source and destination fields.
      integer, parameter :: Inone = 0 ! none
      integer, parameter :: Ibilin = 1 ! bilinear
      integer, parameter :: Ipatch = 2 ! high-order patch recovery
      integer, parameter :: Iconsv1 = 3 ! first-order conservative
      integer, parameter :: Iconsv2 = 4 ! second-order conservative
      integer, parameter :: InStoD = 5 ! nearest neighbor Src to Dst
      integer, parameter :: InDtoS = 6 ! nearest neighbor Dst to Src

      character (len=4), dimension(0:6) :: IntrpType = &
      & (/ 'NONE', &
      & 'BLIN', &
      & 'PTCH', &
      & 'CNS1', &
      & 'CNS2', &
      & 'NS2D', &
      & 'ND2S' /)
    • Extrapolation method for unmapped destination points.
      integer, parameter :: Enone = 0 ! none
      integer, parameter :: ExStoD = 1 ! nearear neighbor Src to Dst
      integer, parameter :: Eidavg = 2 ! inverse distance average
      integer, parameter :: Ecreep = 3 ! creep fill
      integer, parameter :: E2steps = 4 ! Turuncoglu two steps

      character (len=4), dimension(0:4) :: ExtrpType = &
      & (/ 'NONE', &
      & 'NS2D', &
      & 'IDAV', &
      & 'CREE', &
      & '2STP' /)
    • The number of levels to output for the extrapolation methods that fill levels, like creep fill (ESMF_EXTRAPMETHOD_CREEP). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels. For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding (ESMF Reference Manual, v 8.0.0).
      integer :: extrapNumLevels = 1
    • Interpolation connectors mask interaction flags.
      integer, parameter :: OverLand = 1
      integer, parameter :: OverOcean = 2
      integer, parameter :: OverAll = 3

      character (len=3), dimension(3) :: MaskType = &
      & (/ 'LND', &
      & 'OCN', &
      & 'ALL'/)
    • Coupling run mode: sequential or concurrent.
      integer, parameter :: Iseq = 1
      integer, parameter :: Ipar = 2

      character (len=10), dimension(2) :: RunMode = &
      & (/ 'SEQUENTIAL', &
      & 'CONCURRENT' /)
    • Compling standard input parameters filename.
      character (len=256) :: CinpName
    • ESM free-format run sequence configuration filename.
      character (len=256) :: CONFname
    • ROMS coupling YAML configuration filename.
      character (len=256), allocatable :: CPLname
    • Standard input filename for each coupled model, [Nmodels].
      character (len=256), allocatable :: INPname(:)
    • Standard output units and log messages filename for coupler and ESMF library.
      integer :: cplout = 77 ! coupling driver
      integer :: dataout = 77 ! data component

      character (len= 8), parameter :: ESMnameLog = 'log.esmf'
      character (len= 8), parameter :: CouplerLog = 'log.coupler'
    • Output NetCDF file used to store field snapshot attributes needed for time interpolation by the ESM component kernel during concurrent coupling.
      character (len=17), parameter :: AttFileName = 'time_intrp_att.nc'
    • ESM single and double precision constants.
      integer (i4b), parameter :: MAPPED_MASK = 99_i4b
      integer (i4b), parameter :: UNMAPPED_MASK = 98_i4b

      real (dp), parameter :: MISSING_dp = 1.0E20_dp
      real (r4), parameter :: MISSING_r4 = 1.0E20_r4
      real (r8), parameter :: MISSING_r8 = 1.0E20_r8

      real (dp), parameter :: TOL_dp = 0.5E20_dp
      real (r4), parameter :: TOL_r4 = 0.5E20_r4
      real (r8), parameter :: TOL_r8 = 0.5E20_r8

Capabilities

The ROMS cap module contains a set of subroutines that are required by NUOPC. According to a predefined calling sequence, these subroutines are called by the NUOPC infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.

The initialization sequence is the most complex and is governed by the NUOPC technical rules. Details about the initialization sequence can be found in the NUOPC Reference Manual. The ROMS cap requires ESMF version 8 or higher.


ROMS_SetServices Entry point to the ROMS cap and the only public routine. It sets the ROMS component chared-object entry points for using NUOPC generic methods for initialize, run, and finalize.
ROMS_SetInitializeP1 ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.
ROMS_SetInitializeP2 ROMS component phase 2 initialization which initializes the ROMS component (ROMS_initialize), sets component grid (ROMS_SetGridArrays), and adds fields into import and export into respective states.
ROMS_DataInit Exports ROMS component fields during initialization or restart.
ROMS_SetClock Sets ROMS component date calendar, start and stop times, and coupling interval.
ROMS_SetRunClock Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.
ROMS_CheckImport Checks if ROMS component import field is at the correct time.
ROMS_SetGridArrays Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.
ROMS_SetStates Adds ROMS component export and import fields into its respective state.
ROMS_ModelAdvance Advances ROMS component for a coupling interval. It calls ROMS_Import and ROMS_Export routines.
ROMS_SetFinalize Finalizes ROMS component execution.
ROMS_Import Loads import fields into ROMS internal kernel arrays.
ROMS_Export Exports ROMS fields to other gridded components.
ROMS_Rotate Rotates vector components from computational grid to geographical EAST and NORTH directions or vice versa.


Other NUOPC cap modules are available within the ROMS native framework:


ESMF/NUOPC Modules Tested Description
Master/esmf_driver.h yes ESMF/NUOPC Master Driver
Master/esmf_esm.F yes Sets ESMF/NUOPC Services and RunSequence for each coupled component
Master/esmf_coupler.h yes Computes, Execute, and Release the Connectors between source and destination fields
Master/mod_esmf_esm.h yes Defines framework object structures and variables and includes support routines
Master/esmf_atm_coamps.h yes Coupled Ocean-Atmosphere Mesoscale Prediction System (COAMPS)
Master/esmf_atm_coamps.h yes Coupled Ocean-Atmosphere Mesoscale Prediction System (COAMPS)
Master/esmf_atm_regcm.h no ICTP Regional Climate Model (RegCM, Version 4.5)
Master/esmf_atm_wrf.h yes Weather Research and Forecasting model (WRF)
Master/esmf_data.F yes Generic DATA component via NetCDF files
Master/esmf_ice_cice.h no Los Alamos Sea Ice Model (CICE)
Master/esmf_roms.F yes Regional Ocean Modeling System (ROMS)
Master/esmf_wav_wam.h no ECMWF's Wave Model (WAM)
Master/esmf_wav_ww3.h no WaveWatch III (WW3), under development

Import and Export Fields

The export and import fields are configured and specified in the YAML file coupling_esmf_atm.yaml. Check the following wikiROMS page for detailed information. The user has full control of the coupling exchange fields metadata.

The ESMF/NUOPC library version 8.0.0 or higher supports native nesting capabilities. The NUOPC layer now allows coupling sets in the import and export states of the ESM components. A nested grid is represented as another ESMF_Grid object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The NUOPC connector can recognize the different grids and exchange fields to a connected ESM component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The NUOPC_AddNestedState routine is used to advertise the nested couple sets.

For example, the ROMS NOUPC cap module routine ROMS_SetInitializeP1 illustrates how easily the coupled sets is added to the ImportState and ExportState as shown below in the green blocks of code.


ImportCoupledStates.png ExportCoupledStates.png