Difference between revisions of "ROMS-JEDI Compiling"
(Created page with "<div class="title">Compiling ROMS-JEDI</div> ==Configuring GIT== Before downloading the '''ROMS-JEDI''' repository, ensure that your '''~/.gitconfig''' has the appropriate '...") (change visibility) |
|||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<div class="title">Compiling ROMS-JEDI</div> | <div class="title">Compiling ROMS-JEDI</div> | ||
<!-- Edit Template:ROMS-JEDI_TOC to modify this Table of Contents--> | |||
<div style="float: left;margin: 0 20px 0 0;">{{ROMS-JEDI_TOC}}</div>__TOC__ | |||
<div style="clear: both"></div> | |||
==Configuring GIT== | ==Configuring GIT== | ||
| Line 23: | Line 29: | ||
==Compiling ROMS-JEDI Default Application WC13== | ==Compiling ROMS-JEDI Default Application WC13== | ||
By default, '''ROMS-JEDI''' is configured with a coarse (54x53x30 grid points) US West Coast ('''WC13''') application. See [[4DVar Tutorial Introduction]] for detailed information about the '''WC13''' configuration. However, any other ROMS application can be configured by including in the '''ecbuild''' command the directives ''' | By default, '''ROMS-JEDI''' is configured with a coarse (54x53x30 grid points) US West Coast ('''WC13''') application. See [[4DVar Tutorial Introduction]] for detailed information about the '''WC13''' configuration. However, any other ROMS application can be configured by including in the '''ecbuild''' command the directives '''-D<span class="twilightBlue">ROMS_APP</span>=MyAppCPP''' and '''-D<span class="twilightBlue">ROMS_APP_DIR</span>=MyAppDirPath'''. | ||
The strategy is to create a '''Bundle''' and '''build''' subdirectory for each '''ROMS-JEDI''' application. For '''WC13''' (default), we create '''Bundle_wc13''' and '''build_wc13''' below. Please follow the following steps to configure '''ROMS-JEDI''' in your computer:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span> !> If the first time on your computer<br /><br /><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_wc13</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_wc13</span><br /><span class="red">mkdir build_wc13</span><br /><span class="red">cd build_wc13</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DCMAKE_BUILD_TYPE=Release ../Bundle_wc13</span><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Tests<br /><br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi/test/Data<br /> !> to check the results in various sub-directories</div> | The strategy is to create a '''Bundle''' and '''build''' subdirectory for each '''ROMS-JEDI''' application. For '''WC13''' (default), we create '''Bundle_wc13''' and '''build_wc13''' below. Please follow the following steps to configure '''ROMS-JEDI''' in your computer:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span> !> If the first time on your computer<br /><br /><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_wc13</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_wc13</span><br /><span class="red">mkdir build_wc13</span><br /><span class="red">cd build_wc13</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DCMAKE_BUILD_TYPE=Release ../Bundle_wc13</span><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Tests<br /><br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi/test/Data<br /> !> to check the results in various sub-directories</div> | ||
The '''ecbuild''' command will clone the required '''JEDI''' components that are specified in '''Bundle_wc13/CMakeList.txt'''. For example:<div class="box"># Required repositories<br /><br /> ecbuild_bundle( PROJECT roms GIT "<nowiki>https://github.com/ | The '''ecbuild''' command will clone the required '''JEDI''' components that are specified in '''Bundle_wc13/CMakeList.txt'''. For example:<div class="box"># Required repositories<br /><br /> ecbuild_bundle( PROJECT roms GIT "<nowiki>https://github.com/myroms/roms.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT oops GIT "<nowiki>https://github.com/jcsda-internal/oops.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT vader GIT "<nowiki>https://github.com/jcsda-internal/vader.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT saber GIT "<nowiki>https://github.com/jcsda-internal/saber.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT gsw GIT "<nowiki>https://github.com/jcsda-internal/GSW-Fortran.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ioda GIT "<nowiki>https://github.com/jcsda-internal/ioda.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ufo GIT "<nowiki>https://github.com/jcsda-internal/ufo.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT roms-jedi SOURCE ../ )</div> | ||
The '''ecbuild''' tool builds by default the '''RelWithDebInfo''' ('''-O2 -g''' options) for optimized with debugging information version of '''JEDI''' and '''ROMS-JEDI''', rendering slower execution. However, for faster and optimized execution, we include the directive '''-DCMAKE_BUILD_TYPE=Release''' to the '''ecbuild''' command. For debugging with TotalView, we include instead the directive '''-DCMAKE_BUILD_TYPE=Debug''', which is much slower. | The '''ecbuild''' tool builds by default the '''RelWithDebInfo''' ('''-O2 -g''' options) for optimized with debugging information version of '''JEDI''' and '''ROMS-JEDI''', rendering slower execution. However, for faster and optimized execution, we include the directive '''-DCMAKE_BUILD_TYPE=Release''' to the '''ecbuild''' command. For debugging with TotalView, we include instead the directive '''-DCMAKE_BUILD_TYPE=Debug''', which is much slower. | ||
Latest revision as of 13:00, 31 August 2023
| ROMS-JEDI Menu |
|---|
| 1. Introduction |
| 2. Compiling |
| 3. Implementation |
| 4. Observations |
| 5. Unit Test Cases |
Configuring GIT
Before downloading the ROMS-JEDI repository, ensure that your ~/.gitconfig has the appropriate git-lfs configuration for correctly downloading the default application (U.S. West Coast, WC13 option) input and observation NetCDF files. Otherwise, the Unit Test Cases requiring input NetCDF files will fail. The Git LFS is a command line extension and specification for managing large files with Git. A sample of the configuration file looks like this:
[User]
name = GivenName MiddleName FamilyName
email = your@email
[credential]
helper = cache --timeout=7200
helper = store --file ~/.my-credentials
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = true
Alternatively, you may execute git lfs pull at your location of this repository to download a viable version of the Git LFS files from the remote repository in GitHub. Also, to automatically add the LFS filter to your existing ~/.gitconfig, you could use git lfs install from anywhere in your computer directories.
Downloading ROMS-JEDI
The ROMS-JEDI repository is private within the Joint Center for Satellite Data Assimilation (JCSDA) community. Users need to be partners, contributors, or members of JCSDA to access all the JEDI components that are part of the abstract layer infrastructure centered around the Object Oriented Prediction System (OOPS).
Use the following command to download the ROMS-JEDI interface:
Requirements
- Make sure you have the spack-stack software installed for either gfortran or ifort containing several packages need to run the ROMS-JEDI interface. For more information, please check https://github.com/JCSDA/spack-stack.
In our computers at Rutgers University, the spack-stack can be loaded by executing either:module purge; module load stack-gcc; module list (gfortran compiler)Installing the spack-stack is not trivial and requires computer software expertise.
module purge; module load stack-intel; module list (ifort compiler)
- We also like to define the MPIRUN environmental variable to specify the MPI executable in a particular computer (say, srun, mpirun, etc.). We need the full path for ctest to work. Notice that we specify this executable in the ecbuild command below.
For example:setenv MPIRUN /opt/slurm/bin/srun
- Several environment variables can be activated for verbose debugging of the building blocks for the ROMS-JEDI interface: (use 1 or true to activate and 0 or false to deactivate)setenv LdebugGeometry 1
setenv LdebugAnalyticInit 1
setenv LdebugField 1
setenv LdebugFields 1
setenv LdebugFieldsUtils 1
setenv LdebugGeometry 1
setenv LdebugLinearModel 1
setenv LdebugLinearModel2Geovals 1
setenv LdebugModel 1
setenv LdebugModel2Geovals 1
setenv LdebugTrajectory 1
Compiling ROMS-JEDI Default Application WC13
By default, ROMS-JEDI is configured with a coarse (54x53x30 grid points) US West Coast (WC13) application. See 4DVar Tutorial Introduction for detailed information about the WC13 configuration. However, any other ROMS application can be configured by including in the ecbuild command the directives -DROMS_APP=MyAppCPP and -DROMS_APP_DIR=MyAppDirPath.
The strategy is to create a Bundle and build subdirectory for each ROMS-JEDI application. For WC13 (default), we create Bundle_wc13 and build_wc13 below. Please follow the following steps to configure ROMS-JEDI in your computer:
cd roms-jedi !> ROMS-JEDI interface directory, <interface_dir>
mkdir Bundle_wc13
cp bundle/.gitignore bundle/CMakeLists.txt Bundle_wc13
mkdir build_wc13
cd build_wc13
ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DCMAKE_BUILD_TYPE=Release ../Bundle_wc13
make -j 10
cd roms-jedi !> sub-directory: <interface_dir>/build_wc13/roms-jedi
ctest -N !> lists all the Unit Tests available
ctest -E -V get_ !> runs all the Unit Tests
cd test/Data !> sub-directory: <interface_dir>/build_wc13/roms-jedi/test/Data
!> to check the results in various sub-directories
The ecbuild command will clone the required JEDI components that are specified in Bundle_wc13/CMakeList.txt. For example:
ecbuild_bundle( PROJECT roms GIT "https://github.com/myroms/roms.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT oops GIT "https://github.com/jcsda-internal/oops.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT vader GIT "https://github.com/jcsda-internal/vader.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT saber GIT "https://github.com/jcsda-internal/saber.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT gsw GIT "https://github.com/jcsda-internal/GSW-Fortran.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT ioda GIT "https://github.com/jcsda-internal/ioda.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT ufo GIT "https://github.com/jcsda-internal/ufo.git" BRANCH develop UPDATE )
ecbuild_bundle( PROJECT roms-jedi SOURCE ../ )
The ecbuild tool builds by default the RelWithDebInfo (-O2 -g options) for optimized with debugging information version of JEDI and ROMS-JEDI, rendering slower execution. However, for faster and optimized execution, we include the directive -DCMAKE_BUILD_TYPE=Release to the ecbuild command. For debugging with TotalView, we include instead the directive -DCMAKE_BUILD_TYPE=Debug, which is much slower.
Compiling other ROMS-JEDI Applications
Any ROMS application can be run in ROMS-JEDI by specifying the appropriate CPP option and necessary input files. Only WC13 is provided with the interface. However, we facilitate how to run such applications. Notice that the WC13 configuration and input files are located in roms-jedi/test/Applications/wc13. The template input YAML files are located roms-jedi/test/templates. We use a Perl script to create the application input YAML files from the templates. Please check in roms-jedi/test/Applications/wc13/testinput.
For example, if running the U.S. East Coast DOPPIO application (240x104x40 grid points) or any other, please follow the following steps to configure ROMS-JEDI:
mkdir Bundle_doppio
cp bundle/.gitignore bundle/CMakeLists.txt Bundle_doppio
mkdir build_doppio
cd build_doppio
ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DROMS_APP="DOPPIO" \
-DROMS_APP_DIR="DOPPIOpath" -DCMAKE_BUILD_TYPE=Release ../Bundle_doppio
make -j 10
cd roms-jedi !> sub-directory: <interface_dir>/build_doppio/roms-jedi
ctest -N !> lists all the Unit Tests available
ctest -E -V get_ !> runs all the Unit Test
cd test/Data !> sub-directory: <interface_dir>/build_doppio/roms-doppio/test/Data
!> to check the results in various sub-directories
