Frequently Asked Questions
Installation and Configuration
My build finished with no errors. Where is the ROMS executable?
It is either in oceanS (serial), oceanO (shared-memory, OpenMP), oceanM (distributed-memory, MPI), or oceanG (debug). Check the makefile to see which options are on. The makefile definition BINDIR controls where to write the compiled ROMS executable.
What do I have to do to run an application?
There are basically two modes to run a ROMS application: serial or parallel
- Mostly all canned applications that come with the ROMS distribution do not need input NetCDF files. The grid, initial conditions and forcing conditions are specified with analytical expressions in Functionals. Any of these test cases can be run by editing the ROMS_APPLICATION definition in the makefile or the build Script. A list of all pre-defined model applications can be found in header file cppdefs.h. The next step is to compile and link using the make tool and, if you are lucky, the program will build. It is good practice to execute make clean first. Then, the application should be run in serial with the command: oceanS < ocean_APPLICATION.in > & log &where APPLICATION is the lowercase of any of the CPP options defining a distributed test case. For example, the upwelling (UPWELLING) test case uses the input script ocean_upwelling.in which is located in the ROMS/External directory.
- A few canned applications that come with ROMS require input NetCDF files which are located in the Data/ROMS directories. Same as above, but if you want run in parallel on a distributed-memory computer use: mpirun -np 2 oceanM ocean_test_head.in > & log &or to run in parallel on a shared-memory machine:setenv OMP_THREAD_NUM 2for the test headland case, for example. You need to be sure that the product of NtileI and NtileJ in ocean_test_head.in is 2 since the above command specifies two processors to run (-np 2 or OMP_THREAD_NUM=2). Notice that to compile the model in distributed-memory, you need to edit the makefile (or build Script) and activate USE_MPI. Also, if you use MPICH rather than a native MPI library, define USE_MPIF90 as well. The MPICH library uses a script called mpif90 for compiling.
oceanO < ROMS/External/ocean_upwelling.in > & log &
Why is only one node created when executing with mpirun?
This usually occurs when you are not using the correct mpirun. For Example, if you compiled with Open MPI and try to use the mpirun command from MPICH, you will see this error. To see which mpirun you are executing type 'which mpirun'; the first part of the path should match what the ROMS output reports as your 'Compiler command'.
/opt/pgisoft/mpich/bin/mpirun
> mpirun -np 2 ./oceanM ocean.in
Process Information:
Node # 0 (pid= 150433) is active.
Model Input Parameters: ROMS/TOMS version 3.4
. . .
Compiler system : pgi
Compiler command : /opt/pgisoft/openmpi/bin/mpif90
Compiler flags : -O3 -Mfree
. . .
ERROR: Illegal domain partition.
>
Notice that only one node is active and the compiler command shows the model was compiled with Open MPI but mpirun is from the MPICH directory. To fix this problem, update your path to use the mpirun from the openmpi directory or re-compile with MPICH.
Compiling and Linking
The compiling and linking information is all in Compiling and Linking, makefile, and build Script. Please let us know if any of this is unclear.
Errors at compile time
First of all, read the error messages from the compiler carefully to see if you can figure out what they are talking about. There are several classes of errors (add more as you see them):
- Errors from build.bash:
./build.bash: line 43: $'\r': command not found
./build.bash: line 46: syntax error near unexpected token '$'in\r'
./build.bash: line 46: ' case "$1" in
You need to run the dos2unix command on build.bash.
- Errors from cpp: If there is a mismatch in the #ifdef/#endif pairs, cpp will be unhappy and will tell you so. There is a perl script called "ifdefs" which checks your #ifdefs/#endif pairings.
- Errors like this:
/bin/sh: -c: not found
The command:
in the Compilers/xxx-xxx.mk file has failed to find the compiler you specified, setting FC to the null string. You need to find out which compiler you were asking it to look for and to make sure it is in your path. If the flag USE_MPIF90 is set, it will be looking for "mpif90". What do you get from the command "which mpif90"? If you are trying for a serial run first, what does "which gfortran" give you?
- Syntax errors in the fortran: check the intermediate .f90 file for the exact code at the line numbers in the report. Be sure to edit the corresponding .F file once you determine what the trouble is.
- Trouble with the netcdf module file: was it created with the same version of the same compiler? If not, there could be trouble. I've seen gfortran give a cryptic message about expecting a left parenthesis when faced with a module file from an older version of gfortran. Usually if it is from a completely different compiler, it will tell you so.
- Trouble with other modules: one instance of this is due to incorrect dependency information in MakeDepend. The ROMS code must be compiled in the correct order and failure to do so might give something like:
[MOD_PARAM]
USE mod_param
----------^
fortcom: Error: bc_2d.f90, line 32: Error in opening the Library module file.
Here, it failed to compile mod_param.F before bc_2d.F, leading to trouble. The '--objdir=$(SCRATCH_DIR)' option in MDEPFLAGS is not optional.
Errors at link time
- One common problem is that differing options were used when compiling libraries vs. the ROMS code proper. For instance, you get a message about:crash.f90:(.text+0x19): undefined reference to 'nf_close__'when you know you linked to the netcdf library. You can query the library to see what it has:nm libnetcdf.a | grep nf_closeClearly, there's a mismatch in the number of underscores which is preventing it from linking successfully. See if you can find a fortran option for controlling the number of underscores and try to match how the library was compiled. Otherwise, you can recompile the library to match how you compile ROMS.
0000000000000e60 T nf_close_
U nf_close_
- Another recent problem is that NetCDF 4.x sometimes creates a libnetcdff.a in addition to the libnetcdf.a. The link statement needs to include both. See this thread for a discussion on it.
- Also new in NetCDF is the option to create a library with OpenDAP capabilities. In that case, you need to also link to libcurl, in the makefile as the option USE_DAP. This option is needed if you are missing functions with "curl" in the name.
Basic Usage
What are the ways that ROMS can receive the wind and other surface forcing?
- One option is to provide the stresses and fluxes via analytic expressions in ana_smflux.h and ana_stflux.h.
- One option is to provide analytic expressions for the winds and other atmospheric fields in ana_winds.h and friends, and define BULK_FLUXES.
- ROMS can also read fields from NetCDF files, also stresses or winds, depending on the state of BULK_FLUXES. These can be provided as:
- Point measurements, in which ROMS will rotate to the correct directions from East (u) and North (v).
- Gridded fields on the ROMS grid, in which case the directions have to match the directions on your grid.
- Gridded fields on a larger, usually coarser grid. These are in East and North, with ROMS doing the rotation. ROMS picks up its cue on whether to interpolate+rotate or not based on the grid dimensions of these fields. If the dimensions exactly match that of your own grid, ROMS will assume that it is on your grid. If you plan on using global fields of dimension 180x90, say, don't build a grid with those dimensions. The cdl for a working example is shown here:
dimensions:
rain_time = UNLIMITED ; // (708 currently)
lat = 94 ;
lon = 192 ;
variables:
float rain(rain_time, lat, lon) ;
rain:_FillValue = -1.e+34f ;
rain:units = "Kg_meter-2_sec-1\000Kg_meter-2_sec-1" ;
rain:long_name = "rain fall rate" ;
rain:coordinates = "lon lat" ;
double lat(lat) ;
lat:units = "degrees_north" ;
lat:point_spacing = "uneven" ;
lat:axis = "Y" ;
double lon(lon) ;
lon:units = "degrees_east" ;
lon:modulo = 360. ;
lon:point_spacing = "even" ;
lon:axis = "X" ;
float SNOW(rain_time, lat, lon) ;
SNOW:missing_value = -1.e+34f ;
SNOW:_FillValue = -1.e+34f ;
SNOW:long_name = "SNOW_FRAC[D=2,GXYT=PRC_MOD@ASN]*PRC_MOD" ;
SNOW:long_name_mod = "L=1:12" ;
double rain_time(rain_time) ;
rain_time:units = "days since 1900-01-01 00:00:00" ;
rain_time:axis = "T" ;
rain_time:bounds = "TIME_bnds" ;
rain_time:time_origin = "1-JAN-1948" ;
rain_time:calendar = "LEAP" ;
In this example, the variable names "rain" and "SNOW" would have to match with what's in the varinfo.dat file.
How can I create the initial/boundary/forcing NetCDF files for ROMS?
- Please don't use the old Fortran codes unless you are a glutton for punishment.
- Most use the Matlab scripts on the ROMS svn server.
- There are Python tools at github. A thorough description of how to get these running on a Mac is here. An alternate way to go is to install a VirtualBox with Linux and install them there.
- There are probably Ferret and other scripts out there. Ask on the ROMS forum if you have question about a specific data source.
What are the ways that ROMS can receive tides on the boundaries?
- You can specify a time-dependent boundary condition file that temporally resolves the tides and skip the tides file entirely.
- You can specify SSH_TIDES and/or UV_TIDES and provide a tides file and skip the boundary condition file entirely - though this option might require you to tell ROMS in globaldefs.h that you really, really don't need an OBC_FILE. Usual practice is to provide both surface elevation and currents in the tidal file and to use FSCHAPMAN and M2FLATHER on the open sides of your domain.
- You can specify other 2-d currents and surface elevation signals as coming in, plus have tides and use the ADD_M2OBC and ADD_FSOBC options.
How can I set up passive tracers?
You need to #define T_PASSIVE to get passive tracers. The flag ANA_PASSIVE determines whether or not the initial conditions are analytic (from ana_passive.h) or from the initial condition file. As for boundary conditions, all tracers have either analytic boundary conditions or they all come from the same boundary condition file. If you need to read values from a file, the field names are dye_01, dye_02, etc. I successfully set up a uniform dye_01 using ANA_PASSIVE with boundary conditions from a file. The boundary file has u,v,T,S for many times and (analytic) biological values for the beginning and end times. I have a script to read one biology value, change the attributes and the value to my uniform value, then write it into the boundary file with the name dye_01_west, etc. I then had to edit varinfo.dat to reflect that it's using bio_time rather than ocean_time.
Algorithm Design
Why the _r8 at the end of real constants?
Some computers use 32 bits for single precision real numbers and 64 bits for double precision, and some use 64 and 128, respectively. For consistent results on various machines, ROMS takes advantage of the intrinsic F90/95 function SELECTED_REAL_KIND( ). See ROMS/Modules/mod_kinds.F. This allows you to associate an integer parameter with a specific data type...in this case r8 is associated with 64-bit precision. RTFM (Read the Fortran90 Manual), or better yet, chapter 11 in Chapman, 2004.
Why we cannot have zero bathymetry (h=0)?
In ROMS the the bathymetry, h, is positive and represents the vertical water column thickness (meters) from the ocean rest state (zeta=0). We cannot have h=0 in any horizontal grid point even if it is masked by land. Therefore, during grid generation the h values are limited to a minimum value hmin. Recall that in terrain-following vertical coordinates h is used to compute the vertical level thickness (Hz). If you examine the governing equations, we divide by Hz when computing the vertical viscosity and diffusion terms. In addition, in several places of the numerical kernel we also divide by Hz. This is usually computed in the internal array oHz=1/Hz. In mathematics, the division by zero is not defined and computers will return Inf or NaN. Users need to consider carefully what value of hmin to use by taking into account the horizontal grid resolution, number of vertical levels, and vertical coordinates stretching function for a particular application. In ROMS, all the vertical levels are located above hmin. This will limit the vertical time-step of the model due to the Currant-Friedrichs-Levy (CFL) condition. Notice that we also need a critical water column thickness (Dcrit) in wetting and drying for the same reasons mentioned above.
Input/Output NetCDF files
How to add a new variable to ROMS output files?
This question has been asked several times. ROMS IO design is very flexible and easy to expand. There are several steps that need to be followed to add new variables to any of the output files:
- The ROMS output NetCDF files are defined by any the routines with the prefix def_ in the ROMS/Utilities directory. For example, in def_his.F a variable can be defined in the history file by just adding:IF (Hout(idUvel,ng)) THENCorrespondingly, the field time records are written in routines with the prefix wrt_. For example, a field can be written in wrt_his.F as:
Vinfo( 1)=Vname(1,idUvel)
Vinfo( 2)=Vname(2,idUvel)
Vinfo( 3)=Vname(3,idUvel)
Vinfo(14)=Vname(4,idUvel)
Vinfo(16)=Vname(1,idtime)
# if defined WRITE_WATER && defined MASKING
Vinfo(20)='mask_u'
# endif
Vinfo(22)='coordinates'
Aval(5)=REAL(Iinfo(1,idUvel,ng),r8)
status=def_var(ncHISid(ng),hisVid(idUvel,ng),NF_FOUT, &
& nvd4,u3dgrd,Aval,Vinfo,ncname)
END IFIF (Hout(idUvel,ng)) THENAss you may noticed, some knowledge of ROMS internal structure is required to define a new field. Please follow any of output fields examples in such files to define and write the new field.
scale=1.0_r8
gtype=gfactor*u3dvar
status=nf_fwrite3d(ng, iNLM, ncHISid(ng), hisVid(idUvel,ng), &
& tHISindx(ng), gtype, &
& LBi, UBi, LBj, UBj, 1, N(ng), scale, &
# ifdef MASKING
& GRID(ng) % umask(LBi,LBj), &
# endif
& OCEAN(ng) % u(LBi,LBj,1,NOUT))
IF (status.ne.nf90_noerr) THEN
IF (Master) THEN
WRITE (stdout,10) TRIM(Vname(1,idUvel)), tHISindx(ng)
END IF
exit_flag=3
ioerror=status
RETURN
END IF - Notice that ROMS state variables are defined on a horizontal, staggered Arakawa C-grid. In addition, the variables are also staggered in the vertical. Therefore, you need choose the appropriate identifier flag:
- p2dvar: 2D field at ψ-points
- r2dvar: 2D field at ρ-points
- u2dvar: 2D field at u-points
- v2dvar: 2D field at v-points
- p3dvar: 3D field at ψ-points
- r3dvar: 3D field at ρ-points
- u3dvar: 3D field at u-points
- v3dvar: 3D field at v-points
- w3dvar: 3D field at w-points
- b3dvar: 3D bed-sediment field
- Select a unique six-character, case sensitive, field identifier which is defined in file ROMS/Modules/mod_ncparam.F. Notice that all the identifiers are defined in alphabetic order to facilitate the search. Choose a meaningfull identifier prefixed by id. This indentifier needs to be assigned in routine initialize_ncparam during the reading of variable metadata form file varinfo.dat. You need to add code similar to:CASE ('idUvel')Check the example above to see how the idUvel identifier is used in the internal field metadata arrays.
idUvel=varid - Define new field metadata in file ROMS/External/varinfo.dat. Follow the examples to define the following field information:
- Field variable name string. This is the variable name in the output NetCDF file. You need to choose a unique and meaninful variable name.
- Field long-name string. This is the variable long_name attribute in the output NetCDF file.
- Field units string. This is the variable units attribute in the output NetCDF file.
- Field type string. This is the variable field attribute in the output NetCDF file.
- Associated time-variable name string. This is the variable time attribute in the output NetCDF file.
- Field identifier string. This is the field six-character unique index used in information arrays.
- Field staggered C-grid variable type. Use one of flags defined above.
- Field scale floating-point value. This value is only used in input data to scale field to model units, if necessary. Otherwise, use a unity value.
Notice that all string information above is specified within single quotes to facilitate free-format reading in mod_ncparam.F. For example:'u' ! Input/Output
'u-momentum component'
'meter second-1' ! [m/s]
'u-velocity, scalar, series'
'ocean_time'
'idUvel'
'u3dvar'
1.0d0
- Add logical switch Hout(id....) in input script ocean.in to activate the processing of the new field. For example:Hout(idUvel) == T ! 3D U-velocityThis new switch needs to be read in file ROMS/Utility/inp_par.F, routine read_PhyPar. For example:ELSE IF (TRIM(KeyWord).eq.'Hout(idUvel)') THENIt is also a good idea to report the new logical switch to standard output. For example:
IF (idUvel.eq.0) THEN
WRITE (out,280) 'idUvel'
STOP
END IF
Npts=load_l(Nval, Cval, Ngrids, Hout(idUvel,1))IF (Hout(idUvel,ng)) WRITE (out,170) Hout(idUvel,ng), &
& 'Hout(idUvel)', &
& 'Write out 3D U-momentum component.' - Follow the same steps to process time-averaged fields. In addition, you need to define, allocate, and initialize in ROMS/Modules/mod_average.F the new ROMS variable used to accumulate the time records. For example:real(r8), pointer :: avgu3d(:,:,:)The time-averaged fields are computed in ROMS/Nonlinear/set_avg.F. Check this file to see how this is done.
...
allocate ( AVERAGE(ng) % avgu3d(LBi:UBi,LBj:UBj,N(ng)) )
...
AVERAGE(ng) % avgv3d(i,j,k) = IniVal
