Wet-points only ROMS

[lanerolle]

I have helped develop several realistic ROMS applications for Bays & Estuaries around the US and one of the things that makes me feel guilty about them is that in each of these applications, although only about 20% of the grid points are wet, ROMS computes the equations on all of the points without any discrimination (wet/dry) and hence expends a significant amount of unncessary effort. If ROMS could be made to compute only on wet points, there are many obvious benefits :

(1) significant computational efficiecy,
(2) affordability of much higher spatial resolution (ie.much larger grids) due to (1),
(3) the ability to compete with unstructure ocean models and,
(4) direct use of the wet-point only grids generated by say the DELFT3D grid generator.

In addition, structured models (orthogonal, curvilinear grids with controlled orthogonality errors - say less than 5-degrees) have better spatial numerical error properties than their equivalent unstructured grid counterparts.

I do realize that getting ROMS to compute on wet-points only will prove to be tricky due to its : (i) need to have 4 boundaries by design, (ii) higher order accurate, upstream-biased advection schemes and (ii) harmonic stress tensor based viscosity/diffusivity formulation both of which require extended numerical stencils. These will no doubt introduce complications.

Anyway, I was wondering whether there are plans to make ROMS run on wet-points only and/or whether there is a wet-point only version of ROMS developed by some group which is freely available for general use?

[mathieu]

I think computing only on wet points would lead to a big performance loss. You see the processors are pipelined and thus an if test wrongly predicted can lead to a several cycles lost. That is why you have

Code: Select all

zeta(i,j)=zeta_new(i,j)*rmask(i,j)

and not

Code: Select all

IF (rmask(i,j).eq.1) THEN
  zeta(i,j)=zeta_new(i,j)
END IF

So point (1) is I think not correct.

[arango]

Your assumptions are not correct. As Mathieu mentioned, this actually affects performance tremendously. We will loss pipelining and vectorization.

This argument is an old one. You always heard this from people in the unstructured modeling community to justify the advantage of their solution technique. As in any geophysical modeling technique, there are advantages and disadvantages in all the solution methodologies. The finite element technique has some advantages when discretizing complicate coastal applications with estuaries. However, these algorithms usually have low order numerics for stability. The solution may be wrongly affected by the topology of the grid in some applications. High order numerical kernels have obvious advantages. You just need to compare the solutions to see the differences in the circulations and conservation properties.

There are other things that will be affected by changing the ROMS kernel as you suggest. It will affect shared-memory and distributed-memory parallelism, nesting via composed, mosaic or refinement grids, tensors, matrix operations in sophisticated adjoint-based algorithms, numerical performance, and debugging is much complicated.

You will find out in the near future when the nesting version of ROMS is released, that you will be able to have multiple grids in an application using composed, mosaic, or refinement grids that minimize the use of excessive land/sea masking and variable resolution were is really needed. Just be patient. I am heavily working on this now. You will be amaze of the things that you will be able to do. I am removing all the CPP options for advection and boundary conditions. It probably it will take longer to compile few routines but the advantages are enormous because you can choose different options for different grids. The only disadvantage that I foresee is that we are going to need more sophisticated grid generation tools. We will address this when we get there.

The arguments that you present maybe relevant in the past with the available computer architecture and number of CPUs. Nowadays, this is not the case. In my opinion, I prefer high order and accurate kernels even if the solution takes a little longer. This is not a matter of competition of which model give you the solution faster but what model give you the more realistic and accurate solution.