| 9 | | An **ESM/build_ufs.sh** is provided to facilitate configuring and compiling any ROMS application. It is currently, configured for the Hurricane Irene (**IRENE**) application. See [https://github.com/myroms/roms_test/blob/feature/ufs/IRENE/Coupling/roms_data_cmeps/Readme.md roms_test] for more information. It works for the **ROMS-DATA** coupling system using the **CDEFS** connectors or the **CMEPS** mediator. The **DATA** component is replacing the atmosphere model, and it tests **ROMS** connection to **CDEPS**, **CMEPS**, and **UFS** coupling driver. |
| | 9 | An **ESM/build_ufs.sh** is provided to facilitate configuring and compiling any ROMS application. It is currently configured for the Hurricane Irene (**IRENE**) application. See [https://github.com/myroms/roms_test/blob/feature/ufs/IRENE/Coupling/roms_data_cmeps/Readme.md roms_test] for more information. It works for the **ROMS-DATA** coupling system using the **CDEFS** connectors or the **CMEPS** mediator. The **DATA** component is replacing the atmosphere model, and it tests **ROMS** connection to **CDEPS**, **CMEPS**, and **UFS** coupling driver. |
| | 10 | |
| | 11 | ---- |
| | 12 | |
| | 13 | === [[span(style=color: #0000FF,LAPACK/ARPACK Legacy Library:)]] === |
| | 14 | |
| | 15 | The **ARPACK** and its MPI-based parallel (**PARPACK**) version are well-tested libraries to solve large eigenvalue matrix problems. Our primary use in ROMS is to use its Implicit Restarted Arnoldi Method (**IRAM**) to solve for the Ritz eigenfunctions in several of our Tangent and Adjoint-based **propagators** that are part of the Generalized Linear Stability (**GLS**) package described in Moore ''et al.'' (2004) Ocean Modelling paper https://doi.org/10.1016/j.ocemod.2003.11.001. As far as I know, we cannot use a modern replacement library. Therefore, we will continue using that library when running any of the drivers of the **GLS** package. |
| | 16 | |
| | 17 | However, the **ARPACK** library includes selected Linear Algebra functions from the **BLAS** and **LAPACK** libraries. In particular, we use **LAPACK**'s **DSTEQR** function to solve for the eigenvalues/eigenvectors of the tri-diagonal system used in the various 4D-Var minimizations available in ROMS. |
| | 18 | |
| | 19 | Thus, I spent some time looking for a replacement. It turns out that any modern software out there is built on top of **LAPACK** (Phyton, MATLAB, R, !SciPy, and others). There are also bindings to **C** and **C++**. We could use the **eigen**, written in **C++**, but it will be complicated since I will have to write and test the inter-operability mechanism in ROMS for **C++** to **Fortran 2003** and vice versa (**Fortran 2003** to **C++**). It turns out that the **libeigen** has **BLAS** and **LAPACK** components. See: https://gitlab.com/libeigen/eigen. On top of that, there are reports that the **eigen** library is much slower than Fortran libraries. Thus, I am giving up on the **eigen** library. It is beyond our scope. |
| | 20 | |
| | 21 | I decided to adapt the functions we need from **LAPACK** into module **lapack_mod.F**, which I converted to **Fortran 2003**. It includes a total of 19 **LAPACK** functions. I modernized the code, which had initially 41 **GOTO**s. I managed to remove 18 of them. The remaining 23 are located in the function **DSTEQR**. I was able to get identical 4D-Var solutions during testing. That eigenfunction solver is tricky, and removing the remaining **GOTO**s is dangerous. |
| | 22 | |
| | 23 | The complicated **GOTO**s are still in selected functions but can be removed from the final C-preprocessing file (**lapack_mod.f90**) by activating the CPP option **REMOVE_LAPACK_GOTOS**. For example, in the function **DLASRT** used to sort the eigenvalues, I have: |
| | 24 | |
| | 25 | {{{ |
| | 26 | #ifdef REMOVE_LAPACK_GOTOS |
| | 27 | DecLOOP_3 : DO ! forever LOOP 3 |
| | 28 | DecLOOP_1 : DO ! forever LOOP 1 |
| | 29 | J = J - 1 |
| | 30 | IF (D(J) .GE. DMNMX) EXIT DecLOOP_1 ! terminate LOOP 1 |
| | 31 | END DO DecLOOP_1 |
| | 32 | ! |
| | 33 | DecLOOP_2 : DO ! forever LOOP 2 |
| | 34 | I = I + 1 |
| | 35 | IF (D(I) .LE. DMNMX) EXIT DecLOOP_2 ! terminate LOOP 2 |
| | 36 | END DO DecLOOP_2 |
| | 37 | ! |
| | 38 | IF (I .LT. J) THEN |
| | 39 | TMP = D(I) |
| | 40 | D(I) = D(J) |
| | 41 | D(J) = TMP |
| | 42 | CYCLE DecLOOP_3 ! iterate LOOP 3 |
| | 43 | ELSE |
| | 44 | EXIT DecLOOP_3 ! terminate LOOP 3 |
| | 45 | END IF |
| | 46 | END DO DecLOOP_3 |
| | 47 | #else |
| | 48 | 60 CONTINUE |
| | 49 | 70 CONTINUE |
| | 50 | J = J - 1 |
| | 51 | IF (D(J) .LT. DMNMX) GO TO 70 |
| | 52 | 80 CONTINUE |
| | 53 | I = I + 1 |
| | 54 | IF (D(I) .GT. DMNMX) GO TO 80 |
| | 55 | IF (I .LT. J) THEN |
| | 56 | TMP = D(I) |
| | 57 | D(I) = D(J) |
| | 58 | D(J) = TMP |
| | 59 | GO TO 60 |
| | 60 | END IF |
| | 61 | #endif |
| | 62 | }}} |
| | 63 | |
| | 64 | Notice that the above piece of code is slightly complicated. It requires knowledge of the modern Fortran Standard and Objective-Oriented Programming principles to remove the **GOTO**s, which were unavailable in Fortran and compilers over 40 years ago. This doesn't imply that the **LAPACK** library is incorrect and produces the wrong results. I am keeping the original code with the **GOTO**s for reference and testing. |
| | 65 | |
| | 66 | [[span(style=color: #FF0000, We no longer need the **LAPACK/ARPACK** libraries to run any 4D-Var drivers in ROMS because the functions that we need are available in our modern adaptation module **lapack_mod.F**)]]. |
