Changes between Version 2 and Version 3 of Ticket #884


Ignore:
Timestamp:
05/10/21 21:10:24 (3 years ago)
Author:
arango
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #884

    • Property Component Nonlinear → Parallelism
    • Property Resolution → Done
    • Property Status new → closed

  • Ticket #884 – Description

    v2 v3  
    2020In parallel I/O, writing is usually a more frequent and more complicated operation than reading.  Generally, there are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037 Preprint]):
    2121
    22  1. Single file, single writer: serial I/O. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries.
    23 
    24  2. Single file, multiple writers: parallel I/O with each tile writing its data to a single file. In ROMS, this capability is achieved by activating '''PARALLEL_IO''' and '''HDF5'''. It is only possible with the '''NetCDF4/HDF5''' libraries.
    25 
    26  3. Single file, collective writers: Parallel I/O with either one or a subset of processes performing I/O operations. The I/O operations can be synchronous or asynchronous. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when '''PIO_LIB''' is activated.
    27 
    28  4. Multiple files, multiple writers: Each tile writes its own file, but post-processing is required. Currently, this capability is not available in ROMS but can be easily implemented with the changes introduced in this update.
     22 1. [[span(style=color: blue, Single file, single writer:)]] serial I/O. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries.
     23
     24 2. [[span(style=color: blue, Single file, multiple writers:)]] parallel I/O with each tile writing its data to a single file. In ROMS, this capability is achieved by activating '''PARALLEL_IO''' and '''HDF5'''. It is only possible with the '''NetCDF4/HDF5''' libraries.
     25
     26 3. [[span(style=color: blue, Single file, collective writers:)]] Parallel I/O with either one or a subset of processes performing I/O operations. The I/O operations can be synchronous or asynchronous. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when '''PIO_LIB''' is activated.
     27
     28 4. [[span(style=color: blue, Multiple files, multiple writers:)]] Each tile writes its own file, but post-processing is required. Currently, this capability is not available in ROMS, but it can be easily implemented with the changes introduced in this update.
    2929
    3030----
    … …  
    3232= __'''What Is New:'''__ =
    3333
    34 The changes to ROMS are numerous but subtle, and 285 files were changed to include additional code.
    35 
    36 * All the standard input file '''roms.in''' was modified to include all the parameters necessary for Parallel I/O:
     34The changes to ROMS are numerous but subtle, and around 285 files were changed to include additional code related to the '''PIO''' implementation.
     35
     36* Several files were renamed for uniqueness in coupled systems. The '''ocean''' descriptor is renamed to '''roms''':
     37 
     38 {{{#!rst
     39               OLD                               NEW
     40-----------------------------------  --------------------------------
     41Master/ocean.h                       Master/roms.h
     42Master/ocean_control.F               Master/roms_kernel.F
     43ROMS/Drivers/ad_ocean.h              ROMS/Drivers/ad_roms.h
     44ROMS/Drivers/adsen_ocean.h           ROMS/Drivers/adsen_roms.h
     45ROMS/Drivers/afte_ocean.h            ROMS/Drivers/afte_roms.h
     46ROMS/Drivers/fsv_ocean.h             ROMS/Drivers/fsv_roms.h
     47ROMS/Drivers/fte_ocean.h             ROMS/Drivers/fte_roms.h
     48ROMS/Drivers/hessian_op_ocean.h      ROMS/Drivers/hessian_op_roms.h
     49ROMS/Drivers/hessian_so_ocean.h      ROMS/Drivers/hessian_so_roms.h
     50ROMS/Drivers/i4dvar_ocean.h          ROMS/Drivers/i4dvar_roms.h
     51ROMS/Drivers/nl_ocean.h              ROMS/Drivers/nl_roms.h
     52ROMS/Drivers/op_ocean.h              ROMS/Drivers/op_roms.h
     53ROMS/Drivers/optobs_ocean.h          ROMS/Drivers/optobs_roms.h
     54ROMS/Drivers/pert_ocean.h            ROMS/Drivers/pert_roms.h
     55ROMS/Drivers/picard_ocean.h          ROMS/Drivers/picard_roms.h
     56ROMS/Drivers/r4dvar_ocean.h          ROMS/Drivers/r4dvar_roms.h
     57ROMS/Drivers/rbl4dvar_ocean.h        ROMS/Drivers/rbl4dvar_roms.h
     58ROMS/Drivers/rp_ocean.h              ROMS/Drivers/rp_roms.h
     59ROMS/Drivers/so_ocean.h              ROMS/Drivers/so_roms.h
     60ROMS/Drivers/so_semi_ocean.h         ROMS/Drivers/so_semi_roms.h
     61ROMS/Drivers/sp4dvar_ocean.h         ROMS/Drivers/sp4dvar_roms.h
     62ROMS/Drivers/split_i4dvar_ocean.h    ROMS/Drivers/split_i4dvar_roms.h
     63ROMS/Drivers/split_r4dvar_ocean.h    ROMS/Drivers/split_r4dvar_roms.h
     64ROMS/Drivers/split_rbl4dvar_ocean.h  ROMS/Driver/split_rbl4dvar_roms.h
     65ROMS/Drivers/split_sp4dvar_ocean.h   ROMS/Drivers/split_sp4dvar_roms.h
     66ROMS/Drivers/tl_ocean.h              ROMS/Drivers/tl_roms.h
     67ROMS/Drivers/tl_r4dvar_ocean.h       ROMS/Drivers/tl_r4dvar_roms.h
     68ROMS/Drivers/tl_rbl4dvar_ocean.h     ROMS/Drivers/tl_rbl4dvar_roms.h
     69ROMS/Drivers/tlcheck_ocean.h         ROMS/Drivers/tlcheck_roms.h
     70}}}
     71
     72* Added the following include files for '''Biology''' and '''Sediment''':[[BR]] [[BR]]
     73 * [[span(style=color: blue, ROMS/Nonlinear/Biology/ecosim_def_pio.h)]]
     74 * [[span(style=color: blue, ROMS/Nonlinear/Biology/ecosim_wrt_pio.h)]]
     75 * [[span(style=color: blue, ROMS/Nonlinear/Biology/fennel_def_pio.h)]]
     76 * [[span(style=color: blue, ROMS/Nonlinear/Biology/fennel_wrt_pio.h)]]
     77 * [[span(style=color: blue, ROMS/Nonlinear/Biology/hypoxia_srm_def_pio.h)]]
     78 * [[span(style=color: blue, ROMS/Nonlinear/Biology/hypoxia_srm_wrt_pio.h)]]
     79 * [[span(style=color: blue, ROMS/Nonlinear/Biology/nemuro_def_pio.h)]]
     80 * [[span(style=color: blue, ROMS/Nonlinear/Biology/nemuro_wrt_pio.h)]]
     81 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_Franks_def_pio.h)]]
     82 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_Franks_wrt_pio.h)]]
     83 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_Powell_def_pio.h)]]
     84 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_Powell_wrt_pio.h)]]
     85 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_iron_def_pio.h)]]
     86 * [[span(style=color: blue, ROMS/Nonlinear/Biology/npzd_iron_wrt_pio.h)]]
     87 * [[span(style=color: blue, ROMS/Nonlinear/Biology/oyster_floats_def_pio.h)]]
     88 * [[span(style=color: blue, ROMS/Nonlinear/Biology/oyster_floats_wrt_pio.h)]]
     89 * [[span(style=color: blue, ROMS/Nonlinear/Biology/red_tide_def_pio.h)]]
     90 * [[span(style=color: blue, ROMS/Nonlinear/Biology/red_tide_wrt_pio.h)]]
     91 * [[span(style=color: blue, ROMS/Nonlinear/Sediment/sediment_def_pio.h)]]
     92 * [[span(style=color: blue, ROMS/Nonlinear/Sediment/sediment_wrt_pio.h)]]
     93
     94* The standard input file '''roms.in''' was modified to include all the parameters necessary for Parallel I/O:
    3795 {{{
    3896! Input and Output files processing library to use:
    … …  
    101159 Check the '''GLOSSARY''' for the detailed description of these parameters. Notice the ROMS supports the Standard NetCDF library and the Parallel I/O at the same time. The user choose the library to use at run time for input ('''INP_LIB''') and output ('''OUT_LIB''') files. All these parameters are documented [https://www.myroms.org/wiki/roms.in in the roms.in wiki page].
    102160
    103  * Added module '''set_pio.F''' to initialize and configure Parallel I/O. And the module '''mod_pio_netcdf.F''' including all the interfaces for same routine overloading similar to the '''mod_netcdf.F''' for the standard NetCDF library.
    104 
    105  * The '''T_IO''' structure in '''mod_param.F''' was modified to include new fields to include file IO type, file descriptor, and variables descriptors.
    106 
    107  * The changes to the I/O routines are now modules with the following design:
     161 * Added module '''set_pio.F''' to initialize and configure Parallel I/O. And the module '''mod_pio_netcdf.F''' including all the interfaces for same name routine overloading similar to the '''mod_netcdf.F''' for the standard NetCDF library.
     162
     163 * The '''T_IO''' structure in '''mod_param.F''' was modified to include new fields for file IO type ('''IOtype'''), file descriptor ('''pioFile'''), and variables descriptors ('''pioVar(:)''' and '''pioTrc(:)''').
     164
     165 * The changes to all I/O routines (now modules) have the following design: (history file example)
    108166 {{{
    109167!
    … …  
    149207 So the subroutine '''def_his_nf90''' has calls to the Standard NetCDF Fortran90 API. Contrarily, the subroutine '''def_his_pio''' call the PIO Fortran90 API.
    150208
    151  Similarly, the writing routines have the same design:
     209 Similarly, all the writing routines (now modules) have the following design:
    152210 {{{
    153211!
    … …  
    213271 * All readers ('''nf_fread*.F''') and writers ('''nf_fwrite*.F''') were enhanced to include the '''PIO''' reading and writing subroutines.
    214272
     273 * Added a new '''bitsum''' subroutine to '''get_hash.F'''.  It is now the default method to compute the checksum. The floating-point data is converted to integers to facilitate the invariant order of the sum in tiled parallel applications (Hallberg and Adcroft, 2014; doi:10.1016/j.parco.2014.04.007). It allows MPI reduction operations for I/O processed with the '''PIO''' library.
     274
    215275----
    216276