Custom Query (964 matches)

Filters
 
Or
 
  
 
Columns

Show under each result:


Results (649 - 651 of 964)

Ticket Owner Reporter Resolution Summary
#781 arango Done Miscelaneous Update
Description

This update includes several improvements to ROMS code:

  • Several ROMS array allocation routines are called from read_phypar after several of the dimension parameters values are known after processed from the standard input script ocean.in. It has come to my attention that some users repeat such important parameters (like Ngrids and NtileJ) without commenting on (!) the previous assignment. The entries in ocean.in can be repeated, but the user needs to be aware the ROMS read each value and retains the last value processed. Anyway, a safeguard is coded in the following routines located in their associated modules:
    • allocate_param (mod_param.F)
    • allocate_parallel (mod_parallel.F)
    • allocate_iounits (mod_iounits.F)
    • allocate_stepping (mod_stepping.F)
    • allocate_fourdvar (mod_fourdvar.F)
    • allocate_scalars (mod_scalars.F)
    • allocate_ncparam (mod_ncparam.F)

to suppress allocation errors because variables were already allocated. Each variable in the above modules are tested for allocation:

      IF (.not.allocated(VAR)) THEN
        allocate ( VAR(...) )
      END IF

Warning: If the entries for Ngrids and NtileJ are repeated, ROMS will use the first processed value to allocate variables in the above modules. Therefore, the user needs to be careful when setting their values in ocean.in.

  • Routine read_phypar.F now use the I0 formatted statement for an nteger in a couple of places:
  70  FORMAT (/,' Resolution, Grid ',i2.2,': ',i0,'x',i0,'x',i0,        &
     &        ',',2x,'Parallel Nodes: ',i0,',',2x,'Tiling: ',i0,        &
     &        'x',i0)

  90  FORMAT (/,' Resolution, Grid ',i2.2,': ',i0,'x',i0,'x',i0,        &
     &        ',',2x,'Parallel Threads: ',i0,',',2x,'Tiling: ',i0,      &
     &        'x',i0)

to avoid overflow error in the integer format. The Iw.m descriptor with w=0 is allowed in the Fortran 2003 Standard, and it is supported by all modern compilers. The I0.m edit descriptor produces a reult with at leas m digits, and as many more digits as are necessary to represent th integer value.

  • Increased the integer format in timers.F to avoid overflow of large integers in the number of CPUs used:
  10       FORMAT (a,i5,a,f12.3)
  • The Proleptic Calendar attribute is renamed in dateclock.F to it is standard CF compliance value so it can be used by third-party software:
            calendar='proleptic_gregorian'
    
    Many thanks to Ivica Janekovic for bringing it to my attention.
#782 arango Done VERY IMPORTANT: Time units attribute in NetCDF files
Description

This change is a significant improvement, and I recommend users to update their code if have a version newer than Jun 21, 2018.

In update src:ticket:771, I made changes on how the reference time is processed in ROMS. The time attribute in input NetCDF is essential in the processing of external data. It usually has the form of:

  'time-units since YYYY-MM-DD hh:mm:ss'
  'time-units since YYYY-MM-DD hh:mm:ss.s'
  'time-units since YYYY-MM-DD hh:mm:ss.ss'

But the several variants that are still CF compliant. For example, one can have:

   'days since 1900-01-01 00:00:00'
   'seconds since 1968-05-23 12:00:00 -6'
   'years since -4713-11-24 00:00:00'        (Nov 24, 4713 BC)
   'hours since 1990-1-1 0:0:0'
   'days since 1582-10-15 00:00:0.000Z'
   'days since 1-1-1 0:0:0'
   'hours since 2010-12-1 12:5:30.5'
   'days since 1900/01/01 00:00:00' 
   'seconds since 1960-1-1'
   'days since 1-07-15 0:0:0'
   'days since 0000-01-01 0:0:0'

As you can see, people creating NetCDF files are not consistent in the format for YYYY-MM-DD hh:mm:ss. Therefore, I added a generic routine time_units to the module dateclock.F.

I am using the ASCII character set to decode the time units attribute string and replace unneeded characters with a blank space, CHAR(32). Only the following characters are retained:

!    Char  Dec  Control Action
!    ------------------------------
!    SP    32   Space
!    +     43   Plus
!    -     45   Hyphen, dash, minus
!    .     46   Period
!    0     48   Zero
!    1     49   One
!    2     50   Two
!    3     51   Three
!    4     52   Four
!    5     53   Five
!    6     54   Six
!    7     55   Seven
!    8     56   Eight
!    9     57   Nine
!
      DO i=1,lstr
        Schar=ICHAR(Tstring(i:i))
        IF (.not.(((48.le.Schar).and.(Schar.le.57)).or.                 &
     &            (Schar.eq.32).or.(Schar.eq.46))) THEN
           Tstring(i:i)=CHAR(32)                          ! blank space
        END IF
      END DO
      Tstring=ADJUSTL(TRIM(Tstring))
      lstr=LEN_TRIM(Tstring)

Then, every numerical value in the string is converted to a real variable to allow floating-point values for seconds. The year, month, day, hour, and minutes are output as integers.

Many thanks to John Wilkin for bringing this issue to my attention. If the time attribute in your input file is not in the form of time-units since YYYY-MM-DD hh:mm:ss, you will get an error when calling function netcdf_get_time.


I also updated function decode_line in inp_par.F to remove control-keys introduced when generating or editing input script ocean.in. It now checks and replaces ASCII characters CHAR(0) to CHAR(31) with a blank space:

!  Char  Dec  Key  Control Action
!  ----------------------------------------------------------------------
!  NUL   0    ^@   Null character
!  SOH   1    ^A   Start of heading, = console interrupt
!  STX   2    ^B   Start of text, maintenance mode on HP console
!  ETX   3    ^C   End of text
!  EOT   4    ^D   End of transmission, not the same as ETB
!  ENQ   5    ^E   Enquiry, goes with ACK; old HP flow control
!  ACK   6    ^F   Acknowledge, clears ENQ logon hand
!  BEL   7    ^G   Bell, rings the bell...
!  BS    8    ^H   Backspace, works on HP terminals/computers
!  HT    9    ^I   Horizontal tab, move to next tab stop
!  LF    10   ^J   Line Feed
!  VT    11   ^K   Vertical tab
!  FF    12   ^L   Form Feed, page eject
!  CR    13   ^M   Carriage Return
!  SO    14   ^N   Shift Out, alternate character set
!  SI    15   ^O   Shift In, resume default character set
!  DLE   16   ^P   Data link escape
!  DC1   17   ^Q   XON, with XOFF to pause listings; ":okay to send".
!  DC2   18   ^R   Device control 2, block-mode flow control
!  DC3   19   ^S   XOFF, with XON is TERM=18 flow control
!  DC4   20   ^T   Device control 4
!  NAK   21   ^U   Negative acknowledge
!  SYN   22   ^V   Synchronous idle
!  ETB   23   ^W   End transmission block, not the same as EOT
!  CAN   24   ^X   Cancel line, MPE echoes !!!
!  EM    25   ^Y   End of medium, Control-Y interrupt
!  SUB   26   ^Z   Substitute
!  ESC   27   ^[   Escape, next character is not echoed
!  FS    28   ^\   File separator
!  GS    29   ^]   Group separator
!  RS    30   ^^   Record separator, block-mode terminator
!  US    31   ^_   Unit separator
!
!  SP    32        Space
!
      inpline=TRIM(ADJUSTL(line_text))
      Linp=LEN_TRIM(inpline)
      DO i=1,LEN_TRIM(inpline)
        j=ICHAR(inpline(i:i))
        IF (j.lt.32) THEN
          inpline(i:i)=char(32)                           ! blank space
        END IF
      END DO
      inpline=TRIM(inpline)
#783 arango Done VERY IMPORTANT: ROMS Dynamic, Automatic, and Static Memory Requirements
Description

Currently, ROMS uses primarily dynamic and automatic memory which is allocated at running time. It uses small static memory allocation at compile time.

The dynamical memory is that associated with the ocean state arrays, and it is allocated at runtime, and it is persistent until the ROMS termination of the execution.

The automatic arrays appear in subroutines and functions for temporary local computations. They are created on entry to the subroutine for intermediate computations and disappear on exit. The automatic arrays (meaning non-static) are either allocated on heap or stack memory. If using the ifort compiler, the option -heap-arrays directs the compiler to put automatic arrays on the heap instead of the stack. However, it may affect performance by slowing down the computations. If using stack memory, the application needs to have enough to avoid weird segmentation faults during execution. In Linux operating systems, unlimited stack memory is possible by setting:

  ulimit -s unlimited              in your .bashrc
  limit stacksize unlimited        in your .cshrc, .tcshrc

The static arrays are allocated at compilation time and the memory reserved can be neither increased or decreased. Only a few static arrays are used in ROMS and mainly needed for I/O processing in the mod_netcdf routines.

In serial and shared-memory (OpenMP) applications, the dynamic memory associated with the ocean state is for full, global variables. Contrarily, in distributed-memory (MPI) applications, the dynamical memory related to the ocean state is for the smaller tiled arrays with global indices. Recall that the tiling in only done in the horizontal I- and J-dimensions and not in the vertical dimension.

Mostly all the ocean state arrays are dereferenced pointers and are allocated after processing ROMS standard input parameters. Recall that arrays represent a continuous linear sequence of memory. The pointer indicates the beginning of the state variable in the memory block.

ROMS is updated to compute an estimate of the dynamic and automatic memory requirements needed to run an application. The automatic memory is difficult to estimate since it is volatile. The maximum automatic memory is computed by looking at step2d.F, step3d_t.F, and I/O routines. Check mod_arrays.F to see how it is done. Also, information is provided in ROMS/memory.txt.


We can use the memory requirements to optimize partitions in parallel computers by examining the memory available for each Persistent Execution Thread (PET) or CPU. We need to make sure that the memory required by each distributed-memory tile fits on the PET to accelerate computations and optimize the computer resources.

The memory requirements are written to the standard output file after the activated CPP options report. For example, for three grids nested application on four distributed-memory PETs, we get:

 Process Information:

 Node #    0 (pid=    7420) is active.
 Node #    3 (pid=    7423) is active.
 Node #    1 (pid=    7421) is active.
 Node #    2 (pid=    7422) is active.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 Dynamic and Automatic memory (MB) usage for Grid 01:  240x104x40  tiling: 2x2

     tile          Dynamic        Automatic            USAGE

        0           145.33            16.94           162.27
        1           146.47            16.94           163.42
        2           147.88            16.94           164.83
        3           149.05            16.94           165.99

      SUM           588.73            67.78           656.51

 Dynamic and Automatic memory (MB) usage for Grid 02:  204x216x40  tiling: 2x2

     tile          Dynamic        Automatic            USAGE

        0           217.32            29.46           246.78
        1           215.32            29.46           244.78
        2           215.43            29.46           244.89
        3           213.45            29.46           242.91

      SUM           861.52           117.84           979.36

 Dynamic and Automatic memory (MB) usage for Grid 03:  276x252x40  tiling: 2x2

     tile          Dynamic        Automatic            USAGE

        0           334.33            46.32           380.66
        1           332.02            46.32           378.34
        2           331.81            46.32           378.13
        3           329.51            46.32           375.83

      SUM          1327.67           185.29          1512.95

    TOTAL          2777.92           370.90          3148.82

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Notice that the information is provided in decimal megabytes (MB) for each PET (tile). The USAGE column is the sum of dynamic and automatic memory requirements for each PET (tile). The report is also done for each nested grid. The TOTAL row provides memory requirements for all tree nested grids. Its value is a little underestimated. It will give you a guideline of what amounts to use in supercomputer queueing batch jobs. This application needs around 3.5 GB if we want a nice rounded number.

In a shared-memory 2x2 partitions for the UPWELLING test case with the BIO_FENNEL ecosystem model, we get:

 Process Information:

 Thread #    3 (pid=   70227) is active.
 Thread #    0 (pid=   70227) is active.
 Thread #    1 (pid=   70227) is active.
 Thread #    2 (pid=   70227) is active.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 Dynamic and Automatic memory (MB) usage for Grid 01:  41x80x16  tiling: 2x2

     tile          Dynamic        Automatic            USAGE

        0           216.11             2.93           219.04
        1             0.00             2.81             2.81
        2             0.00             2.93             2.93
        3             0.00             2.81             2.81

    TOTAL           216.11            11.48           227.59

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Since it is a shared-memory application, the dynamic memory requirements are reported only for PET (tile) zero.

Identical values are obtained in a serial 2x2 partitions:

 Process Information:

 Thread #    0 (pid=   36227) is active.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 Dynamic and Automatic memory (MB) usage for Grid 01:  41x80x16  tiling: 2x2

     tile          Dynamic        Automatic            USAGE

        0           216.11             2.93           219.04
        1             0.00             2.81             2.81
        2             0.00             2.93             2.93
        3             0.00             2.81             2.81

    TOTAL           216.11            11.48           227.59

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

and similar values in a serial 1x1 partitions:

 Process Information:

 Thread #    0 (pid=   18037) is active.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 Dynamic and Automatic memory (MB) usage for Grid 01:  41x80x16  tiling: 1x1

     tile          Dynamic        Automatic            USAGE

        0           216.11             9.75           225.86

    TOTAL           216.11             9.75           225.86

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

WARNING:

The memory requirement values are reported in the International System (SI) Units for megabyte (MB):

In ROMS:

   one r4 array element = 32 bits = 4 bytes    (single-precission)
   one r8 array element = 64 bits = 8 bytes    (double-precission)

In the metric decimal system (SI):

   1 byte               8 bits

   1 kilobyte (kB)      1E+3  bytes (1000)
   1 megabyte (MB)      1E+6  bytes (1000^2)
   1 gigabyte (GB)      1E+9  bytes (1000^3)
   1 terabyte (TB)      1E+12 bytes (1000^4)
   1 petabyte (PB)      1E+15 bytes (1000^5)

In the binary system (deprecated):

   1 kibibyte (KiB)     1024              bytes (2^10)
   1 mebibyte (MiB)     1,048,576         bytes (2^20, 1024^2)
   1 gibibyte (GiB)     1,073,741,834     bytes (2^30, 1024^3)
   1 tebibyte (TiB)     1,099,511,627,776 bytes (2^40, 1024^4)

Batch Modify
Note: See TracBatchModify for help on using batch modify.
Note: See TracQuery for help on using queries.