Updated Matlab processing scripts for NetCDF

[arango]

I updated several Matlab scripts to allow processing of NetCDF file(s) on OpenDAP servers. Previously, all the Matlab scripts in the ROMS repository (​https://www.myroms.org/svn/src/matlab) used the MEXNC interface to Matlab. Currently, the processing of URL files (single or aggregated) on an OpenDAP server can be only processed with the Java interface of SNCTOOLS.

The following files were updated:

• utility/nc_dim.m: Added the capability to inquire about the dimensions in input URL NetCDF file(s).

• utility/nc_getatt.m: Added the capability to inquire about a specified global or variable attribute in input URL NetCDF file(s).

• utility/nc_inq.m: Added the capability to print the dimensions and variables available in input URL NetCDF file(s).

• utility/nc_read.m: Added the capability to read data from input URL NetCDF file(s). The function was updated to read correctly any type of numerical data (double, single, interger, short, and byte). It uses the new SNCTOOLS function nc_vargetr which processes the input data in its native (raw) data type. This is extremely important for post-processing the data correctly with _FillValue, missing_value, scale_factor, and add_offset attributes. The default SNCTOOLS function nc_varget converts all numeric data types to double precession before the post-processing is done. In my opinion this is incorrect when unpacking data. Sometimes, it also introduces round-off errors with surprising results. A more robust post-processing of the input data is introduced. For example, to find all fill or missing values, stored in spval, we compute:

     ind=find(abs(f-spval) < 4*eps(double(spval));
  

  Then, the scaling of byte/short data and subsequent fill/missing value replacement is, for example:

     f=double(f)*scale+offset;
     f(ind)=ReplaceValue;         % usually zero or NaN for ploting
  

  Therefore, the steps are as follows: i) read requested data in its native (raw) numerical data type, ii) search for fill or missing values if any, iii) unpack the data if the attributes scale_factor and/or add_offset are present, iv) replace fill/missing values with the provided replace value. Notice that the function call to nc_read is now:

     f=nc_read(Fname,Vname,Tindex,ReplaceValue,PreserveType);
  

  That is, a new optional fifth argument is added to nc_read.m to allow the user to preserve input numerical data type. The user may set PreseveType=true, so the native data type is returned. If this argument is omitted, the default is to convert numerical data to double precision. If the data is packed to store the most significant bits with scale_factor and add_offset, the returned data is in double precision.

• utility/nc_url.m: New function to inquire if the input NetCDF file name is a URL from an OpenDAP server.

• utility/nc_varinfo.m: New script returning a structure containing the variable information. This function was adapted and have similar capabilities to the SNCTOOLS nc_getvarinfo function, except that all numerical values are returned in its native data type. The original function nc_getvarinfo unwisely converted all the numerical values to double precision. This is problematic when dealing with the variable numerical attributes. Specially, when using the NetCDF attributes: _FillValue, missing_value, scale_factor, and add_offset. And to lesser extend when using the attributes: valid_min, valid_max, and valid_range.

• utility/nc_vinfo.m: added the capability of inquiring the variable dimensions and the C-grid type in input URL NetCDF file(s).

• utility/nc_vname.m: added the capability of inquiring all the variable names in input URL NetCDF file(s).

• utility/nc_write.m: Added the capability of writing byte and short type data. Updated the processing of the _FillValue attribute.

Notice that we can only read data from URL NetCDF file(s). We cannot write neither modify URL NetCDF file(s). We don't own them, have the privileges to modify them, and the Java interface does not allow it.

In order for nc_read.m to work correctly, the user need to update the SNCTOOL scripts. The new function nc_vargetr was added on Dec 18, 2011. Many thanks to John Evans for providing this function to return the data in its native (raw) data type.

Recall that the SNCTOOLS can be downloaded/updated from the svn repository at sourceforge:

svn checkout https://mexcdf.svn.sourceforge.net/svnroot/mexcdf/snctools/trunk ~/matlab/snctools

Similarly, the MEXNC interface can be downloaded/updated from the svn repository at sourceforge:

svn checkout https://mexcdf.svn.sourceforge.net/svnroot/mexcdf/mexnc/trunk ~/matlab/mexnc

Happy New Year 2012...