BUGhdf5-ncrename var to named dims

Developers
ocehugo
2012-11-21
2013-10-17
  • ocehugo
    ocehugo
    2012-11-21

    Hello,

    ncrename raise an error if i try to rename a variable to the same name as a dimension if we are working with HDF/N4 files:

    /dev/shm $ file a.nc
    a.nc: Hierarchical Data Format (version 5) data
    /dev/shm $ ncdump -h a.nc 
    netcdf a {
    dimensions:
        time = 10 ;
        lat = 10 ;
        lon = 10 ;
    variables:
        double atime(time) ;
        float tair(time, lat, lon) ;
        double lat(lat) ;
        double lon(lon) ;
    ...
    

    Now we call :

    ncrename -D9 -v atime,time a.nc
    atime
    time
    ncrename: INFO nc__open() will request file buffer of default size
    ncrename: INFO nc__open() opened file with buffer size = 0 bytes
    ncrename: TIMER Metadata setup and file layout before main loop took    0.00 s
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_rename_var()
    nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
    

    For a netcdf3 file this is not happening:

    /dev/shm $ ncks -D9 -O -3 a.nc a_n3.nc && ncrename -D9 -v atime,time a_n3.nc && ncdump -h a_n3.nc
    ncks: INFO nc__open() will request file buffer of default size
    ncks: INFO nc__open() opened file with buffer size = 0 bytes
    ncks: INFO nco_inq() reports file contains 4 variables, 3 dimensions, and 2 global attributes
    ncks: INFO nco4_inq() reports file contains 1 group comprising 4 variables, 3 dimensions, and 2 global attributes
    ncks: INFO nco4_var_lst_mk() reports group /, / has 4 variables:
    var_nm=atime, var_nm_fll=/atime
    var_nm=tair, var_nm_fll=/tair
    var_nm=lat, var_nm_fll=/lat
    var_nm=lon, var_nm_fll=/lon
    ncks: INFO nco4_var_lst_mk() reports file contains 1 group comprising 4 total variables
    ncks: nco_fl_out_open() reports sizeof(pid_t) = 4 bytes, pid = 25659, pid_sng_lng = 6 bytes, strlen(pid_sng) = 5 bytes, fl_out_tmp_lng = 26 bytes, strlen(fl_out_tmp) = 25, fl_out_tmp = a_n3.nc.pid25659.ncks.tmp
    ncks: INFO nco_aed_prc() examining variable Global
    ncks: TIMER Metadata setup and file layout before main loop took    0.00 s
    atime, lat, lon, tair, ncks: INFO Moving a_n3.nc.pid25659.ncks.tmp to a_n3.nc...done
    ncks: TIMER Wallclock-elapsed time for command is    0.00 s
    atime
    time
    ncrename: INFO nc__open() will request file buffer of default size
    ncrename: INFO nc__open() opened file with buffer size = 8192 bytes
    ncrename: TIMER Metadata setup and file layout before main loop took    0.00 s
    ncrename: Renamed variable 'atime' to 'time'
    ncrename: TIMER Wallclock-elapsed time for command is    0.00 s
    netcdf a_n3 {
    dimensions:
        time = 10 ;
        lat = 10 ;
        lon = 10 ;
    variables:
        double time(time) ;
        double lat(lat) ;
        double lon(lon) ;
        float tair(time, lat, lon) ;
    ...
    

    very annoying because hdf5/n4 attributes tables and names are much better handle than in n3 (avoid rewriting the whole file). Hopefully we can change the dimension name to othe rname since the variables names are more "sensitive" from offline reading.

    And yes…changing the dimension name to some temporary name, changing the variable name and rolling back the dimension name do not work either.

    using: NCO netCDF Operators version "4.2.0"

    hope that helps

     
  • Charlie Zender
    Charlie Zender
    2012-11-21

    Hello there,
    I believe this is a symptom of a bug in the netCDF4 library. I filed a bug report some years ago.
    Information on these issues is in the Known Bugs Not Yet Fixed portion of the NCO ANNOUNCEments
    From the latest:

    B. NOT YET FIXED
       netCDF4 library fails when renaming dimension and variable using
       that dimension, in either order. Works fine with netCDF3.
       Problem with netCDF4 library implementation.

       Demonstration:
       ncks -O -4 -v lat_T42 ~/nco/data/in.nc ~/foo.nc
       ncrename -O -D 2 -d lat_T42,lat -v lat_T42,lat ~/foo.nc ~/foo2.nc # Breaks with "NetCDF: HDF error"
       ncks -m ~/foo.nc

       20121025: Verified problem still exists
       Bug report filed: netCDF #YQN-334036: problem renaming dimension and coordinate in netCDF4 file

    I suggest you encourage Unidata to look into this issue if getting it resolved is important to you.

    cz

     
  • ocehugo
    ocehugo
    2012-11-22

    Hello zender,

    thanks for the answer. I'll look into the announcements next time, sorry about that. About the netcdf i'm using 4.2.1.1,  so since your bug report is from 2011, certanly the netcdf-snapshot will not help with this problem. i'll try to send another email to the unidata support.

    cheers