create _FillValue

Help
Ibe
2010-09-13
2013-10-17
  • Ibe

    Ibe - 2010-09-13

    Hi,

    I am trying to rename all missing values to fill values using:

    ncrename -a .missing_value,_FillValue pk1.nc

    It changed missing value to 'FillValue' but also gave following error:
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): main
    nco_err_exit(): ERROR Error code is -43. Translation into English with nc_strerror(-43) is "NetCDF: Attribute not found"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

    Is it OK to proceed by neglecting above error or else. Any help would be highly appreciated.

    Thanks,
    Ibe

     
  • Charlie Zender

    Charlie Zender - 2010-09-13

    Ibe,
    Thank you for pointing this out.
    This is a bug, since it should be legal not to have any matches to the
    old_name as stated in the documentation. Not sure when this broke.
    It used to work, honest! Will post again when I have committed a patch
    that will be in NCO 4.0.4. This is TODO nco992.
    cz

     
  • Charlie Zender

    Charlie Zender - 2010-09-13

    Ibe,

    The patch has been committed to the main trunk of ncrename.c.

    In answer to your earlier question, yes, you may neglect the error above.
    The file was being written correctly and then the code was inadvertently exiting
    as if it had failed because I had forgotten to reset a return code.
    That has been fixed. But it was writing the correct files the whole time.

    cz

     
  • Ibe

    Ibe - 2010-09-13

    I appreciate your help and comments.

    -Ibe

     
  • Kyle Wilcox

    Kyle Wilcox - 2011-03-18

    I'm also seeing this issue in ncatted (4.0.6):

    ]$ ncatted -a _FillValue,variable_name,o,f,-999 out.nc
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_rename_att()
    nco_err_exit(): ERROR Error code is -43. Translation into English with nc_strerror(-43) is "NetCDF: Attribute not found"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

    To get around the issue I've been creating a missing_value attribute and renaming it to _FillValue, which works:
    ncatted -a missing_value,variable_name,o,f,-999 out.nc
    ncrename -a variable_name@missing_value,_FillValue out.nc

     
  • Charlie Zender

    Charlie Zender - 2011-03-18

    Thanks for pointing this out, and the workaround. Clearly we need better regression tests.
    Here is a matrix of what does/does not work with ncatted -a overwrite mode in 4.0.7:
    ncatted -O -a _FillValue,fll_val,o,f,-999 ~/nco/data/in.nc ~/foo.nc   # Works on netCDF3 file when attribute already exists
    ncatted -O -a _FillValue,lon,o,f,-999     ~/nco/data/in.nc ~/foo.nc   # Works on netCDF3 file when attribute does not yet exist
    ncatted -O -a _FillValue,fll_val,o,f,-999 ~/nco/data/in_4.nc ~/foo.nc # Works on netCDF4 file when attribute already exists
    ncatted -O -a _FillValue,lon,o,f,-999     ~/nco/data/in_4.nc ~/foo.nc # Fails on netCDF4 file when attribute does not yet exist

    So, it appears to fail only on netCDF4 files where the attribute already exists.
    netCDF4 handles _FillValue unlike netCDF3. If it is possible to get this working
    it will be in nco 4.0.8. Until then, use the workaround you suggested (or convert
    to netCDF3 in/out files).
    cz

     
  • Charlie Zender

    Charlie Zender - 2011-03-18

    correction, it fails when the attribute does not yet exist on netcdf4 files.
    am looking into it.
    cz

     
  • Charlie Zender

    Charlie Zender - 2011-03-18

    the fix is now in CVS and in the beta releases and will be in 4.0.8.
    cz

     
  • e ddc

    e ddc - 2012-12-10

    Hello

    I am having a similar problem with nco-4.2.3 using ncrename.
    The name of the attribute is effectively changed but NCO exits with a error message.
    This also ends my scripts since I am using and have to use "set -e".

    ncrename -O -h -a tracking_id,input_tid llixo_tas_piControl.nc lixo1.nc
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): main
    nco_err_exit(): ERROR Error code is -43. Translation into English with nc_strerror(-43) is "NetCDF: Attribute not found"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

    Should I install release 4.0.8?

    Thanks a lot in advance for you help.

    Regards,
    e ddc

     
  • Charlie Zender

    Charlie Zender - 2012-12-10

    please be more specific about the problem you are having.
    include the commands and output that demonstrate why ncrename's actions are incorrect.
    also include output of ncrename -r
    your  error message suggests that tracking_id does not exist.

    ncrename renames attributes fine for me.

    ncrename -O -h -a units,you_nits ~/nco/data/in.nc ~/foo.nc

    cz

     
  • e ddc

    e ddc - 2012-12-12

    Hello again

    Sorry for the late reply.

    The exact commands are

    Modifying header of llixo_pr_piControl.nc
    + ncrename -O -h -a tracking_id,input_tid llixo_pr_piControl.nc lixo1.nc
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): main
    nco_err_exit(): ERROR Error code is -43. Translation into English with nc_strerror(-43) is "NetCDF: Attribute not found"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
    + set -e
    + set +x

    file llixo_pr_piControl.nc in here http://www.filedropper.com/llixoprpicontrol
    file lixo1.nc here http://www.filedropper.com/lixo1

    Thanks.

     
  • e ddc

    e ddc - 2012-12-12

    Hello cz

    Sorry I forgot to include this :
    ncrename -r
    NCO netCDF Operators version "4.2.3" last modified 2012/10/22 built Dec 10 2012 on ariel by dacosta
    ncrename version 4.2.3
    Linked to netCDF library version 4.2.1.1, compiled Dec 10 2012 09:36:03
    Copyright (C) 1995-2012 Charlie Zender
    NCO is free software and comes with a BIG FAT KISS and ABOLUTELY NO WARRANTY
    License: GNU General Public License (GPL) Version 3

    Configuration Option:   Active?
    Check _FillValue        Yes    
    Check missing_value     No     
    Compressed netCDF3      No     
    DAP clients (libdap)    No     
    DAP clients (libnetcdf) No     
    Debugging: Custom       No     
    Debugging: Symbols      No     
    GNU Scientific Library  No     
    Internationalization    No     
    MPI parallelization     No     
    netCDF3 64-bit files    Yes    
    netCDF4/HDF5            Yes    
    OpenMP SMP threading    No     
    Optimization: run-time  No     
    Parallel netCDF3        No     
    Regular Expressions     Yes    
    Shared libraries built  Yes    
    Shell globbing          No     
    Static libraries built  Yes    
    UDUnits conversions     Yes    
    UDUnits2 conversions    Yes    

    It keeps saying
    Sorry, the text entered looks like spam and cannot be posted.
    but I will find a way!

     
  • Charlie Zender

    Charlie Zender - 2012-12-12

    Thanks for reporting this. You have found a previously unknown bug in which _global attributes_ (and only global attributes) are not correctly handled by ncrename. I will post here again after I have committed a patch. FYI, the test case for this is
    ncrename -O -h -a Conventions,Geneva ~/nco/data/in.nc ~/foo.nc
    This is now TODO nco1082.
    cz

     
  • Charlie Zender

    Charlie Zender - 2012-12-12

    The temporary workaround is to prefix the global attribute name with a period:
    ncrename -O -h -a .Conventions,Geneva ~/nco/data/in.nc ~/foo.nc
    This follows a different code path and works fine.
    cz

     
  • Charlie Zender

    Charlie Zender - 2012-12-12

    I spolke too soon. It turns out this is not a bug. It is exactly the documented behavior of ncrename.
    By not prefixing the old attribute name with a period, you are implicitly telling ncrename that the attribute exists both as a global attribute and for all variables, so when ncrename finds a variable without this attribute, it fails.
    This is a useful convention for variables yet is a silly convention to follow for global attributes, and it needs changing.
    Anyway, use the workaround for now.
    cz

     
  • Charlie Zender

    Charlie Zender - 2012-12-15

    I committed changes to the CVS trunk that solve this problem more elegantly (without the workaround).
    These changes will be in 4.2.4. Please test them if you can.
    From the upcoming ANNOUNCE:

    A. ncrename supports restricting actions to global/group attributes
       Formerly ncrename arguments like -a old_nm,new_nm had no way of
       restricting actions to global/group attributes and changing them
       required implicit assumptions about whether variables could have
       attributes of the same name. Now attribute renaming options can
       be restricted to global/group attributes either by leaving the
       var_nm portion empty (and retaining the @), or by using var_nm=
       global to indicated that att_nm refers to a global attribute.
       ncrename -a  global@old_att_nm,new_att_nm in.nc
       ncrename -a        @old_att_nm,new_att_nm in.nc
       ncrename -a global@.old_att_nm,new_att_nm in.nc
       http://nco.sf.net/nco.html#xmp_ncrename

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks