parsing attributes with ncatted

Developers
2010-10-19
2013-10-17
  • jsteinkraus
    jsteinkraus
    2010-10-19

    I’m wondering if there’s a bug/behavior-change with ncatted related to recent work on nco989 (“implement nco_sng_cnv_err() checking throughout code-base”).  Here’s my command:

    ncatted -O -h -a _FillValue,LST_Day_1km,o,s,0s -a scale_factor,LST_Day_1km,o,f,0.02 -a long_name,LST_Day_1km,o,c,'Daily daytime 1km grid Land-surface Temperature' -a LST,LST_Day_1km,o,c,'LST data * scale_factor' -a calibrated_nt,LST_Day_1km,o,s,5 -a units,LST_Day_1km,o,c,'K' -a scale_factor_err,LST_Day_1km,o,f,0. -a add_offset_err,LST_Day_1km,o,f,0. -a grid_mapping,LST_Day_1km,d,, /home/myuser/foo.nc /home/myuser/foo2.nc

    And the output:

    nco_sng_cnv_err(): ERROR an NCO function or main program attempted to convert the user-defined string "0s" to an integer-type using the standard C-library function "strtoll()". This function stopped converting the input string when it encountered the illegal (i.e., non-numeric or non-integer) character 's'. This probably indicates a syntax error by the user. Please check the argument syntax and re-try the command. Exiting…

    The “0s” here (value of the first attribute) seems to be the problem; when I just make it “0”, things run fine.  Obviously the “s” indicates the attribute is a short-value… this is the formatting used in NetCDF, and it seems like it should be possible to just pass that straight through here… indeed, I believe this used to work.   I’m hoping to avoid having to add a lot of extra handling to manually parse the values to remove things like this.

    I downloaded the code for v4.0.5, and in nco/nco_att_utl.c, I see this on line 723-734:

    val_arg_lng_lng=strtoll(arg_lst,&sng_cnv_rcd,NCO_SNG_CNV_BASE10);
    if(*sng_cnv_rcd) nco_sng_cnv_err(arg_lst,"strtoll",sng_cnv_rcd);

    So if there are any unrecognized characters, it throws up an error.  However, my case seems like a perfectly reasonable thing to do (and presumably the reason that strtoll() is the way it is, allowing unrecognized characters at the end is exactly for this case).

    What's the thinking behind this decision to throw an error in this case?  And is this something that changed recently?

    Thanks

     
  • Charlie Zender
    Charlie Zender
    2010-10-20

    Jeff,

    This change probably appeared in NCO 4.0.4.
    While it breaks your usage, it solves other problems.
    Essentially we now use the strto() functions to convert
    user-specified strings to numbers (except for numbers that go through
    a lexical scanner (lexer) first, like ncap2 input).
    This improves diagnostics for user-input errors.
    Hence I am disinclined to revert to the old "non strict string checking".
    I also note that the workaround for the problem you are experiencing
    is straightforward, and, in my opinion, the benefits to the new code
    still outweigh the problems it creates. That said, I think your usage
    should be valid, it's a question of how to implement more intelligent
    string checking.

    Your are correct that, e.g., 0s and 0l are valid input for netCDF 
    shorts and unsigned longs. They are also valid (as are 0u, 0ull…)
    to NCO in other contexts, mainly in ncap2 scripts. However, they no
    longer work as numeric arguments to NCO command options. I did not
    foresee this when I implemeted the stricter string checking in 4.0.4.

    One solution would be to use a full lexer to convert all strings
    representing numeric arguments. However, this would be too much work for me
    (patches by others gladly accepted!).
    Another solution would be to use supersets or wrappers for
    strto() that understand the suffixes
    u,s,l,us,ul,ull,U,S,L,US,UL,ULL.
    My guess is that such code must exist somewhere already because this
    seems like a generic problem, and I do not want to reinvent the wheel.
    I would be grateful if someone points me to such source code with a
    GPL v3 comliant license.

    Charlie