segmentation fault with ncwa -y ttl

[John]

I run

ncwa -y ttl -a lat,lon in.nc out.nc

and when I print out the sum

ncks out.nc

I get segmentation fault. I'm using NCO 4.4.6.

I have a small sample nc file. I'm not seeing a file attachment button here. Hopefully I can attach in a reply.

[John]

My sample file in.nc (1kb) is attached. Actually, I ran

ncks in.nc

and I get a segmentation fault. Here is ncdump of the whole file:

netcdf in {
dimensions:
    year = UNLIMITED ; // (1 currently)
    lat = 10 ;
    lon = 10 ;
variables:
    short FIRE(year, lat, lon) ;
        FIRE:_FillValue = -9999s ;
        FIRE:long_name = "Mode ofVTYPE" ;
        FIRE:units = "None. The values are categories." ;
    float lat(lat) ;
    float lon(lon) ;
    short year(year) ;

// global attributes:
        :Title = "Mode of VTYPE" ;
        :history = "Sun Mar  6 15:57:50 2016: ncks -d lat,,9 -d lon,240,249 t_mask2.nc test.nc" ;
        :NCO = "4.4.6" ;
data:

 FIRE =
  1, 1, 1, _, _, _, _, _, _, _,
  1, 1, 1, 1, _, _, _, _, _, _,
  1, 1, 1, 1, 1, _, _, _, _, _,
  1, 1, 1, 1, 1, 1, 1, _, _, _,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ;

 lat = 49, 48.99166, 48.98333, 48.975, 48.96667, 48.95833, 48.95, 48.94167, 
    48.93333, 48.925 ;

 lon = -122.7333, -122.725, -122.7167, -122.7083, -122.7, -122.6917, 
    -122.6833, -122.675, -122.6667, -122.6583 ;

 year = _ ;
}

Is there something wrong with this file? I thought maybe ncwa doesn't like to have year be null, so I tried setting that to 1 and that didn't help.

Thanks.

[Charlie Zender]

Hi John,
It seems to work fine with the latest version. Not sure exactly which version would have fixed this, but suggest you upgrade.
cz

zender@aerosol:~/Downloads$ ncwa -y ttl -a lat,lon in.nc out.nc
zender@aerosol:~/Downloads$ ncks out.nc
Summary of out.nc: filetype = NC_FORMAT_CLASSIC, 0 groups (max. depth = 0), 1 dimensions (0 fixed, 1 record), 4 variables (4 atomic-type, 0 non-atomic), 10 attributes (4 global, 0 group, 6 variable)

Root record dimension 0: name = year, size = 1

Global attributes:
Global attribute 0: Title, size = 13 NC_CHAR, value = Mode of VTYPE
Global attribute 1: history, size = 137 NC_CHAR, value = Sun Mar 6 16:19:16 2016: ncwa -y ttl -a lat,lon in.nc out.nc
Sun Mar 6 15:57:50 2016: ncks -d lat,,9 -d lon,240,249 t_mask2.nc test.nc
Global attribute 2: NCO, size = 16 NC_CHAR, value = "4.5.6-alpha03"
Global attribute 3: nco_openmp_thread_number, size = 1 NC_INT, value = 1

FIRE: type NC_SHORT, 1 dimension, 4 attributes, compressed? no, chunked? no, packed? no
FIRE size (RAM) = 1sizeof(NC_SHORT) = 12 = 2 bytes
FIRE dimension 0: year, size = 1 NC_SHORT (Record coordinate is year)
FIRE attribute 0: _FillValue, size = 1 NC_SHORT, value = -9999
FIRE attribute 1: long_name, size = 12 NC_CHAR, value = Mode ofVTYPE
FIRE attribute 2: units, size = 32 NC_CHAR, value = None. The values are categories.
FIRE attribute 3: cell_methods, size = 13 NC_CHAR, value = lat, lon: sum

lat: type NC_FLOAT, 0 dimensions, 1 attribute, compressed? no, chunked? no, packed? no
lat size (RAM) = 1sizeof(NC_FLOAT) = 14 = 4 bytes
lat attribute 0: cell_methods, size = 9 NC_CHAR, value = lat: mean

lon: type NC_FLOAT, 0 dimensions, 1 attribute, compressed? no, chunked? no, packed? no
lon size (RAM) = 1sizeof(NC_FLOAT) = 14 = 4 bytes
lon attribute 0: cell_methods, size = 9 NC_CHAR, value = lon: mean

year: type NC_SHORT, 1 dimension, 0 attributes, compressed? no, chunked? no, packed? no
year size (RAM) = 1sizeof(NC_SHORT) = 12 = 2 bytes
year dimension 0: year, size = 1 NC_SHORT (Record coordinate is year)

year[0]=-32767 FIRE[0]=79 None. The values are categories.

lat = 48.9625

lon = -122.696

year[0]=-32767

zender@aerosol:~/Downloads$

[John]

Updated to 4.5.5. and it works fine now. Thanks.

FWIW, before I got the update, I deleted all global attributes:

ncatted -O -a ,global,d,, -h in.nc in.nc

and the problem went away. So ... it must've been something to do with attributes....

[John]

I take that back. ncwa 4.5.5 works OK on a small sample file, but on my larger project file, it gives a weird answer. I have a 900x1048x1 grid, which is 1's and 0's (and _FillValue's) (attached). I do

ncwa -y ttl -a year,lat,lon -h in.nc out.nc

on it, and I get a negative total. Is it because my variable is a short? If so, is there a way to tell ncwa to use a larger variable type to calculate a sum?

Thanks.

[John]

OK, I converted the variable type from SHORT to INT and now all's well.

I guess if you ask ncwa to calculate a total, it doesn't automagically change types for you.

Lesson learned.

[Charlie Zender]

ncwa converts to double precision for arithmetic, but then it converts back to the original type for storage. and if your number is too big for that type, yes, the answer looks weird.
cz