Count missing values at each gridbox

Help
2011-01-03
2013-10-17
  • Imran Nadeem
    Imran Nadeem
    2011-01-03

    Dear All,
      I have gridded time series of data in the form var(time,lat,lon) and I want to count missing values at each gridbox. Any hint??
    Thanks in advance
    Nadeem

     
  • Charlie Zender
    Charlie Zender
    2011-01-05

    Here are some commands that compute totals of missing and valid
    elements in various hyperslabs:

    ncap2 -O -s 'missing_flag=1;where(three_dmn_var_dbl) missing_flag=0;missing_count=missing_flag.total();print(missing_count)' ~/nco/data/in.nc ~/foo.nc

    ncap2 -O -s 'missing_flag=1;where(three_dmn_var_dbl) missing_flag=0;missing_count=missing_flag.total($time);print(missing_count)' ~/nco/data/in.nc ~/foo.nc

    ncap2 -O -s 'valid_flag=0;where(three_dmn_var_dbl) valid_flag=1;valid_count=valid_flag.total();print(valid_count)' ~/nco/data/in.nc ~/foo.nc

    ncap2 -O -s 'valid_flag=0;where(three_dmn_var_dbl) valid_flag=1;valid_count=valid_flag.total($time);print(valid_count)' ~/nco/data/in.nc ~/foo.nc

    See http://nco.sf.net/nco.html#where for more info.

    hth,
    cz

     
  • Charlie Zender
    Charlie Zender
    2011-01-05

    p.s. thanks for this question. the answer makes it clear that ncap2 needs .ismissing() and .isvalid() methods that return same-sized arrays of ones and zeros. will put that on the TODO list
    cz

     
  • SNINCR
    SNINCR
    2013-01-15

    Perhaps this method to check for missing values fails for points in three_dmn_var_dbl that are equal to zero?
    For example, the following test code:

    *array_dim1_size=3;
    defdim("array_dim1",array_dim1_size);
    array1=array(-1.0,1.0,$array_dim1);
    *the_missing_value=-999999999.0;
    array1.set_miss(the_missing_value);
    array1(0)=the_missing_value;
    print(array1);
    is_array1_missing=array1;
    is_array1_missing = 1;
    where(array1){
     is_array1_missing=0;
    }
    print(is_array1_missing);
    

    Yields:

    array1[0]=-999999999
    array1[1]=0
    array1[2]=1
    is_array1_missing[0]=1
    is_array1_missing[1]=1
    is_array1_missing[2]=0
    

    The second entry in array1 is "0" and the where statement evaluates to false for "0", so is_array1_missing is "1".

    Perhaps it is more robust to use:

    where(array1==array1)
    

    instead of:

    where(array1)
    

    If one adds to the end of the previous code:

    is_array1_missing_b=array1;
    is_array1_missing_b = 1;
    where(array1==array1){
     is_array1_missing_b=0;
    }
    print(is_array1_missing_b);
    

    one gets:

    is_array1_missing_b[0]=1
    is_array1_missing_b[1]=0
    is_array1_missing_b[2]=0
    
     
  • henry Butowsky
    henry Butowsky
    2013-01-15

    HI Charlie & Nadeem,
    sorry I should have mentioned that 6 months ago I added the function "number_miss" which returns the number of missing values in a var

    …hmb

     
  • Charlie Zender
    Charlie Zender
    2013-01-18

    > Perhaps it is more robust to use: where(array1==array1)

    This is a very good point. Thanks.

    I added a brief description of number_miss() to the manual. The ncap2 section needs editing and re-writing. Any help appreciated.