Unfold multidimensional variables to 1D

Help
NewUSer
2013-07-27
2013-10-17
  • NewUSer
    NewUSer
    2013-07-27

    How do I unfold multidimensional variables to 1d arrays.

    I see this being mentioned as a way to subsetting multidimensional variables but have no clue how to do this. Could somebody please help.

     
  • Charlie Zender
    Charlie Zender
    2013-07-27

    NCO does not support this. To subset multidimensional variables, see http://nco.sf.net/nco.html#hyp

     
  • NewUSer
    NewUSer
    2013-07-27

    Charlie,
    I was referring to your response here.
    http://sourceforge.net/projects/nco/forums/forum/9829/topic/4836600

    I would like subset based on variables. For example Temperature >  7. Temperature is a multidimensional variable with latitude, longitude, depth etc. Isnt there a way to do this in NCO other than writing FullValue everywhere except when Temperature > 7.

    I greatly appreciate your help.

     
  • Charlie Zender
    Charlie Zender
    2013-07-27

    It's still not clear to what you want. there is where()
    nco.sf.net/nco.html#where
    otherwise please be more precise.
    cz

     
  • NewUSer
    NewUSer
    2013-07-27

    ncdump -h POP2.nc
    netcdf POP2 {
    dimensions:
            depth_t = 40 ;
            t_lat = 384 ;
            t_lon = 320 ;
            u_lon = 320 ;
            u_lat = 384 ;
            w_dep = 41 ;
    variables:
            float TEMP(depth_t, t_lat, t_lon) ;
                    TEMP:_FillValue = -999.f ;
                    TEMP:long_name = "Potential Temperature" ;
                    TEMP:missing_value = -1.e+34f ;
                    TEMP:units = "degC" ;
    }

    The above is a short version of the file I have. I want to select all the Temperature values greater than 7 and write them to another netcdf file.

    Please let me know if it is clear.

     
  • Charlie Zender
    Charlie Zender
    2013-07-28

    what about ncap2 where(), in previous suggestion, does not suffice for this?

     
  • NewUSer
    NewUSer
    2013-07-28

    ./ncap2 -O -s 'where (TEMP > 7.0) TEMP=TEMP' ../../dataset/POP/POP.nc temp2.nc

    The above query does not work. The result temp2.nc file contains TEMP values that are less than 7.

    ./ncap2 -O -s 'where (TEMP > 7.0) TEMP=TEMP elsewhere TEMP=TEMP@_FillValue' ../../dataset/POP/POP.nc temp2.nc

    The above statement uses a FillValue at all place where TEMP <= 7.0.

    What I want is a temp.nc file in which only TEMP>7.o values are stored along with corresponding latitude, longitude and other dimensions.

     
  • Charlie Zender
    Charlie Zender
    2013-07-29

    thanks for explaining. i was afraid this was what you were asking for, but i wasn't sure. although not a technically difficult problem (compared, say, to permuting) unrolling variables is not yet support in NCO. it is TODO nco1076, and intended to be an option to ncpdq, but your use case makes clear that offering it in ncap2 would be desirable too, so that it could be used in tandem with where() to define new output dimension sizes that fit only the matching points. i will post again when there is progress on this TODO.
    cz

     
  • augusto
    augusto
    2013-07-29

    Hi,
    I'd solve this problem in 2 steps: 1) I'd collect temp > 7 using:
    ncap2 -O -s 'where (TEMP > 7.0) new_tmp=TEMP' inf.nc outf.nc
    I use this statement very often and it works alright (make sure that you have the latest NCO version).
    2) I'd transform the multi-dimension TEMP of file outf.nc into one-dimension using:
    ncap2 -s 'TEMP=new_tmp(depth_t,1,1,1 …)' outf.nc newf.nc

    Hope it helps,

    Augusto

     
  • Charlie Zender
    Charlie Zender
    2013-07-30

    1-dimensional unrolling works and is documented in the manual at http://nco.sf.net/nco.html#sort
    Helene's procedure works for her.
    I trust Helene, but I don't have the time to test and document her multi-dimensional procedure step-by-step.
    The best way is to create a _small_ test file with known answers and verify the procedure works for you.

    Regarding Augusto's suggestion, I do not understand why it would work. He needs to be more explicit in step 2,
    which is the key to the problem.

    cz