Cryptic error message from 'ncpdq'

Help
2013-07-12
2013-10-17
  • James Adams
    James Adams
    2013-07-12

    I am trying to reorient the dimensions of a netCDF file from (lat, lon, time) to (time, lat, lon), using the netCDF operator 'ncpdq'.

    I'm getting the following cryptic error message:

    $ ncpdq -a time,lat,lon spi_gpcc_latlontime.nc spi_gpcc_timelatlon.nc
    nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_inq_var_deflate()
    nco_err_exit(): ERROR Error code is 1. Translation into English with nc_strerror(1) is "Operation not permitted"
    nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

    The above is useless to me (maybe it's good debugging info for a developer) - can someone give me a hint as to where I'm going wrong?  I haven't used netCDF in years so it's very possible that I've created my original netCDF incorrectly, or made other bonehead rookie mistakes somewhere along the line (i.e. don't assume that I know what I'm doing here).

    In case it helps here's an ncdump of the netCDF file I'm trying to reorient:

    $ ncdump -h /home/jadams/spi/data/spi_gpcc_latlontime.nc

    netcdf spi_gpcc_latlontime {
    dimensions:
            time = 1356 ;
            lat = 180 ;
            lon = 360 ;
    variables:
            int time(time) ;
                    time:units = "days since 1900-01-01" ;
            float lat(lat) ;
                    lat:units = "degrees_north" ;
            float lon(lon) ;
                    lon:units = "degrees_east" ;
            float spi_01(lat, lon, time) ;
                    spi_01:_FillValue = 1.401298e-45f ;
                    spi_01:missing_value = 1.40129846432482e-45 ;
                    spi_01:Conventions = "CF-1.3" ;
            float spi_02(lat, lon, time) ;
                    spi_02:_FillValue = 1.401298e-45f ;
                    spi_02:missing_value = 1.40129846432482e-45 ;
                    spi_02:Conventions = "CF-1.3" ;
            float spi_03(lat, lon, time) ;
                    spi_03:_FillValue = 1.401298e-45f ;
                    spi_03:missing_value = 1.40129846432482e-45 ;
                    spi_03:Conventions = "CF-1.3" ;
            float spi_06(lat, lon, time) ;
                    spi_06:_FillValue = 1.401298e-45f ;
                    spi_06:missing_value = 1.40129846432482e-45 ;
                    spi_06:Conventions = "CF-1.3" ;
            float spi_09(lat, lon, time) ;
                    spi_09:_FillValue = 1.401298e-45f ;
                    spi_09:missing_value = 1.40129846432482e-45 ;
                    spi_09:Conventions = "CF-1.3" ;
            float spi_12(lat, lon, time) ;
                    spi_12:_FillValue = 1.401298e-45f ;
                    spi_12:missing_value = 1.40129846432482e-45 ;
                    spi_12:Conventions = "CF-1.3" ;
            float spi_24(lat, lon, time) ;
                    spi_24:_FillValue = 1.401298e-45f ;
                    spi_24:missing_value = 1.40129846432482e-45 ;
                    spi_24:Conventions = "CF-1.3" ;

    // global attributes:
                    :missing_value = 1.40129846432482e-45 ;
                    :Conventions = "CF-1.3" ;
    }

    Thanks a lot for any suggestions or comments you can offer.

    -James

     
  • Charlie Zender
    Charlie Zender
    2013-07-13

    What version of NCO? Do ncpdq -r
    Post the dataset where we can retrieve it from a public website.
    Thx,
    cz

     
  • James Adams
    James Adams
    2013-07-15

    Thanks for your help, Charlie.

    The version of NCO that I am using was installed onto a CentOS machine using yum and the EPEL repository:

    $ yum list installed | grep nco
    nco.x86_64                       4.0.5-2.el6                      @epel

    $ ncpdq -r
    NCO netCDF Operators version "4.0.5" last modified 2010/10/08 built Dec 13 2010 on x86-14.phx2.fedoraproject.org by mockbuild
    ncpdq version 4.0.5
    Linked to netCDF library version 4.1.1, compiled Mar 22 2013 18:46:24

    I can guess that the easiest solution here is to build the latest version of NCO from source and install it manually, as the NCO package that I've installed via yum is quite out of date.  A better solution for me would be to have the latest version of NCO built and pushed out to the EPEL repository so I can manage the NCO installation via yum.  Is there a chance of that happening?

    -James

     
  • Charlie Zender
    Charlie Zender
    2013-07-15

    Hi James,
    I'm afraid I have devolved the ability to create EPEL RPMs, though you might search the usual places for for more recent versions than you have installed. Fedora RPMs are nearly up-to-date so it should be possible for an RPM guru to build
    EPEL from that. you're right that upgrading will probably solve the problem.
    cz

     
  • James Adams
    James Adams
    2013-07-15

    Thanks, Charlie.

    Where can I find the nearly up-to-date Fedora RPMs you mention above?  I can probably use one of those without a problem (at least it'll be a lot less problematic than building from source), but the NCO RPMs I see available from the binaries section of the NCO page are even older than the NCO that I have installed (RPMs of versions 3.9.x are available on the NCO page, and I already have 4.0.5 installed).

    If I do have to build from source then I'll be happy to contribute the resulting RPM or yum package, but it sounds like I'm asking for butt hurt if I go that route.  It may be easier for me to translate a portion of your C code into Python to run as part of my process, but I haven't looked into it yet to see how daunting that task may be.  Can you advise?  Does solving my use case (i.e. reorienting from  to  with ncpdq) involve much black magic, or should it instead be reasonably straightforward to accomplish?

    -James

     
  • James Adams
    James Adams
    2013-07-15

    Thanks, Charlie, but I'm not sure what that link does for me.  I need an RPM I can use to install a more recent version of NCO and I can't find anything of the sort there.  Or maybe it's there somewhere and you just assume that I know my way around more than I do.  ;)

    -James

     
  • James Adams
    James Adams
    2013-07-16

    Can you please make an announcement once an updated version for CentOS is available?  I have spent several hours on this and in the end it was all a waste of time since none of the RPMs at this site work for my platform (CentOS 6) save the one I already have installed from EPEL, and that one is out of date and apparently buggy.  Fortunately I've been able to code up a simple axis reorientation method relatively painlessly so I guess I didn't really need this after all, but in the future it'd be nice to be able to access the functionality of this product to save me some coding.  However if it's not supported for my platform (your earlier comment about the EPEL build process going belly up isn't encouraging) and building from source is as herculean of a task as it's been described then I guess I'll just be out of luck, and should get used to cooking up my own netCDF manipulation codes.  :(

    In any event thanks for your help throughout this little adventure.

    -James

     
  • Charlie Zender
    Charlie Zender
    2013-07-16

    building re-distributable rpms is harder than simply building from source.
    we try to make this easy:
    tar xvzf nco-4.3.2.tar.gz
    cd nco-4.3.2
    ./configure -prefix=/usr/local
    make
    make install

    i assume you tried something like this and it failed.
    if so, please send us the failure information so we can fix nco so that it builds from source on centos.

    cz

     
  • James Adams
    James Adams
    2013-07-16

    Thanks for your continued support, Charlie.  To be honest I never tried building from source for two reasons: 1) your documentation warns that it will be painful, and 2) building from source and installing to /usr/local is an easy way to allow your system to quickly become quite crufty if you're not very careful, as there is little in the way of package management using that approach and instead using RPMs/yum is a much better solution.  Hence my attempt to solve this first by searching out an appropriate yum package or RPM for my system.  However since that proved to be fruitless and you've tried to help me by spoon feeding me the build commands I gave it a whirl.  It didn't completely work, although it appeared to make quite some progress until it croaked with the following messages (only the last of the output is included below):

    make: Leaving directory `/root/tmp/install/nco-4.3.2/src/nco_c++'
    Making all in nco++
    make: Entering directory `/root/tmp/install/nco-4.3.2/src/nco++'
    g++ -DHAVE_CONFIG_H -I. -I../.. -I./ -I../nco  -I/usr/include   -g -O2 -MT Invoke.o -MD -MP -MF .deps/Invoke.Tpo -c -o Invoke.o Invoke.cc
    mv -f .deps/Invoke.Tpo .deps/Invoke.Po
    g++ -DHAVE_CONFIG_H -I. -I../.. -I./ -I../nco  -I/usr/include   -g -O2 -MT ncap2.o -MD -MP -MF .deps/ncap2.Tpo -c -o ncap2.o ncap2.cc
    ncap2.cc: In function âint main(int, char**)â:
    ncap2.cc:527: error: ânco_gsl_clsâ was not declared in this scope
    ncap2.cc:527: error: expected â;â before ânco_gsl_objâ
    ncap2.cc:530: error: ânco_gsl_objâ was not declared in this scope
    make: ***  Error 1
    make: Leaving directory `/root/tmp/install/nco-4.3.2/src/nco++'
    make: ***  Error 1
    make: Leaving directory `/root/tmp/install/nco-4.3.2/src'
    make: ***  Error 1
    make: Leaving directory `/root/tmp/install/nco-4.3.2'
    make: ***  Error 2

    So it appears that in the file ncap2.cc there's an object named "nco_gsl_obj" being referenced outside of its scope.

    I hope the above is helpful, and please let me know if I can do anything further to help iron out this or other CentOS related wrinkles.

    -James

     
  • Charlie Zender
    Charlie Zender
    2013-07-16

    did you do this first as instructed on the homepage?:

    sudo yum install antlr antlr-C++ # ANTLR
    sudo yum install curl-devel libxml2-devel # DAP-prereqs
    sudo yum install libdap libdap-devel # DAP
    sudo yum install bison flex gcc gcc-c++ # GNU toolchain
    sudo yum install gsl gsl-devel # GSL
    sudo yum install netcdf netcdf-devel # netCDF
    sudo yum install librx librx-devel # RX
    sudo yum install udunits2 udunits2-devel # UDUnits

     
  • Pedro Vicente
    Pedro Vicente
    2013-07-17

    Hi James

    That error shows that the GSL library was not found.

    Can you give a try to install GSL from source?

    You can obtain GSL here

    https://www.gnu.org/software/gsl/

    Pedro

     
  • James Adams
    James Adams
    2013-07-17

    Thanks, Pedro.  As Charlie pointed out above I had neglected to install the package dependencies before going forward with the configure/make steps of a manual build/install.  After installing the package dependencies via yum, including GSL, I was able to build and install NCO to /usr/local as advertised.  A quick test of ncpdq confirms that things are now working as expected there, in that I can now use ncpdq to reorder the dimensions of my variables.

    -James