NCO netCDF Operators

Hello all. I just spent three days attempting to build NCO and its dependencies on RHEL / CentOS 5. I attempted with both the GNU compiler and the Intel Compiler. I finally succeeded and hope others may avoid the pain with these steps. Cheers. http://idolinux.blogspot.com/2011/02/nco-netcdf-operators-build-log.html

Compiling NCO (especially ncap2) builds character more often than it builds itself. Thanks for your scripts. Please submit patches to help simplify the build system! cz

Thanks Gavin, that got a few of us on the right page.... below is the script that finally got it built for us on RedHat5 #!/bin/bash INSTALL_PATH=/usr/local mkdir -p $INSTALL_PATH/src # ANTLR2 APP=antlr-2.7.7 ANTLR_PATH=$INSTALL_PATH/$APP rm -rf $ANTLR_PATH cd $INSTALL_PATH/src rm -rf antlr-2.7.7* wget http://www.antlr2.org/download/antlr-2.7.7.tar.gz tar xzf antlr-2.7.7.tar.gz ; cd antlr-2.7.7 CC=gcc CXX='' ./configure \ --prefix=$ANTLR_PATH \ --disable-csharp \ --disable-java \ --disable-python 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # UDUNITS APP=udunits-2.1.20 UDUNITS_PATH=$INSTALL_PATH/$APP rm -rf $UDUNITS_PATH cd $INSTALL_PATH/src rm -rf udunits-2.1.20* wget ftp://ftp.unidata.ucar.edu/pub/udunits/udunits-2.1.20.tar.gz tar xzf udunits-2.1.20.tar.gz ; cd udunits-2.1.20 CC=gcc CXX='' F77=gfortran ./configure \ --prefix=$UDUNITS_PATH 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # ZLIB APP=zlib-1.2.5 ZLIB_PATH=$INSTALL_PATH/$APP rm -rf $ZLIB_PATH cd $INSTALL_PATH/src rm -rf zlib-1.2.5* wget http://zlib.net/zlib-1.2.5.tar.gz tar zxf zlib-1.2.5.tar.gz ; cd zlib-1.2.5 CC=gcc FC=gfortran CXX='' ./configure \ --prefix=$ZLIB_PATH | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # HDF5 APP=hdf5-1.8.6 HDF5_PATH=$INSTALL_PATH/$APP rm -rf $HDF5_PATH cd $INSTALL_PATH/src rm -rf hdf5-1.8.6* wget http://www.hdfgroup.org/ftp/HDF5/hdf5-1.8.6/src/hdf5-1.8.6.tar.gz tar xzf hdf5-1.8.6.tar.gz ; cd hdf5-1.8.6 CC=gcc FC=gfortran CXX='' ./configure \ --prefix=$HDF5_PATH \ --enable-fortran \ --with-zlib=$ZLIB_PATH \ --with-pic 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # NetCDF4 APP=netcdf-4.1.1 NETCDF4_PATH=$INSTALL_PATH/$APP rm -rf $NETCDF4_PATH cd $INSTALL_PATH/src rm -rf netcdf-4.1.1* wget http://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-4.1.1.tar.gz tar xzf netcdf-4.1.1.tar.gz ; cd netcdf-4.1.1 CC=gcc FC=gfortran F77=gfortran CXX='' ./configure \ --prefix=$NETCDF4_PATH \ --enable-fortran \ --enable-static \ --enable-shared\ --enable-f77 \ --disable-cxx \ --enable-netcdf4 \ --with-hdf5=$HDF5_PATH \ --with-zlib=$ZLIB_PATH \ --with-pic 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # NCO APP=nco-4.0.6 NCO_PATH=$INSTALL_PATH/$APP rm -rf $NCO_PATH cd $INSTALL_PATH/src rm -rf nco-4.0.6* wget http://nco.sourceforge.net/src/nco-4.0.6.tar.gz tar xzf nco-4.0.6.tar.gz ; cd nco-4.0.6 export LD_LIBRARY_PATH=$HDF5_PATH/lib:$LD_LIBRARY_PATH export PATH=$HDF5_PATH/bin:$PATH export LD_LIBRARY_PATH=$NETCDF4_PATH/lib:$LD_LIBRARY_PATH export PATH=$NETCDF4_PATH/bin:$PATH export LD_LIBRARY_PATH=$ANTLR_PATH/lib:$LD_LIBRARY_PATH export PATH=$ANTLR_PATH/bin:$PATH export LD_LIBRARY_PATH=$UDUNITS_PATH/lib:$LD_LIBRARY_PATH export PATH=$UDUNITS_PATH/bin:$PATH export LD_LIBRARY_PATH=$ZLIB_PATH/lib:$LD_LIBRARY_PATH export PATH=$ZLIB_PATH/bin:$PATH CC=gcc CXX='' \ NETCDF_INC=$NETCDF4_PATH/include \ NETCDF_LIB=$NETCDF4_PATH/lib \ NETCDF4_ROOT=$NETCDF4_PATH \ HDF5_LIB_DIR=$HDF5_PATH/lib \ UDUNITS2_PATH=$UDUNITS_PATH \ LDFLAGS="-L$ANTLR_PATH -lantlr \ -lhdf5_hl -lhdf5 -L$NETCDF4_PATH/lib -lnetcdf" \ CFLAGS="-I$HDF5_PATH/include \ -L$HDF5_PATH/lib \ -I$ANTLR_PATH/include \ -L$ANTLR_PATH/lib" \ CPPFLAGS="-I$HDF5_PATH/include \ -L$HDF5_PATH/lib \ -I$ANTLR_PATH/include \ -L$ANTLR_PATH/lib" \ ./configure \ --prefix=$NCO_PATH \ --disable-shared \ --enable-netcdf-4 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install

Thanks for posting these build scripts! I have linked them to the NCO homepage in both the RPM and in the build-from-source sections: http://nco.sf.net#rpm http://nco.sf.net#bld cz

Updated to work with a RHEL6 or CentOS6 'minimal' install. Also added support for C++ and HDF4 reading in NetCDF. antlr-2.7.7 udunits-2.1.24 zlib-1.2.5 hdf-4.2.6 hdf5-1.8.8 netcdf-4.1.3 nco-4.0.8 #!/bin/bash yum groupinstall base yum install -y gcc gcc-c++ gcc-gfortran curl-devel libxml2-devel bison flex wget make compat-gcc-34-g77 libjpeg-devel expat-devel INSTALL_PATH=/usr/local mkdir -p $INSTALL_PATH/src # ANTLR2 APP=antlr-2.7.7 ANTLR_PATH=/usr cd $INSTALL_PATH/src rm -rf antlr-2.7.7* wget http://www.antlr2.org/download/antlr-2.7.7.tar.gz tar xzf antlr-2.7.7.tar.gz ; cd antlr-2.7.7 # Bug in antlr-2.7.7 sed -i "13a #include <strings.h>" lib/cpp/antlr/CharScanner.hpp sed -i "14a #include <cstdio>" lib/cpp/antlr/CharScanner.hpp CC=gcc CXX='' ./configure \ --prefix=$ANTLR_PATH \ --disable-csharp \ --disable-java \ --disable-python 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # UDUNITS APP=udunits-2.1.24 UDUNITS_PATH=$INSTALL_PATH/$APP rm -rf $UDUNITS_PATH cd $INSTALL_PATH/src rm -rf udunits-2.1.24* wget ftp://ftp.unidata.ucar.edu/pub/udunits/udunits-2.1.24.tar.gz tar xzf udunits-2.1.24.tar.gz ; cd udunits-2.1.24 CC=gcc CXX='' F77=gfortran ./configure \ --prefix=$UDUNITS_PATH 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # ZLIB APP=zlib-1.2.5 ZLIB_PATH=$INSTALL_PATH/$APP rm -rf $ZLIB_PATH cd $INSTALL_PATH/src rm -rf zlib-1.2.5* wget http://zlib.net/zlib-1.2.5.tar.gz tar zxf zlib-1.2.5.tar.gz ; cd zlib-1.2.5 CC=gcc FC=gfortran CXX='' ./configure \ --prefix=$ZLIB_PATH | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # HDF4 APP=hdf-4.2.6 HDF4_PATH=$INSTALL_PATH/$APP rm -rf $HDF4_PATH cd $INSTALL_PATH/src rm -rf hdf-4.2.6* wget http://www.hdfgroup.org/ftp/HDF/HDF_Current/src/hdf-4.2.6.tar.gz tar xzf hdf-4.2.6.tar.gz ; cd hdf-4.2.6 # Bug in configure script, need to define EGREP variable here EGREP=egrep CC=gcc FC=gfortran F77=gfortran ./configure \ --prefix=$HDF4_PATH \ --disable-netcdf \ --enable-fortran \ --with-zlib=$ZLIB_PATH \ --with-pic 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install # HDF5 APP=hdf5-1.8.8 HDF5_PATH=$INSTALL_PATH/$APP rm -rf $HDF5_PATH cd $INSTALL_PATH/src rm -rf hdf5-1.8.8* wget http://www.hdfgroup.org/ftp/HDF5/prev-releases/hdf5-1.8.8/src/hdf5-1.8.8.tar.gz tar xzf hdf5-1.8.8.tar.gz ; cd hdf5-1.8.8 CC=gcc FC=gfortran CXX='' ./configure \ --prefix=$HDF5_PATH \ --enable-fortran \ --with-zlib=$ZLIB_PATH \ --with-pic 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install export LD_LIBRARY_PATH=$HDF4_PATH/lib:$LD_LIBRARY_PATH export PATH=$HDF4_PATH/bin:$PATH export LD_LIBRARY_PATH=$HDF5_PATH/lib:$LD_LIBRARY_PATH export PATH=$HDF5_PATH/bin:$PATH export LD_LIBRARY_PATH=$ANTLR_PATH/lib:$LD_LIBRARY_PATH export PATH=$ANTLR_PATH/bin:$PATH export LD_LIBRARY_PATH=$UDUNITS_PATH/lib:$LD_LIBRARY_PATH export PATH=$UDUNITS_PATH/bin:$PATH export LD_LIBRARY_PATH=$ZLIB_PATH/lib:$LD_LIBRARY_PATH export PATH=$ZLIB_PATH/bin:$PATH # NetCDF4 APP=netcdf-4.1.3 NETCDF4_PATH=$INSTALL_PATH/$APP rm -rf $NETCDF4_PATH cd $INSTALL_PATH/src rm -rf netcdf-4.1.3* wget http://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-4.1.3.tar.gz tar xzf netcdf-4.1.3.tar.gz ; cd netcdf-4.1.3 CPPFLAGS="-I$HDF5_PATH/include -I$ZLIB_PATH/include -I$HDF4_PATH/include" \ LDFLAGS="-L$HDF5_PATH/lib -L$ZLIB_PATH/lib -L$HDF4_PATH/lib" \ LIBS="-lmfhdf -lhdf5_hl -lhdf5 -lantlr -ldf -ljpeg -lz" \ CXX=g++ CC=gcc FC=gfortran F77=gfortran ./configure \ --prefix=$NETCDF4_PATH \ --with-hdf5=$HDF5_PATH \ --with-hdf4=$HDF4_PATH \ --enable-fortran \ --enable-static \ --enable-shared \ --enable-f77 \ --enable-f90 \ --enable-netcdf4 \ --enable-hdf4 \ --enable-cxx-4 \ --with-pic 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install export LD_LIBRARY_PATH=$NETCDF4_PATH/lib:$LD_LIBRARY_PATH export PATH=$NETCDF4_PATH/bin:$PATH # NCO APP=nco-4.0.8 NCO_PATH=$INSTALL_PATH/$APP rm -rf $NCO_PATH cd $INSTALL_PATH/src rm -rf nco-4.0.8* wget http://nco.sourceforge.net/src/nco-4.0.8.tar.gz tar xzf nco-4.0.8.tar.gz ; cd nco-4.0.8 CC=gcc CXX='' \ NETCDF_INC=$NETCDF4_PATH/include \ NETCDF_LIB=$NETCDF4_PATH/lib \ NETCDF4_ROOT=$NETCDF4_PATH \ HDF5_LIB_DIR=$HDF5_PATH/lib \ UDUNITS2_PATH=$UDUNITS_PATH \ LDFLAGS="-L$ZLIB_PATH/lib -L$NETCDF4_PATH/lib -L$HDF5_PATH/lib -L$HDF4_PATH/lib -L$ANTLR_PATH/lib -L$UDUNITS_PATH/lib" \ CPPFLAGS="-I$ZLIB_PATH/include -I$NETCDF4_PATH/include -I$HDF5_PATH/include -I$HDF4_PATH/include -I$ANTLR_PATH/include -I$UDUNITS_PATH/include" \ LIBS="-lmfhdf -lhdf5_hl -lhdf5 -lantlr -ldf -ljpeg -lz -ludunits2" \ ./configure \ --prefix=$NCO_PATH \ --disable-shared \ --enable-netcdf-4 2>&1 | tee $APP.config make 2>&1 | tee $APP.make make install 2>&1 | tee $APP.install Output of `/usr/local/netcdf-4.1.3/bin/nc-config --all` This netCDF 4.1.3 has been built with the following features: --cc -> gcc --cflags -> -I/usr/local/testingNCO/netcdf-4.1.3/include -I/usr/local/testingNCO/hdf5-1.8.8/include -I/usr/local/testingNCO/zlib-1.2.5/include -I/usr/local/testingNCO/hdf-4.2.6/include --libs -> -L/usr/local/testingNCO/netcdf-4.1.3/lib -lnetcdf --cxx -> g++ --has-c++ -> yes --fc -> gfortran --fflags -> -g -O2 -I/usr/local/testingNCO/netcdf-4.1.3/include --flibs -> -L/usr/local/testingNCO/netcdf-4.1.3/lib -lnetcdff -lnetcdf --has-f77 -> yes --has-f90 -> yes --has-dap -> yes --has-nc2 -> yes --has-nc4 -> yes --has-hdf5 -> yes --has-hdf4 -> yes --has-pnetcdf-> no --has-szlib -> --prefix -> /usr/local/testingNCO/netcdf-4.1.3 --includedir-> /usr/local/testingNCO/netcdf-4.1.3/include --version -> netCDF 4.1.3

anything special needed for CentOS 6.2...I'm 2 days in.

kwilcox's instructions for RHEL6 should work for CentOs 6.2. I'm curious, though, as to why the pre-built NCO in the EPEL repository is insufficient. There are RPMs in http://dl.fedoraproject.org/pub/epel/6/x86_64/ , and you can install EPEL as a repository in CentOS so that you can just "yum install nco" (directions at: http://fedoraproject.org/wiki/EPEL ) I understand that you might want a more recent version than what's already out there and packaged (4.0.5), though.

Or, it appears that the epel nco was not compiled with support for NetCDF v4. (If I understand correctly.) So, we have to recompile, from source, with "--enable-netcdf-4". And, the CentOS 5 instructions above almost got me there...up to "make" for nco. gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I/usr/local/netcdf-4.1.1/include -I/usr/local/netcdf-4.1.1/include -I/usr/local/hdf 5-1.8.8/include -L/usr/local/hdf5-1.8.8/lib -I/usr/local/antlr-2.7.7/include -L/usr/local/antlr-2.7.7/lib -I/usr/include -I/usr/ local/udunits-2.1.20/include -I/usr/local/hdf5-1.8.8/include -L/usr/local/hdf5-1.8.8/lib -I/usr/local/antlr-2.7.7/include -L/usr /local/antlr-2.7.7/lib -std=c99 -D_BSD_SOURCE -D_POSIX_SOURCE -MT ncap.o -MD -MP -MF .deps/ncap.Tpo -c -o ncap.o ncap.c mv -f .deps/ncap.Tpo .deps/ncap.Po test -f ncap_yacc.c || /bin/sh ../../autobld/ylwrap ncap_yacc.y y.tab.c ncap_yacc.c y.tab.h ncap_yacc.h y.output ncap_yacc.output -- byacc -d --name-prefix=nco_yy byacc: e - line 96 of "/usr/local/src/nco-4.0.6/src/nco/ncap_yacc.y", syntax error %pure-parser ^ make[3]: *** [ncap_yacc.c] Error 1 make[3]: Leaving directory `/usr/local/src/nco-4.0.6/src/nco' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/nco-4.0.6/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/nco-4.0.6' make: *** [all] Error 2 If you see this, byacc is the problem. Install bison (and bison -y will be used by default, if installed). (Not that I'm finished...stuck on not finding -lantler, though it's there... *sigh*)

As far as I know, the NCO homepage instructions for installing the base software needed to build NCO are up-to-date. They include installing bison and flex and some other libraries (http://nco.sf.net#bld): sudo yum install antlr antlr-c++-devel # ANTLR sudo yum install curl-devel libxml2-devel # DAP-prereqs sudo yum install libdap libdap-devel libnc-dap libnc-dap-devel # DAP sudo yum install bison flex gcc g++ # GNU toolchain sudo yum install gsl gsl-devel # GSL sudo yum install netcdf netcdf-devel # netCDF sudo yum install librx librx-devel # RX sudo yum install udunits udunits-devel # UDUnits Updates welcome, as the package names change occasionally...

Ah, I see. Looking at the build logs for EPEL (http://koji.fedoraproject.org/koji/packageinfo?packageID=2612), the latest nco build in their testing repo is 4.0.5, and though there is a successful build, the NCO build process couldn't detect netcdf4, so hence the problem. Good to know. Somehow, the fedora NCO rpms build with netcdf4 just fine, but their build setup refuses to acknowledge netcdf4 on epel. Hmmm.

OK. I hope to have a new programmer on-board by this summer who can modify nco.spec to build on Fedora and RHEL/CentOS. TODO nco 1043. cz