Issues with building Cygwin NCO from source

2013-11-18
2013-11-24
  • Mark Hadfield

    Mark Hadfield - 2013-11-18

    Hello all

    On my new PC I have 64-bit and 32-bit installations of Cygwin, with the latter to be kept until the former is fully functional.

    On Cygwin-32, NCO (most recently version 4.3.8) builds OK, except that configure does not find any DAP supprt. This is odd, as my netCDF installation (the system package, up-to-date) does support DAP, eg., "nc-config --has-dap" reports "yes" and ncdump can read external URLs. And, to the best of recollection I have built NCO on this platform with DAP support many times in the past. If I can find the time, I will attempt to sort this one out.

    On Cygwin-64, DAP support is found, however make fails with errors like this

    /bin/sh ../../libtool --tag=CXX  --tag=CC   --mode=compile gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../..   -I/usr/include -DHAVE_NETCDF4_H -I/usr/include/gsl -I/usr/include -I/usr/include  -g -O2 -std=c99 -D_BSD_SOURCE -D_POSIX_SOURCE -MT nco_lmt.lo -MD -MP -MF .deps/nco_lmt.Tpo -c -o nco_lmt.lo nco_lmt.c
    In file included from nco_fl_utl.h:41:0,
                     from nco_fl_utl.c:9:
    /usr/include/arpa/nameser.h:121:2: error: unknown type name 'u_char'
    

    Other unknown types that are reported are u_int, u_short and u_long.

    I'm afraid this one is a bit beyond a mere scientist like myself.

    Mark

     
  • Charlie Zender

    Charlie Zender - 2013-11-18

    Hi Mark,

    The workaround on the 64-bit system may be to ensure that the token WIN32 is defined (even though
    it is 64-bits). e.g., you need -DWIN32 in each compile command. We should probably change the name of that token to WIN32OR64 in a future release. Let us know if that works. As to why it is not defined, that is another question we need to address.

    cz

     
  • Mark Hadfield

    Mark Hadfield - 2013-11-19

    Thanks Charlie

    I built NCO 4.3.8 with the following configure command

    LIBS=$(nc-config --libs) CPPFLAGS="-DWIN32 -DHAVE_NETCDF4_H -I/usr/include/gsl" NETCDF4_ROOT=/usr nice ./configure --disable-shared --enable-netcdf4 --enable-dap-netcdf --disable-nco_cplusplus --disable-ncap2 --prefix=/usr/local
    

    Compilation, linking and installation all (appear to) work fine, however the executables don't run:

    $ ncks
    : ERROR executable name  not registered in nco_prg_prs()
    $ ncea
    : ERROR executable name  not registered in nco_prg_prs()
    $ ncwa
    : ERROR executable name  not registered in nco_prg_prs()
    
     
  • Charlie Zender

    Charlie Zender - 2013-11-19

    Aaarrrggghhhhh :) The only imaginable cause of this is the presence of older versions of NCO shared libraries. In other words, before you build 4.3.8, delete all previous NCO source code and object code and libraries and executables. 'make distclean' and 'make uninstall' may be helpful.
    cz

     
  • Mark Hadfield

    Mark Hadfield - 2013-11-19

    Hmmmmmmmmm :) I tried clearing out everything that could conceivably matter and rebuilding from scratch, but the result was the same.

    I then investigated the issue further, using the secret debugging trick of adding print statements to the code, rebuilding and rerunning. I added such a statement after line 818 of nco_clt.c, thus

    nm_out_orig=nm_out_tmp=(char *)strdup(nm_in);
    (void)fprintf(stdout,"nm_in, nm_out_tmp, nm_out_orig: %s,%s,%s\n",nm_in,nm_out_tmp,nm_out_orig);
    

    The executable then printed the following output

    $ ncks
    nm_in, nm_out_tmp, nm_out_orig: ncks,,
    : ERROR executable name  not registered in nco_prg_prs()
    

    In other words, the contents of nm_in have not been copied into nm_out_tmp and nm_out_orig. Presumably because the strdup function is not working. Looking back at some harmless-looking warnings I got during compilation, I see this

    nco_ctl.c: In function 'nco_lbr_vrs_prn':
    nco_ctl.c:649:3: warning: implicit declaration of function 'strdup' [-Wimplicit-function-declaration]
       lbr_sng=(char *)strdup(nc_inq_libvers());
                ^
    

    hmmmmmmmmmmmmmmmm :) Maybe those warnings were not so harmless after all. strdup is a POSIX function, is it not? Has setting WIN32 prevented it from being defined?

    BTW, the following links include some discussion of porting to Cygwin-64 and the associated preprocessor tokens.

    http://cygwin.com/faq/faq.html#faq.programming.64bitporting-fail
    http://cygwin.com/faq/faq.html#faq.programming.64bitporting-cygwin64

    Since Cygwin attempts as far as possible to duplicate Unix, it seems to me that very little Windows-specific code should be required on this platform. But what would I know?

     
  • Charlie Zender

    Charlie Zender - 2013-11-20

    Mark,
    Good job tracking this down. Apparently my imagination is not vivid enough to think of bugs like this.
    Pedro: please install cygwin-64 on your Windows laptop, try to replicate the problem, then bring your laptop up (with emacs installed in cygwin) and we'll try to fix it. We can fix this problem by fiddling with tokens if I have a working cygwin-64 environment to test it with.
    cz

     
  • Pedro Vicente

    Pedro Vicente - 2013-11-21

    Mark

    We did some changes for cygwin-64 to the sources. Can you try the following?

    1) get the new sources

    cvs -z3 -d:pserver:anonymous@nco.cvs.sf.net:/cvsroot/nco co -kk nco

    2) There is no antlr package for cygwin, you can download the source from here

    svn co http://glace.ess.uci.edu/svn/antlr-2.7.7

    3) define this environment variable to build with antlr

    ANTLR_ROOT=/your/antlr/install/location

    4) to run the NCO tests define these environment variables

    DATA=/your/data_nco
    MY_BIN_DIR=/your/nco_install/bin

    5) build NCO

    ./configure --prefix=/your/nco_install
    make
    make install
    make test

     
  • Mark Hadfield

    Mark Hadfield - 2013-11-21

    Thanks. I will do this as soon as I can. Getting CVS to work through our proxy server has been a problem in the past, but I've just recalled that I sorted this out for the MITgcm model, so I should be able to adapt the procedure here.

    Which reminds me of something I've been meaning to suggest. Given that...

    • The antlr required by NCO is an old version (2.7.7)

    • Only a subset of antlr is used

    • Some Cygwin-specific or Win32-specific or NCO-specific patches are required

    ...wouldn't the simplest solution be to include the antlr source in the NCO distribution?

     
  • Mark Hadfield

    Mark Hadfield - 2013-11-22

    I managed to download the NCO source code with CVS and build & install it on Cygwin-64. I haven't tried the antlr/ncap2 stuff yet: I'll get to it when everything else is sorted out.

    I did notice some "implicit declaration" messages on the way through, eg:

    libtool: compile:  gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I/usr/include -DHAVE_NETCDF4_H -I/usr/include -I/usr/include -g -O2 -std=c99 -D_BSD_SOURCE -D_POSIX_SOURCE -MT nco_grp_trv.lo -MD -MP -MF .deps/nco_grp_trv.Tpo -c nco_grp_trv.c -o nco_grp_trv.o
    nco_fl_utl.c: In function 'nco_create_mode_prs':
    nco_fl_utl.c:58:3: warning: implicit declaration of function 'strcasestr' [-Wimplicit-function-declaration]
       if(strcasestr("classic",fl_fmt_sng) && !strcasestr(fl_fmt_sng,"netcdf4")){
       ^
    nco_fl_utl.c: In function 'nco_fl_out_open':
    nco_fl_utl.c:1461:5: warning: implicit declaration of function 'mkstemp' [-Wimplicit-function-declaration]
         fl_out_hnd=mkstemp(fl_out_tmp_sys);
         ^
    mv -f .deps/nco_dmn_utl.Tpo .deps/nco_dmn_utl.Plo
    

    Also, some experimentation with ncks indicates that it works on local files, but segfaults on OpenDAP urls, eg:

    $ ncks http://tds.hycom.org/thredds/dodsC/GLBa0.08/expt_91.0
    Segmentation fault (core dumped)
    

    However I also see a segfault with "ncdump -h" on the same URL, though it occurs after printing out the file header, so the netCDF package is a bit suspect. I may have to go back to building netCDF from source.

     
    • Mark Hadfield

      Mark Hadfield - 2013-11-24

      There is definitely a problem with the Cygwin-64 netCDF library segfaulting on OpenDAP URLs. I have built version 4.31.-rc4 from source and it shows the same problem. I have reported it to netCDF support (JVN-123940) and I'll report here on any progress.

       
  • Charlie Zender

    Charlie Zender - 2013-11-22

    Thanks, Mark. Any cygwin problems are surmountable with time and access to a cygwin machine. If Pedro will install cygwin(32) an cygwin64 and emacs and xwindows on his laptop, and check out the NCO with CVS on each, then I think we can debug the NCO builds in an afternoon. It's mostly just a matter of passing the right pre-processor tokens.
    cz

     
  • Charlie Zender

    Charlie Zender - 2013-11-22

    Regarding including Antlr files directly in NCO:
    We have certainly considered this.
    It means adding an entire directory of Antlr C++ headers.
    And we still also need to link to libantlr at compile-time.
    So, basically it's an albatross and moving the problem inside NCO won't change that.
    c

     
  • Mark Hadfield

    Mark Hadfield - 2013-11-22

    I just thought it's easier to carry an albatross around your neck than to go out and shoot it every time you need it. :-) But it's your call.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks