Autoconf ready for prime time?
Command-line operators for netCDF and HDF files
Brought to you by:
zender
Menu
▾
▴
Hi Rorik,
Today for the first time I tried your autoconf NCO
changes (from about July 3, actually) and they
worked really well.
I did this on my home linux system, and just used
./configure --prefix=/data/zender/tmp ; make ; make install
I was very impressed that it just worked because I expected
problems. It appears to build the executables and install the
binaries correctly. What are the remaining items to accomplish
before you feel it is ready for primetime?
Here are a few things I can think of
(some of these may already be done):
1. installation of man pages
2. enable make check so it runs nco_tst.sh
3. more testing on non-linux platforms
4. resolution of the ABI=64 question
5. test that DODS-enabled builds work
6. automatic building/testing of libnco_c++ directory
if working C++ compiler is detected
7. cleaning up configure.in
None of these are real show stoppers since we can
keep the existing bld/Makefile method as long as
necessary. I have added you to the NCO developers
list. When you feel the autoconf stuff is good enough,
I'd like you to go ahead and start merging your changes
into the main trunk. That is, if you feel comfortable doing
that. If you don't then I can do the merging if you tell
me what needs merging.
Thanks,
Charlie
I just got back today from spending a couple weeks in Boulder at Unidata. I've actually improved the configure some more from the one you tried, although I haven't decided which way to go with the ABI=64 thing yet.
> 1. installation of man pages
It does this now, in /usr/local/man. We can change that if need be.
> 2. enable make check so it runs nco_tst.sh
Does this as well, but I'm not happy with the way it outputs the results.
> 3. more testing on non-linux platforms
I can use the Sun Ultra60 with Solaris 8, but that is all I have. Fortunately, it can do both 32/64 bit compile, so I've been testing my ABI tests with it, and they seem to work. You'll have to help out with the AIX stuff.
> 4. resolution of the ABI=64 question
yep, I'm still undecided after our last exchange.
> 5. test that DODS-enabled builds work
This works. You can export $DODS_HOME ahead of time if the libraries are not in your default linking path. It's documented in ./configure --help. I've built both with and without.
> 6. automatic building/testing of libnco_c++ directory
> if working C++ compiler is detected
This seems to work as well, although I'll add the conditional. I've never tried actually using the compiled library however.
> 7. cleaning up configure.in
Yep. I'm working on commenting because it is downright cryptic otherwise.
I'll look over it again tonight since it has been a little while since I've seen it. If it seems to be at a functional state, I'll look into uploading. Fortunately, the changes will really only be the addition of several files at the same level at src/ doc/ man/ etc. I think I've only changed one line in a lex file (however, the lex/yacc filenames need to be changed.)
More tomorrow.
rorik
It sounds like things are far enough along to start merging
into the CVS repository. This is good news!
> > 1. installation of man pages
> It does this now, in /usr/local/man. We can change that if need be.
I would say man pages should go in $INSTALL_DIR/man where install dir is
argument to ./configure --prefix=$INSTALL_DIR, if any, and default
$INSTALL_DIR is /usr/local.
> > 2. enable make check so it runs nco_tst.sh
> Does this as well, but I'm not happy with the way it outputs the results.
I'm not sure what you're not happy with, but feel free to change
the formatting of the nco_tst.sh results to something you like.
> > 3. more testing on non-linux platforms
> I can use the Sun Ultra60 with Solaris 8, but that is all I have. Fortunately, it can do both 32/64 bit compile, so I've been testing my ABI tests with it, and they seem to work. You'll have to help out with the AIX stuff.
Once we get the autotool stuff merged into CVS then I will try
./configure commands on all the platforms I have access to.
> > 4. resolution of the ABI=64 question
> yep, I'm still undecided after our last exchange.
>
> > 5. test that DODS-enabled builds work
> This works. You can export $DODS_HOME ahead of time if the libraries are not in your default linking path. It's documented in ./configure --help. I've built both with and without.
Great
> > 6. automatic building/testing of libnco_c++ directory
> > if working C++ compiler is detected
> This seems to work as well, although I'll add the conditional. I've never tried actually using the compiled library however.
There is a small test program named 'tst.cc' that is built in that
directory, building and running the 'tst' executable
should occur as part of 'make check'.
> > 7. cleaning up configure.in
> Yep. I'm working on commenting because it is downright cryptic otherwise.
Good, that will make it easier for me to make the small changes
that may be required for the platforms that only I have access to.
> I'll look over it again tonight since it has been a little while since I've seen it. If it seems to be at a functional state, I'll look into uploading. Fortunately, the changes will really only be the addition of several files at the same level at src/ doc/ man/ etc. I think I've only changed one line in a lex file (however, the lex/yacc filenames need to be changed.)
OK. For now just upload the stuff that is _required_ for autotools
to function properly and leave out the files like
AUTHORS/NEWS/COPYING/INSTALL/README.
One thing that bugs me about autotools is the mess they create in
the top-level directory!
Thanks,
Charlie
Concerning top-level dir clutter -
autoconf definitely generates lots of stuff. We can put all the helper scripts (install-sh, missing, etc) in a separate dir called, say, scripts/. Then if we ignore the AUTHORS/NEWS/COPYING/INSTALL/README stuff for now, additions to the top level would include:
configure configure.in config.h.in Makefile.in Makefile.am scripts/
That's the minimum I can determine right now, but then again, I'm still learning. I think it is good to have the 'configure' executable in the top since that seems to be sort of standard. None of this touches the bld/ dir, so the current make procedure will be unaffected.
Yes, that's fine.
I agree that the ./configure stuff should stay in the top
level dir. Let's call the new dir for the scripts autobld
instead of scripts, for now.
Note that I added direct support for the NEC-SX
architecture last night, so that's a new section
in the Makefile/pvmgetarch (NECSX).
Charlie
I uploaded several files for using the GNU autotools. There are now two makefile skeletons in most directories that configure uses. I put the other supporting scripts in autobld/ to reduce the clutter. The top directory also has a few other files that autoconf and configure need. It still needs a lot of work, but at least we have something to work with.
If you change the Makefiles at all, I used automake 1.4, and it is important use use the flags --include-deps --foreign so the dependencies are created and several GNU-recommended files are not required.
If use use ./configure --help, you can see the options I've included so far. For --enable, there is debug, extreme, optimize, hdf5, and DODS support.
'make check' runs the test program in nco_c++, but you have to link 'in.nc' first. We could do that automatically, but it is probably better for the test program to open ../../data/in.nc instead.
The man pages will be install on 'make install', and the texinfo files will be created and installed in you have the correct software. We should probably put some form of the documentation in a post-processed form on CVS for those that don't have texinfo.
All the configuration is done using the host type detected by configure. This is output at the end of configure for debugging. If you look in configure.in, you'll see case structures for several architectures I could try and guess lined up with the current Makefile. However, to really get it right, we just need to use 'configure' on different architectures and see what $host comes out. It all works on my GNU/Linux machine and Solaris8.
There is an attempt at detecting the ABI. Configure first links with libnetcdf.a using the default compiler flags, if that fails, it adds 64-bit flags if we know them. Again, they are listed in a case structure in configure.in based on host type. Then it tries to link with libnetcdf.a again. If both fail, configure fails; without libnetcdf.a NCO is useless. Again, it works on my Solaris8 machine on 32 and 64-bit, but someone else will need to test it on other machines.
To help me out, the best thing is to test on other machines, and check the configuration parameters output at the end of running configure. If they are wrong, let me know what they are and what they should be, or change configure.in directly.
You only need autoconf if you change configure.in, but you need automake and aclocal if you change the Makefiles.
This will take some work, but I think it wil pay off in the end.
rorik
Hi rorik,
Welcome to nco development team! I'm Henry I've been working on and off for 2 years on the nco software - My chief claim to fame is the development is ncap. If you have any queries, complaints or fresh ideas about it please let me know.
I checked out fresh source today.
/.configure ran ok. But when I did a top level make I got the following messages:
/bin/sh ./config.status --recheck
running /bin/sh ./configure LDFLAGS=-L/usr/src/redhat/SOURCES/netcdf-3.5.0/src/libsrc --no-create --no-recursion
cd . && autoheader
configure.in:6: error: Autoconf version 2.53 or higher is required for this script
configure.in:6: the top level
autoconf: tracing failed
make: *** [stamp-h.in] Error 1
( I have autoconf 2.52)
I am abit confused because I thought autoconf was only needed the the configure script was regenerated.
I am running RedHat 6.1 on a pentium P200MMX
Would it be helpful if I sent you the configure.log and configure.status files ? what is your e-mail address?
Mine is henryb@ntlworld.com
regards Henry
Henry,
I removed the autoconf2.53 requirement; it shouldn't have been in there in the first place. You're correct that you should not need autoconf, automake, aclocal, etc. unless you change configure.in. It worked on a machine with no autotools at all, you'll have to let me know if it works for you now. Things can get screwy if you have versions of autoconf, but not the one that created the files. We can fix that, however.
rorik
ps: email is ffrap1 at uaf dot edu
Actually, there looks to be more problems with configure. It may or may not work for you now. This may take me a little while to sort out; so go ahead and ignore it for now. I'll post when I think I've got in a usuable form.
Concerning ncap: to get autoconf to work, I had to rename ncap.l and ncap.y to ncap_lex.l and ncap_yacc.y because autoconf wants to rename all lex and yacc files after it processes them. I just copied the existing versions to new names and put them in src/nco/; you may have noticed them. Hopefully they are not causing any problems right now, and we can consider renameing them AFTER I get autoconf to work; big IF here.
rorik
I added the maintainer mode to configure (--enable-maintainer-mode), so unless you give that, you shouldn't be bothered with any autotools stuff regardless of the versions.
However, I noticed that the current Makefile also moves the lex-generated c-file lex.yy.c to ncap_lex.c. This can cause a problem if you use ./configure; make; make clean; and then cd to bld/ and make there. This is because 'make clean' does not remove the file ncap_lex.c, however, the autoconf-generated one will not work with the current makefile because they are generated from different directories. You'll get an error that it can't find the header files. if you remove the autoconf-generated ncap_lex.c before 'make', flex will generate a new one that works.
The reverse does not appear to be a problem.
I am trying to help...really.
rorik
Hi,
Good to see the autobuild merge is now comitted!
I checked in a fix for a deep bug in ncra, then
rewrote bld/Makefiel to use ncap_lex.l and ncap_yacc.y
so now both build mechanisms should live happily
together for awhile. Rorik, I'm sure we'll continue finding
little tweaks that should be made for a while, but it's
amazing that your mods appear to work on most
platforms out of the box. Soon maybe we'll have
a Mac OS X port!
Charlie