Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

It will be easier to discuss when 2.0.a3 is out, but ... here are  
examples of the kind of stuff I changed in configure.ac:
1) I noticed that the GA_CHECK_LIB routine only checked for the  
presence of a required library, but did not check whether it could  
actually be used in a program. So I changed all those to  
AC_CHECK_LIB, first making sure that the -I and -L flags were limited  
to the supplibs dir.
2) I avoid double checking for libraries that are required by more  
than one feature within the context of the with/without flags. The  
logic had been mostly in line with setting up the different 1.9  
exectuables, whereas now it's focused on enabling features in a  
single build. (Remember gradsdap will be phased out as soon as  
unidata netcdf 4 is opendap-ready.).
3) I added login to skip to dynamic macros if supplibs doesn't exist.
I am not trying to tear apart what you've already done, only trying  
to improve it.

As for the supplibs/include subdirectories, I renamed grib2->grib2c,  
and split dap into gadap and libnc-dap. But the libpng12 name comes  
from their source code, to distinguish from earlier versions, so I  
think we should leave that as is (they renamed the library too,  
libpng12.a).  The remainder of the include directories that weren't  
in my list may be essential for building all the supplibs in a  
coherent way, but they are not necessary for compiling GrADS -- I  
generally extract only what GrADS needs from a complete library  
installation and put that into my supplibs location. I should mention  
that I never build with GUI, so I don't know what's required there.

The pkgconfig files are a very user-friendly; I hadn't looked at them  
before. I was using executables like "ncdap-config --libs" which were  
giving me inconsistent results.

--Jennifer

On Aug 8, 2008, at 9:45 PM, Arlindo da Silva wrote:

> On Fri, Aug 8, 2008 at 11:25 AM, Jennifer Adams <jm...@co...>  
> wrote:
>> Dear All,
>> I have been poring over the configure scripts. The whole thing got
>> complicated quickly trying to prioritize supplibs v. system  
>> libraries and
>> factor in the --with and --enable switches.
>
> I believe you...
>
>>  I quickly realized that it could
>> take a really long time to get it all straightened out, and I  
>> don't have the
>> right expertise to do that anyway. So, to simplify things for  
>> myself, I
>> focused on getting it right when looking in SUPPLIBS, and will  
>> leave the
>> dynamic checking in the capable hands of Pat and Arlindo and  
>> anyone else who
>> contributed to those *.m4 macros. I am eager to put out 2.0.a3, so  
>> I going
>> to release what I've done so far, and that will allow more time  
>> for you guys
>> to make tweaks and adjustments at a more relaxed pace.
>
> I'd need some guidance from you here. The build scripts in tag
> grads-2_0_a2_oga-1 was my best attempt to build GrADS 2 with the
> supplibs and the so-called "dynamic supplibs"; so far as I can tell
> these were working, at least on darwin ppc/intel, Linux i686, x86_64,
> and freebsd (based on passing the test suite).  I did this by adapting
> the stuff Pat had done for v1.9.0-rc1, and he later on revised what I
> did. In converting the v1.9 build I tried to cleanup as much legacy
> stuff as I could, but perhaps I left some debris here or there.
>
> Would I start tweaking it I would probably end up a place not too far
> from the build scripts in 2.0.a2-oga.1 (there are minor C code changes
> from COLA's v2.0.a2 but these are not related to the build.) I'll need
> to understand which of features of the 2.0.a2-oga.1 build you object
> so that I do not repeat the same mistake. Perhaps a face-to-face merge
> exercise would be he most efficient way of getting this done.
>
>> It should build with
>> Arlindo's gigantic supplibs tarball unpacked and compiled, but you'll
>> probably need to set an environment variable SUPPLIBS to point to  
>> it, since
>> configure.ac will only look in the top directory and one level  
>> above for a
>> directory named "supplibs".
>
> The instructions on the wiki tell you to symlink the untarred supplibs
> to the plain "supplibs" name, just as you require.
>
> BTW, are you making use of pkgconfig where appropriate? The
> supplibs-2.0.x are built with their own version of pkg-config and many
> of the packages are instrumented to use it, e.g.,
>
> gadap.pc
> libcurl.pc
> libdap.pc
> libdapclient.pc
> libdapserver.pc
> libpng.pc
> libsx.pc
> libxml-2.0.pc
>
> By using pkgconfig you get all the necessary flags for the build, and
> the *.m4 macros take advantage of it.
>
>> At this point, the biggest noticeable change is
>> that 2.0.a3 will require subdirectories under supplibs/include/  
>> like this:
>> dap/
>> gd/
>> grib2/
>> hdf/
>> libpng12/
>> netcdf/
>> readline/
>> udunits/
>> zlib/
>
> Is this list supposed to be comprehensive? Currently, the
> supplibs-2.0.1 have the following structure under include/
>
> X11/  <====== (neXawt)
> curl/ <===== (needed for build portability)
> gadap/ <=======
> gd/
> grib2c/  <===== (name change)
> hdf/
> jasper/ [grib2c]
> jpeg/ [hdf]
> libdap/ <===== (name change)
> libnc-dap/  <======
> libpng/ <==== (name change)
> libsx/ <===== [athena]
> libxml2/ <==== (needed for build portability [dap])
> ncurses/  <====== (needed for portability [readline])
> netcdf/
> readline/
> szlib/         <======= (needed for *reading* sziped files [hdf])
> udunits/
> zlib/
>
> The "<====" above indicate discrepancies. We have built the
> supplibs-2.0.1 on a number of platforms. If you could preserve the
> names above it would not require us to go back have to rebuild all of
> these again... Not a big deal, but kind of time consuming.
>
>> The configure.ac algorithm is nominally like this:
>> $use_feature=no
>> if $with_feature != no
>>    if (supplibs has required libraries and include files to  
>> support feature)
>> ; then
>>       $use_feature=yes
>>    fi
>>    if $use_feature=no (i.e., not found in supplibs) and
>> $ga_dyn_supplibs=yes, then
>>       use dynamic macros to find libraries
>>    fi
>> fi
>> Forcing the use of dynamic libraries is achieved by not having a  
>> supplibs
>> directory.
>> Forcing the use of ONLY supplibs is achieved by using the  
>> configure switch
>> "--disable-dyn-supplibs".
>> The dynamic macros I copied almost verbatim from the opengrads m4  
>> directory.
>> I may have changed a name or two for clarity. This will be code  
>> that I am
>> going to generally ignore and leave for you guys to support, since  
>> I will
>> always be building from supplibs. But I would like for it to work,  
>> so please
>> send me patches and changes and I will check them in.
>> The --with-* arguments are trickier, since there are some that  
>> should be
>> just yes/no and others that provide directory locations. The
>> --with-package=yes/no switches that I used are:
>>   --with-readline         command line editing
>>   --with-printim          image output
>>   --with-grib2            GRIB2 data format
>>   --with-sdf              self-describing formats (HDF and NetCDF)
>>   --with-dap              extra gradsdap executable, OPeNDAP-enabled
>>   --with-gui              Athena X11 widget-based GUI
>>  The remainders are in the dynamic *.m4 macros. I used --with-sdf  
>> to avoid
>> conflicts with --with-hdf and --with-netcdf arguments in the dynamic
>> macros.
>> I generally avoid the -l<libname> syntax when using supplibs. The  
>> only
>> exceptions are the -lncurses/-ltermcap thing with readline and  
>> linking with
>> libdap and libnc-dap always requires a few extra libs.
>
> pkgconfig usually takes care of these details.
>
>> I put libcurl.a and
>> libxml2.a in supplibs/lib, but there are still others, and these  
>> are not the
>> same for every OS. The dynamic macros use the output from "ncdap- 
>> config
>> --libs", but on my mac this left out some of the required ones so  
>> I have
>> hard-coded a kludge:
>>     if test "$is_darwin" = "yes" ; then
>>       dap_extra_libs="-lssl -lcrypto -lpthread -liconv -lm"
>>     else
>>       dap_extra_libs="-lidn -lssl -lcrypto -ldl -lpthread -lm"
>>     fi
>>     dap_libs="$gadap_lib $dap_libs $dap_extra_libs"
>> This syntax gives me what I need to build on our mac and linux  
>> boxes, but
>> will probably be wrong for sunOS and others. Since we have checks for
>> host-specific options, any platform-specific dap_extra_libs can be  
>> added as
>> necessary.
>
> Consider using pkgconfig. None better than the library developers to
> know what is required for a given version; pkgconfig provides the
> mechanism for them to convey this information.
>
>> Finally, I had wanted to include library version numbers in the 'q  
>> config'
>> output, but that turned out to be tricky if the library did not  
>> contain a
>> routine for spitting out a version string. Netcdf has  
>> nc_inq_libvers(), but
>> many of the others simply have #define statements in the header  
>> files, but
>> this was highly version-dependent and will cause problems if using  
>> system
>> libraries instead of supplibs. I left some of that code in gacfg.c  
>> but other
>> bits are commented out. If you guys have any ideas for how to do  
>> this, maybe
>> there's a way to do it during the configuration process and  
>> populate GrADS's
>> own config.h with strings like #define GA_ZLIB_VERSION 1.2.3 ... then
>> gacfg.c need only look in config.h to get that info.
>
> The OpenGrADS supplibs have a tag corresponding to each supplib
> version, say 2.0.1 (the one I use to build grads2). In the very least,
> gacfg.c could report the supplibs version used for the build. I may
> need to add a way for this version number to be retrieved...
>
> For tracking the version of the individual packages, as long as the
> packages are using autoconf (true for supplibs-2.0.x) you can get the
> version number by parsing configure.ac/configure.in during the
> supplibs build, e.g.,
>
> % grep ^AC_INIT netcdf/configure.ac
> AC_INIT([netCDF], [3.6.2], [su...@un...])
>
> A simple script could added to the supplibs build to compile this
> information in a simple text file, say "supplibs.h". Are you willing
> to adopt (and help maintain) the supplibs-2.0.x?
>
>> Well, that's enough. If you've read this far, I thank you  
>> sincerely for your
>> attention to these arcane details.
>
> This is the reality of autoconf.
>
>    Arlindo
>
> -- 
> Arlindo da Silva
> da...@al...
>
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...