opengrads-devel

[Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@CO...>]

Dear Experts,

I've been pecking away at the configure scripts and thinking about  
the logic in them. There's a lot of cruft in there -- support of  
"historical" stuff that is no longer relevant, and outdated platform  
dependencies. It seems to me that the default behavior should be to  
look for a directory called "supplibs" (in ./ and ../) and if not  
found then go hunting in the system for the required libraries,  
creating a dynamically linked executable. To create a more portable  
build, the configure flag "--disable-shared" will stick with what's  
in the "supplibs"  directory and use the "-L/ga_supplb_path/ 
libname.a" syntax instead of -L/path -lname" .  The "--with-PACKAGE"  
configure options will be used to guide the hunting for a specific  
library, something like --with-hdf=/opt/local (and then it will look  
for /opt/local/lib/libmfhdf.a and /opt/local/include/mfhdf.h)

Doesn't that seem more reasonable?

Jennifer

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Patrice D. <per...@fr...>]

On Fri, Aug 01, 2008 at 08:09:12AM -0400, Jennifer Adams wrote:
> Dear Experts,
>
> I've been pecking away at the configure scripts and thinking about the 
> logic in them. There's a lot of cruft in there -- support of  
> "historical" stuff that is no longer relevant, and outdated platform  
> dependencies. 

Do you see that in configure.in/acinclude.m4 from, for example, 
grads-2.0.a2, or in the macros used with --enable-dyn-supplibs in 
opengrads? I don't see that in configure.in/acinclude.m4. It is
present in opengrads, but it is not an issue, in my opinion. On the
contrary, it allows to build grads against old libdods, for example,
which is possible and should not be dropped in my opinion. Also those
macros in fact come from other sources and should be kept synchronized
with these sources (mostly opendap).

> It seems to me that the default behavior should be to look 
> for a directory called "supplibs" (in ./ and ../) and if not found then 
> go hunting in the system for the required libraries, creating a 
> dynamically linked executable. To create a more portable build, the 

Does this means that --enable-dyn-supplibs is the default unless a
./supplib or ../supplib directory exists, and if a ./supplib or ../supplib
directory exists one has to add --enable-dyn-supplibs to link against
system libraries? It it is what you meant, this seems reasonnable to me.

> configure flag "--disable-shared" will stick with what's in the 
> "supplibs"  directory and use the "-L/ga_supplb_path/libname.a" syntax 
> instead of -L/path -lname" .  

That is not clear to me. What would the difference be between a build 
with "--disable-shared" and a build with a ./supplib or ../supplib
directory present would be?

> The "--with-PACKAGE" configure options will 
> be used to guide the hunting for a specific library, something like 
> --with-hdf=/opt/local (and then it will look for 
> /opt/local/lib/libmfhdf.a and /opt/local/include/mfhdf.h)

Agreed. It could even be used to use some supplibs but not all.
--without-* could still be used to disable a feature.


I don't understand clearly, however, if you want to use the same macros
to detect the system libraries and the supplib libraries. If it is the 
case it won't be that easy to achieve, because linking staticaly
against a supplib means supplying the path of the static 
library or enclose the static link in -Wl,-static -lname -Wl,-dy (for
gcc, at least). This wouldn't easily be done with the system library 
detection macros.

--
Pat

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

On Aug 1, 2008, at 8:55 AM, Patrice Dumas wrote:

> On Fri, Aug 01, 2008 at 08:09:12AM -0400, Jennifer Adams wrote:
>> Dear Experts,
>>
>> I've been pecking away at the configure scripts and thinking about  
>> the
>> logic in them. There's a lot of cruft in there -- support of
>> "historical" stuff that is no longer relevant, and outdated platform
>> dependencies.
>
> Do you see that in configure.in/acinclude.m4 from, for example,
> grads-2.0.a2, or in the macros used with --enable-dyn-supplibs in
> opengrads?
The cruft I'm talking about is anything related to the CRAY, support  
for the "wi" command, directory names called "supplibs-sol55", stuff  
like that.

> I don't see that in configure.in/acinclude.m4. It is
> present in opengrads, but it is not an issue, in my opinion. On the
> contrary, it allows to build grads against old libdods, for example,
> which is possible and should not be dropped in my opinion.
I don't want GrADS 2.0 to keep trying to support libraries that  
depend on old versions of other libraries and compilers. My remedy  
for GrADS's bad case of versionitis is to forget the past and make it  
work with current versions of the dependent libraries. For example, I  
need to make sure I've got HDF4 built with the --disable-netcdf  
option, and that option is working properly in HDF4.2r3 (this version  
has finally eliminated all dependence on netcdf.h  -- oh, If I had a  
dollar for every pain that netcdf.h has caused me ... I'd get a free  
lunch!) The whole DODS mess is the worst offender. libgadods, libdap+ 
+, libnc-dods, RIP.  Can't wait till libnc-dap.a is swept up into  
libnetcdf.a.

> Also those
> macros in fact come from other sources and should be kept synchronized
> with these sources (mostly opendap).
The dyn-supplibs macros are the ones I want to bring into the GrADS  
mainstream, to be used by default.

>
>> It seems to me that the default behavior should be to look
>> for a directory called "supplibs" (in ./ and ../) and if not found  
>> then
>> go hunting in the system for the required libraries, creating a
>> dynamically linked executable. To create a more portable build, the
>
> Does this means that --enable-dyn-supplibs is the default unless a
> ./supplib or ../supplib directory exists, and if a ./supplib or ../ 
> supplib
> directory exists one has to add --enable-dyn-supplibs to link against
> system libraries? It it is what you meant, this seems reasonnable  
> to me.

I want ./supplibs to be the first place to look for libraries, then  
if they don't exist there, go hunting for them in the system. If a  
user knows where a library is, but it's not in ./supplibs, then the -- 
with configure flag is used to guide the hunting.

>
>> configure flag "--disable-shared" will stick with what's in the
>> "supplibs"  directory and use the "-L/ga_supplb_path/libname.a"  
>> syntax
>> instead of -L/path -lname" .
>
> That is not clear to me. What would the difference be between a build
> with "--disable-shared" and a build with a ./supplib or ../supplib
> directory present would be?

You're right, there would be no difference if "supplibs"  always  
trumps the system location. I guess the --enable-dyn-supplibs flag  
could disable the check for ./supplibs or ../subblibs and then system  
location would trump "supplibs".

>
>> The "--with-PACKAGE" configure options will
>> be used to guide the hunting for a specific library, something like
>> --with-hdf=/opt/local (and then it will look for
>> /opt/local/lib/libmfhdf.a and /opt/local/include/mfhdf.h)
>
> Agreed. It could even be used to use some supplibs but not all.
> --without-* could still be used to disable a feature.

Yes, the --without flag could save configuration time if you don't  
care about enabling a certain feature. The HDF group has a nice model  
for configuration with dependent libraries. I am going to borrow some  
stuff from them and separate out each library that is required. I'll  
look for libz.a, if not found, I can skip looking for a lot of other  
libraries (png, gd, hdf, grib2). If I do find it, add it to my list  
of libs and includes and continue, enabling features as I go through  
the list of libraries.


> I don't understand clearly, however, if you want to use the same  
> macros
> to detect the system libraries and the supplib libraries. If it is the
> case it won't be that easy to achieve, because linking staticaly
> against a supplib means supplying the path of the static
> library or enclose the static link in -Wl,-static -lname -Wl,-dy (for
> gcc, at least). This wouldn't easily be done with the system library
> detection macros.
Hmm.  Let's consider the readline case. Current code is :

   if test "$ga_dyn_supplibs" = "yes" ; then
     GA_LIB_READLINE([use_readline=yes readline_libs=""])
   else
     GA_CHECK_READLINE([use_readline=yes])
   fi

What if I changed that to something like:

   if test "$ga_dyn_supplibs" = "yes" ; then
     GA_DYN_READLINE([use_readline=yes readline_libs=""])
   else
     GA_SUPPLIBS_READLINE([use_readline=yes],[use_readline=no])
     if teset "$use_readline" = "no" ; then
       GA_DYN_READLINE([use_readline=yes readline_libs=""])
     fi
   fi

Where GA_SUPPLIBS_READLINE only looks in ./supplibs and always uses  
the static link -L/supplibs/libname.a

Would that produce the desired result?

Jennifer

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

One thing needs clarification in the logic.

--enable-dyn-supplibs    never look for "supplibs" and always use DYN  
macros to look for libraries
--disable-dyn-supplibs    always use "supplibs" and never use DYN  
macros to look for libraries

The problem is that neither of these represents the default behavior,  
which is to use "supplibs" first and then use DYN macros to look for  
libraries not in "supplibs". Maybe we should have two enable/disable  
flags, one for looking in "supplibs" and one for using DYN macros,  
both of which are on by default. The thing is, the --enable-supplibs  
flag would be redundant if you set up a "supplibs" directory in the  
first place since it will not exist by default. If the user goes to  
the trouble to set up "supplibs", I guess it's safe to assume we  
should use those libraries instead of system versions, unless  
otherwise directed. So maybe the two options above are what we need,  
even they don't really represent a yes/no flag.

Jennifer



On Aug 1, 2008, at 9:59 AM, Jennifer Adams wrote:

>
> On Aug 1, 2008, at 8:55 AM, Patrice Dumas wrote:
>
>> On Fri, Aug 01, 2008 at 08:09:12AM -0400, Jennifer Adams wrote:
>>> Dear Experts,
>>>
>>> I've been pecking away at the configure scripts and thinking about
>>> the
>>> logic in them. There's a lot of cruft in there -- support of
>>> "historical" stuff that is no longer relevant, and outdated platform
>>> dependencies.
>>
>> Do you see that in configure.in/acinclude.m4 from, for example,
>> grads-2.0.a2, or in the macros used with --enable-dyn-supplibs in
>> opengrads?
> The cruft I'm talking about is anything related to the CRAY, support
> for the "wi" command, directory names called "supplibs-sol55", stuff
> like that.
>
>> I don't see that in configure.in/acinclude.m4. It is
>> present in opengrads, but it is not an issue, in my opinion. On the
>> contrary, it allows to build grads against old libdods, for example,
>> which is possible and should not be dropped in my opinion.
> I don't want GrADS 2.0 to keep trying to support libraries that
> depend on old versions of other libraries and compilers. My remedy
> for GrADS's bad case of versionitis is to forget the past and make it
> work with current versions of the dependent libraries. For example, I
> need to make sure I've got HDF4 built with the --disable-netcdf
> option, and that option is working properly in HDF4.2r3 (this version
> has finally eliminated all dependence on netcdf.h  -- oh, If I had a
> dollar for every pain that netcdf.h has caused me ... I'd get a free
> lunch!) The whole DODS mess is the worst offender. libgadods, libdap+
> +, libnc-dods, RIP.  Can't wait till libnc-dap.a is swept up into
> libnetcdf.a.
>
>> Also those
>> macros in fact come from other sources and should be kept  
>> synchronized
>> with these sources (mostly opendap).
> The dyn-supplibs macros are the ones I want to bring into the GrADS
> mainstream, to be used by default.
>
>>
>>> It seems to me that the default behavior should be to look
>>> for a directory called "supplibs" (in ./ and ../) and if not found
>>> then
>>> go hunting in the system for the required libraries, creating a
>>> dynamically linked executable. To create a more portable build, the
>>
>> Does this means that --enable-dyn-supplibs is the default unless a
>> ./supplib or ../supplib directory exists, and if a ./supplib or ../
>> supplib
>> directory exists one has to add --enable-dyn-supplibs to link against
>> system libraries? It it is what you meant, this seems reasonnable
>> to me.
>
> I want ./supplibs to be the first place to look for libraries, then
> if they don't exist there, go hunting for them in the system. If a
> user knows where a library is, but it's not in ./supplibs, then the --
> with configure flag is used to guide the hunting.
>
>>
>>> configure flag "--disable-shared" will stick with what's in the
>>> "supplibs"  directory and use the "-L/ga_supplb_path/libname.a"
>>> syntax
>>> instead of -L/path -lname" .
>>
>> That is not clear to me. What would the difference be between a build
>> with "--disable-shared" and a build with a ./supplib or ../supplib
>> directory present would be?
>
> You're right, there would be no difference if "supplibs"  always
> trumps the system location. I guess the --enable-dyn-supplibs flag
> could disable the check for ./supplibs or ../subblibs and then system
> location would trump "supplibs".
>
>>
>>> The "--with-PACKAGE" configure options will
>>> be used to guide the hunting for a specific library, something like
>>> --with-hdf=/opt/local (and then it will look for
>>> /opt/local/lib/libmfhdf.a and /opt/local/include/mfhdf.h)
>>
>> Agreed. It could even be used to use some supplibs but not all.
>> --without-* could still be used to disable a feature.
>
> Yes, the --without flag could save configuration time if you don't
> care about enabling a certain feature. The HDF group has a nice model
> for configuration with dependent libraries. I am going to borrow some
> stuff from them and separate out each library that is required. I'll
> look for libz.a, if not found, I can skip looking for a lot of other
> libraries (png, gd, hdf, grib2). If I do find it, add it to my list
> of libs and includes and continue, enabling features as I go through
> the list of libraries.
>
>
>> I don't understand clearly, however, if you want to use the same
>> macros
>> to detect the system libraries and the supplib libraries. If it is  
>> the
>> case it won't be that easy to achieve, because linking staticaly
>> against a supplib means supplying the path of the static
>> library or enclose the static link in -Wl,-static -lname -Wl,-dy (for
>> gcc, at least). This wouldn't easily be done with the system library
>> detection macros.
> Hmm.  Let's consider the readline case. Current code is :
>
>    if test "$ga_dyn_supplibs" = "yes" ; then
>      GA_LIB_READLINE([use_readline=yes readline_libs=""])
>    else
>      GA_CHECK_READLINE([use_readline=yes])
>    fi
>
> What if I changed that to something like:
>
>    if test "$ga_dyn_supplibs" = "yes" ; then
>      GA_DYN_READLINE([use_readline=yes readline_libs=""])
>    else
>      GA_SUPPLIBS_READLINE([use_readline=yes],[use_readline=no])
>      if teset "$use_readline" = "no" ; then
>        GA_DYN_READLINE([use_readline=yes readline_libs=""])
>      fi
>    fi
>
> Where GA_SUPPLIBS_READLINE only looks in ./supplibs and always uses
> the static link -L/supplibs/libname.a
>
> Would that produce the desired result?
>
> Jennifer
>
>
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Patrice D. <per...@fr...>]

On Fri, Aug 01, 2008 at 09:59:33AM -0400, Jennifer Adams wrote:
> 
> The cruft I'm talking about is anything related to the CRAY, support  
> for the "wi" command, directory names called "supplibs-sol55", stuff  
> like that.

Right. Indeed, those things can certainly go away. Though they are not
really problematic either.

> > I don't see that in configure.in/acinclude.m4. It is
> > present in opengrads, but it is not an issue, in my opinion. On the
> > contrary, it allows to build grads against old libdods, for example,
> > which is possible and should not be dropped in my opinion.
> I don't want GrADS 2.0 to keep trying to support libraries that  
> depend on old versions of other libraries and compilers. My remedy  
> for GrADS's bad case of versionitis is to forget the past and make it  
> work with current versions of the dependent libraries. For example, I  
> need to make sure I've got HDF4 built with the --disable-netcdf  
> option, and that option is working properly in HDF4.2r3 (this version  
> has finally eliminated all dependence on netcdf.h  -- oh, If I had a  
> dollar for every pain that netcdf.h has caused me ... I'd get a free  
> lunch!) The whole DODS mess is the worst offender. libgadods, libdap+ 
> +, libnc-dods, RIP.  Can't wait till libnc-dap.a is swept up into  
> libnetcdf.a.

I disagree with that view. I think that the automatic detection of
libraries should try to find any compatible library, to leave to the
user the choice of the libraries. Of course, to ease maintainance very
old cruft should be removed, but, for example, the --disable-netcdf
option has varied a bit from 4.2r1 to 4.2r3, and my point of view is that
all those versions should be usable if the burden is not too high. In
RHEL4 the hdf version is still 4.2r1 and will certainly remain as is
until the end of life of RHEL4 which is Feb 2012, and it would be nice
for the maintainer of grads in EPEL4 (who happens to be me, for the time
being, but who knows in 4 years) if grads in 2012 could still use 4.2r1
unless there has been technical reasons not to use it anymore. There
could be very valid reasons not to use 4.2r1 but if there is not it is
better to be able to use it in my opinion. The RHEL4 example is not
necessarily a plea for having grads still build on RHEL/EPEL4 in 2012,
but an indication that some users want to be able to use a stable
system, and I think that these users should be taken into account, as
long as it doesn't cost too much.

> I want ./supplibs to be the first place to look for libraries, then  
> if they don't exist there, go hunting for them in the system. If a  
> user knows where a library is, but it's not in ./supplibs, then the -- 
> with configure flag is used to guide the hunting.

Ok, I have understood now. It certainly covers the grads users need.

> You're right, there would be no difference if "supplibs"  always  
> trumps the system location. I guess the --enable-dyn-supplibs flag  
> could disable the check for ./supplibs or ../subblibs and then system  
> location would trump "supplibs".

If I understand rightly what you said in the other mail, it would become

* no args to configure:
search in supplibs and, if not found, use system libs.
* --enable-dyn-supplibs
never search in supplibs, even if the directory exists
* --disable-dyn-supplibs
never use system libraries

Did I got it right?

> Yes, the --without flag could save configuration time if you don't  
> care about enabling a certain feature. The HDF group has a nice model  

Not only, also build time, and could allow for disabling features. One
may not want the support for some formats for security reasons, for
example.

> for configuration with dependent libraries. I am going to borrow some  
> stuff from them and separate out each library that is required. I'll  
> look for libz.a, if not found, I can skip looking for a lot of other  
> libraries (png, gd, hdf, grib2). If I do find it, add it to my list  
> of libs and includes and continue, enabling features as I go through  
> the list of libraries.

I'd say that speed doesn't really matter, what matters is the
maintainability of th ecode and the correctness of the results.

> Hmm.  Let's consider the readline case. Current code is :
> 
>    if test "$ga_dyn_supplibs" = "yes" ; then
>      GA_LIB_READLINE([use_readline=yes readline_libs=""])
>    else
>      GA_CHECK_READLINE([use_readline=yes])
>    fi
> 
> What if I changed that to something like:
> 
>    if test "$ga_dyn_supplibs" = "yes" ; then
>      GA_DYN_READLINE([use_readline=yes readline_libs=""])
>    else
>      GA_SUPPLIBS_READLINE([use_readline=yes],[use_readline=no])
>      if teset "$use_readline" = "no" ; then
>        GA_DYN_READLINE([use_readline=yes readline_libs=""])
>      fi
>    fi
> 
> Where GA_SUPPLIBS_READLINE only looks in ./supplibs and always uses  
> the static link -L/supplibs/libname.a
> 
> Would that produce the desired result?

It looks good.

--
Pat

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

On Aug 1, 2008, at 11:20 AM, Patrice Dumas wrote:

> On Fri, Aug 01, 2008 at 09:59:33AM -0400, Jennifer Adams wrote:
>>
>> The cruft I'm talking about is anything related to the CRAY, support
>> for the "wi" command, directory names called "supplibs-sol55", stuff
>> like that.
>
> Right. Indeed, those things can certainly go away. Though they are not
> really problematic either.
>
>>> I don't see that in configure.in/acinclude.m4. It is
>>> present in opengrads, but it is not an issue, in my opinion. On the
>>> contrary, it allows to build grads against old libdods, for example,
>>> which is possible and should not be dropped in my opinion.
>> I don't want GrADS 2.0 to keep trying to support libraries that
>> depend on old versions of other libraries and compilers. My remedy
>> for GrADS's bad case of versionitis is to forget the past and make it
>> work with current versions of the dependent libraries. For example, I
>> need to make sure I've got HDF4 built with the --disable-netcdf
>> option, and that option is working properly in HDF4.2r3 (this version
>> has finally eliminated all dependence on netcdf.h  -- oh, If I had a
>> dollar for every pain that netcdf.h has caused me ... I'd get a free
>> lunch!) The whole DODS mess is the worst offender. libgadods, libdap+
>> +, libnc-dods, RIP.  Can't wait till libnc-dap.a is swept up into
>> libnetcdf.a.
>
> I disagree with that view. I think that the automatic detection of
> libraries should try to find any compatible library, to leave to the
> user the choice of the libraries. Of course, to ease maintainance very
> old cruft should be removed, but, for example, the --disable-netcdf
> option has varied a bit from 4.2r1 to 4.2r3, and my point of view  
> is that
> all those versions should be usable if the burden is not too high. In
> RHEL4 the hdf version is still 4.2r1 and will certainly remain as is
> until the end of life of RHEL4 which is Feb 2012, and it would be nice
> for the maintainer of grads in EPEL4 (who happens to be me, for the  
> time
> being, but who knows in 4 years) if grads in 2012 could still use  
> 4.2r1
> unless there has been technical reasons not to use it anymore. There
> could be very valid reasons not to use 4.2r1 but if there is not it is
> better to be able to use it in my opinion. The RHEL4 example is not
> necessarily a plea for having grads still build on RHEL/EPEL4 in 2012,
> but an indication that some users want to be able to use a stable
> system, and I think that these users should be taken into account, as
> long as it doesn't cost too much.
Very persuasive. I guess it depends a lot on which libraries we're  
talking about. It's not fair to lump HDF into the same category as  
DODS, which is far more aggravating to support. But if you have an  
old box with gcc 2.95, you just can't have an opendap-enabled build  
of GrADS 2.0.


>> I want ./supplibs to be the first place to look for libraries, then
>> if they don't exist there, go hunting for them in the system. If a
>> user knows where a library is, but it's not in ./supplibs, then  
>> the --
>> with configure flag is used to guide the hunting.
>
> Ok, I have understood now. It certainly covers the grads users need.
>
>> You're right, there would be no difference if "supplibs"  always
>> trumps the system location. I guess the --enable-dyn-supplibs flag
>> could disable the check for ./supplibs or ../subblibs and then system
>> location would trump "supplibs".
>
> If I understand rightly what you said in the other mail, it would  
> become
>
> * no args to configure:
> search in supplibs and, if not found, use system libs.
> * --enable-dyn-supplibs
> never search in supplibs, even if the directory exists
> * --disable-dyn-supplibs
> never use system libraries
>
> Did I got it right?
Yes. It's a little unorthodox, but it achieves what we need.

>
>> Yes, the --without flag could save configuration time if you don't
>> care about enabling a certain feature. The HDF group has a nice model
>
> Not only, also build time, and could allow for disabling features. One
> may not want the support for some formats for security reasons, for
> example.
>
>> for configuration with dependent libraries. I am going to borrow some
>> stuff from them and separate out each library that is required. I'll
>> look for libz.a, if not found, I can skip looking for a lot of other
>> libraries (png, gd, hdf, grib2). If I do find it, add it to my list
>> of libs and includes and continue, enabling features as I go through
>> the list of libraries.
>
> I'd say that speed doesn't really matter, what matters is the
> maintainability of th ecode and the correctness of the results.
>
>> Hmm.  Let's consider the readline case. Current code is :
>>
>>    if test "$ga_dyn_supplibs" = "yes" ; then
>>      GA_LIB_READLINE([use_readline=yes readline_libs=""])
>>    else
>>      GA_CHECK_READLINE([use_readline=yes])
>>    fi
>>
>> What if I changed that to something like:
>>
>>    if test "$ga_dyn_supplibs" = "yes" ; then
>>      GA_DYN_READLINE([use_readline=yes readline_libs=""])
>>    else
>>      GA_SUPPLIBS_READLINE([use_readline=yes],[use_readline=no])
>>      if teset "$use_readline" = "no" ; then
>>        GA_DYN_READLINE([use_readline=yes readline_libs=""])
>>      fi
>>    fi
>>
>> Where GA_SUPPLIBS_READLINE only looks in ./supplibs and always uses
>> the static link -L/supplibs/libname.a
>>
>> Would that produce the desired result?
>
> It looks good.

OK. I will get back to reworking some of this code and write again  
when I am stuck.
Jennifer

>
> --
> Pat
>
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Arlindo da S. <da...@al...>]

Jennifer,

   It is a busy day for me, I haven't read the exchange in great
detail. A couple of comments,

1) The clean-up/refactoring is very welcome

2) Having a flexible build that would look for dependencies in the
supplibs and if not found look "around" sounds good in principle but
can be tricky to implement. The supplibs not only builds everything
statically, but do so cooperatively: dependencies are resolved within
the supplibs. Having the build look at the supplibs and them elsewhere
needs to be done with extreme care so that we don't ended up using 2
different versions of the external libraries.  For this reason I
always felt it was safer to either build with the supplibs exclusively
or else look around. The point is, we need to make sure that external
libraries being used are built consistently.

3) About the detection m4 macros. More and more packages are providing
tidbits of m4 macros to detect their library which we can incorporate
in our build. As much as it makes sense we should ntry to use these m4
fragments "as is" without much grads customization. This way we would
be able to upgrade it when I new version of the package are brought
in.

4) Regarding

--enable-dyn-supplibs    never look for "supplibs" and always use DYN
macros to look for libraries
--disable-dyn-supplibs    always use "supplibs" and never use DYN
macros to look for libraries

I never liked the term "dynamic supplibs".  It really means, look
around for the dependencies as any other package would do - those
libraries do not have to conform to any of the supplibs conventions.
I'd suggest
something like:

--with-supplibs[=SUPPLIBDIR]        which tries to resolve the
dependencies within the supplibs
--without-supplibs                               the opposite

Now, for those cases where the user would prefer to use the system
libraries and complement it with the supplibs for those few libraries
they don't have this could be done simply with specific macros such as

--with-netcdf=include_path,lib_path

or by simply setting the CPPFLAGS, etc. I'd not try do to this
automatically: it is tricky and prone to error.

Regarding the question of defaults. Building against pre-compiled
supplibs is the easiest and more likely to produce correct binaries.
I have had very good experience with v1.9.0-rc1 and v2.0.ax.oga.y
builds. So, --with-supplibs could be the default. However, if the
supplibs cannot be found you could issue an warning:

- Cannot find supplibs; either download them from http:/.... or
specify --without-supplibs for satisfying the dependencies with
locally installed libraries.

5) Readline. One of the portabilities issues I ran into recently is
termcap/ncurses. Termcap is deprecated and recently I have added
ncurses to the supplibs, although as of yet not all pre-compiled
supplib tarballs have ncurses in them.

6) Labeling the binaries. The opengrads builds have consistently used
the config.guess script to determine the the architecture labeling of
the tarballs, while COLA has been using Linux distro specific
labeling. My experience is that the i686-pc-linux binaries have been
extremely portable, with the x86_64 slightly less so. Even when the
x86_64 did not quite work, rebuilding against the pre-compiled x86_64
supplibs did the trick. And in most cases the i686 built works on
x86_64, so many users just went with that. It took some
experimentation, and excluding the openssl/termcap libraries is one of
the fine tunings I had to do.

  Just my 2 cents.

       Arlindo



On Fri, Aug 1, 2008 at 9:59 AM, Jennifer Adams <jm...@co...> wrote:
>
> On Aug 1, 2008, at 8:55 AM, Patrice Dumas wrote:
>
>> On Fri, Aug 01, 2008 at 08:09:12AM -0400, Jennifer Adams wrote:
>>> Dear Experts,
>>>
>>> I've been pecking away at the configure scripts and thinking about
>>> the
>>> logic in them. There's a lot of cruft in there -- support of
>>> "historical" stuff that is no longer relevant, and outdated platform
>>> dependencies.
>>
>> Do you see that in configure.in/acinclude.m4 from, for example,
>> grads-2.0.a2, or in the macros used with --enable-dyn-supplibs in
>> opengrads?
> The cruft I'm talking about is anything related to the CRAY, support
> for the "wi" command, directory names called "supplibs-sol55", stuff
> like that.
>
>> I don't see that in configure.in/acinclude.m4. It is
>> present in opengrads, but it is not an issue, in my opinion. On the
>> contrary, it allows to build grads against old libdods, for example,
>> which is possible and should not be dropped in my opinion.
> I don't want GrADS 2.0 to keep trying to support libraries that
> depend on old versions of other libraries and compilers. My remedy
> for GrADS's bad case of versionitis is to forget the past and make it
> work with current versions of the dependent libraries. For example, I
> need to make sure I've got HDF4 built with the --disable-netcdf
> option, and that option is working properly in HDF4.2r3 (this version
> has finally eliminated all dependence on netcdf.h  -- oh, If I had a
> dollar for every pain that netcdf.h has caused me ... I'd get a free
> lunch!) The whole DODS mess is the worst offender. libgadods, libdap+
> +, libnc-dods, RIP.  Can't wait till libnc-dap.a is swept up into
> libnetcdf.a.
>
>> Also those
>> macros in fact come from other sources and should be kept synchronized
>> with these sources (mostly opendap).
> The dyn-supplibs macros are the ones I want to bring into the GrADS
> mainstream, to be used by default.
>
>>
>>> It seems to me that the default behavior should be to look
>>> for a directory called "supplibs" (in ./ and ../) and if not found
>>> then
>>> go hunting in the system for the required libraries, creating a
>>> dynamically linked executable. To create a more portable build, the
>>
>> Does this means that --enable-dyn-supplibs is the default unless a
>> ./supplib or ../supplib directory exists, and if a ./supplib or ../
>> supplib
>> directory exists one has to add --enable-dyn-supplibs to link against
>> system libraries? It it is what you meant, this seems reasonnable
>> to me.
>
> I want ./supplibs to be the first place to look for libraries, then
> if they don't exist there, go hunting for them in the system. If a
> user knows where a library is, but it's not in ./supplibs, then the --
> with configure flag is used to guide the hunting.
>
>>
>>> configure flag "--disable-shared" will stick with what's in the
>>> "supplibs"  directory and use the "-L/ga_supplb_path/libname.a"
>>> syntax
>>> instead of -L/path -lname" .
>>
>> That is not clear to me. What would the difference be between a build
>> with "--disable-shared" and a build with a ./supplib or ../supplib
>> directory present would be?
>
> You're right, there would be no difference if "supplibs"  always
> trumps the system location. I guess the --enable-dyn-supplibs flag
> could disable the check for ./supplibs or ../subblibs and then system
> location would trump "supplibs".
>
>>
>>> The "--with-PACKAGE" configure options will
>>> be used to guide the hunting for a specific library, something like
>>> --with-hdf=/opt/local (and then it will look for
>>> /opt/local/lib/libmfhdf.a and /opt/local/include/mfhdf.h)
>>
>> Agreed. It could even be used to use some supplibs but not all.
>> --without-* could still be used to disable a feature.
>
> Yes, the --without flag could save configuration time if you don't
> care about enabling a certain feature. The HDF group has a nice model
> for configuration with dependent libraries. I am going to borrow some
> stuff from them and separate out each library that is required. I'll
> look for libz.a, if not found, I can skip looking for a lot of other
> libraries (png, gd, hdf, grib2). If I do find it, add it to my list
> of libs and includes and continue, enabling features as I go through
> the list of libraries.
>
>
>> I don't understand clearly, however, if you want to use the same
>> macros
>> to detect the system libraries and the supplib libraries. If it is the
>> case it won't be that easy to achieve, because linking staticaly
>> against a supplib means supplying the path of the static
>> library or enclose the static link in -Wl,-static -lname -Wl,-dy (for
>> gcc, at least). This wouldn't easily be done with the system library
>> detection macros.
> Hmm.  Let's consider the readline case. Current code is :
>
>   if test "$ga_dyn_supplibs" = "yes" ; then
>     GA_LIB_READLINE([use_readline=yes readline_libs=""])
>   else
>     GA_CHECK_READLINE([use_readline=yes])
>   fi
>
> What if I changed that to something like:
>
>   if test "$ga_dyn_supplibs" = "yes" ; then
>     GA_DYN_READLINE([use_readline=yes readline_libs=""])
>   else
>     GA_SUPPLIBS_READLINE([use_readline=yes],[use_readline=no])
>     if teset "$use_readline" = "no" ; then
>       GA_DYN_READLINE([use_readline=yes readline_libs=""])
>     fi
>   fi
>
> Where GA_SUPPLIBS_READLINE only looks in ./supplibs and always uses
> the static link -L/supplibs/libname.a
>
> Would that produce the desired result?
>
> Jennifer
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>



-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] dynamic supplibs checking

[Patrice D. <per...@fr...>]

On Fri, Aug 01, 2008 at 11:44:15AM -0400, Jennifer Adams wrote:
> Very persuasive. I guess it depends a lot on which libraries we're  
> talking about. It's not fair to lump HDF into the same category as DODS, 
> which is far more aggravating to support. But if you have an old box with 
> gcc 2.95, you just can't have an opendap-enabled build of GrADS 2.0.

Indeed, but gcc 2.95 is very very old. With anything above you can
certainly (maybe with some stl implementation). Besides, grads doesn't 
link against libdap, but against libncdap which has a far better binary 
stability. and we can imagine that the binary compatibility will never 
break, so this gives room for backward compatibility at the source level.

--
Pat

Re: [Opengrads-devel] dynamic supplibs checking

[Arlindo da S. <da...@al...>]

On Fri, Aug 1, 2008 at 12:08 PM, Patrice Dumas <per...@fr...> wrote:
> On Fri, Aug 01, 2008 at 11:44:15AM -0400, Jennifer Adams wrote:
>> Very persuasive. I guess it depends a lot on which libraries we're
>> talking about. It's not fair to lump HDF into the same category as DODS,
>> which is far more aggravating to support. But if you have an old box with
>> gcc 2.95, you just can't have an opendap-enabled build of GrADS 2.0.
>

Or even for v1.9; in going to the new libdap there have been API level
changes that make it impossible to build against an old libdods. Being
able to use an old libgadods with gcc-2.95 is just too much trouble
and in my judgement, not worth it.

> Indeed, but gcc 2.95 is very very old. With anything above you can
> certainly (maybe with some stl implementation). Besides, grads doesn't
> link against libdap, but against libncdap which has a far better binary
> stability. and we can imagine that the binary compatibility will never
> break, so this gives room for backward compatibility at the source level.
>

Except for the size of the binary, gradsdap appears just as stable as
"grads" built with the vanilla netcdf. Do you have any evidence that
it adds any performance penalty besides the start up time?

Perhaps we could take a bold step: build just one grads which is
nc-dap enabled; and have "gradsnc" built as an option for those who
need it. One could certainly try it out and see if the users can live
with a single grads.  My guess is that it would work just fine. So,
you just add a single switch

--with-nc-dap         (default)
--without-nc-dap

This would really simplify the build...


         Arlindo




-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] dynamic supplibs checking

[Patrice D. <per...@fr...>]

On Fri, Aug 01, 2008 at 12:45:46PM -0400, Arlindo da Silva wrote:
> On Fri, Aug 1, 2008 at 12:08 PM, Patrice Dumas <per...@fr...> wrote:
> 
> Or even for v1.9; in going to the new libdap there have been API level
> changes that make it impossible to build against an old libdods. Being
> able to use an old libgadods with gcc-2.95 is just too much trouble
> and in my judgement, not worth it.

libgadap/libgadods is not grads. There is an interface between them. The
gadap/gadods interface may be stable and hide the libdap changes.

> Perhaps we could take a bold step: build just one grads which is
> nc-dap enabled; and have "gradsnc" built as an option for those who
> need it. One could certainly try it out and see if the users can live
> with a single grads.  My guess is that it would work just fine. So,
> you just add a single switch

That could be doable, since libnc-dap also brings in netcdf support. So
to have gradsnc one would build with different options and it wouldn't
be possible to have both in one build. Is it what you are proposing?

--
Pat

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

A fully-features OPeNDAP-enabled GrADS needs two things: libnc-dap  
and libgadap. The station data interface is not so widely used as  
gridded data, but it is still extremely important.

By the way, please look at ftp://grads.iges.org/grads/Supplibs/2.0/ 
src/gadap-2.0.tar.gz and let me know if I got everything right. I  
have changed the copyright, build scripts, m4 macros, version number,  
etc.

Jennifer


On Aug 1, 2008, at 12:45 PM, Arlindo da Silva wrote:

> On Fri, Aug 1, 2008 at 12:08 PM, Patrice Dumas <per...@fr...>  
> wrote:
>> On Fri, Aug 01, 2008 at 11:44:15AM -0400, Jennifer Adams wrote:
>>> Very persuasive. I guess it depends a lot on which libraries we're
>>> talking about. It's not fair to lump HDF into the same category  
>>> as DODS,
>>> which is far more aggravating to support. But if you have an old  
>>> box with
>>> gcc 2.95, you just can't have an opendap-enabled build of GrADS 2.0.
>>
>
> Or even for v1.9; in going to the new libdap there have been API level
> changes that make it impossible to build against an old libdods. Being
> able to use an old libgadods with gcc-2.95 is just too much trouble
> and in my judgement, not worth it.
>
>> Indeed, but gcc 2.95 is very very old. With anything above you can
>> certainly (maybe with some stl implementation). Besides, grads  
>> doesn't
>> link against libdap, but against libncdap which has a far better  
>> binary
>> stability. and we can imagine that the binary compatibility will  
>> never
>> break, so this gives room for backward compatibility at the source  
>> level.
>>
>
> Except for the size of the binary, gradsdap appears just as stable as
> "grads" built with the vanilla netcdf. Do you have any evidence that
> it adds any performance penalty besides the start up time?
>
> Perhaps we could take a bold step: build just one grads which is
> nc-dap enabled; and have "gradsnc" built as an option for those who
> need it. One could certainly try it out and see if the users can live
> with a single grads.  My guess is that it would work just fine. So,
> you just add a single switch
>
> --with-nc-dap         (default)
> --without-nc-dap
>
> This would really simplify the build...
>
>
>          Arlindo
>
>
>
>
> -- 
> Arlindo da Silva
> da...@al...
>
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Patrice D. <per...@fr...>]

On Fri, Aug 01, 2008 at 12:59:47PM -0400, Jennifer Adams wrote:
> A fully-features OPeNDAP-enabled GrADS needs two things: libnc-dap and 
> libgadap. The station data interface is not so widely used as gridded 
> data, but it is still extremely important.

Indeed, but what a user want may not be a fully-featured GrADS, so the
build should proceed with what the  user has.

In fedora, we add all the features we can add, so when I have time, I'll
submit libgadap for inclusion in fedora if the license is free software.
I currently don't have time, but certainly in a few weeks.

--
Pat

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

Dear All,
I have been poring over the configure scripts. The whole thing got  
complicated quickly trying to prioritize supplibs v. system libraries  
and factor in the --with and --enable switches. I quickly realized  
that it could take a really long time to get it all straightened out,  
and I don't have the right expertise to do that anyway. So, to  
simplify things for myself, I focused on getting it right when  
looking in SUPPLIBS, and will leave the dynamic checking in the  
capable hands of Pat and Arlindo and anyone else who contributed to  
those *.m4 macros. I am eager to put out 2.0.a3, so I going to  
release what I've done so far, and that will allow more time for you  
guys to make tweaks and adjustments at a more relaxed pace. It should  
build with Arlindo's gigantic supplibs tarball unpacked and compiled,  
but you'll probably need to set an environment variable SUPPLIBS to  
point to it, since configure.ac will only look in the top directory  
and one level above for a directory named "supplibs". At this point,  
the biggest noticeable change is that 2.0.a3 will require  
subdirectories under supplibs/include/ like this:
dap/
gd/
grib2/
hdf/
libpng12/
netcdf/
readline/
udunits/
zlib/

The configure.ac algorithm is nominally like this:

$use_feature=no
if $with_feature != no
    if (supplibs has required libraries and include files to support  
feature) ; then
       $use_feature=yes
    fi
    if $use_feature=no (i.e., not found in supplibs) and  
$ga_dyn_supplibs=yes, then
       use dynamic macros to find libraries
    fi
fi

Forcing the use of dynamic libraries is achieved by not having a  
supplibs directory.
Forcing the use of ONLY supplibs is achieved by using the configure  
switch "--disable-dyn-supplibs".
The dynamic macros I copied almost verbatim from the opengrads m4  
directory. I may have changed a name or two for clarity. This will be  
code that I am going to generally ignore and leave for you guys to  
support, since I will always be building from supplibs. But I would  
like for it to work, so please send me patches and changes and I will  
check them in.

The --with-* arguments are trickier, since there are some that should  
be just yes/no and others that provide directory locations. The -- 
with-package=yes/no switches that I used are:
   --with-readline         command line editing
   --with-printim          image output
   --with-grib2            GRIB2 data format
   --with-sdf              self-describing formats (HDF and NetCDF)
   --with-dap              extra gradsdap executable, OPeNDAP-enabled
   --with-gui              Athena X11 widget-based GUI
  The remainders are in the dynamic *.m4 macros. I used --with-sdf to  
avoid conflicts with --with-hdf and --with-netcdf arguments in the  
dynamic macros.

I generally avoid the -l<libname> syntax when using supplibs. The  
only exceptions are the -lncurses/-ltermcap thing with readline and  
linking with libdap and libnc-dap always requires a few extra libs. I  
put libcurl.a and libxml2.a in supplibs/lib, but there are still  
others, and these are not the same for every OS. The dynamic macros  
use the output from "ncdap-config --libs", but on my mac this left  
out some of the required ones so I have hard-coded a kludge:
     if test "$is_darwin" = "yes" ; then
       dap_extra_libs="-lssl -lcrypto -lpthread -liconv -lm"
     else
       dap_extra_libs="-lidn -lssl -lcrypto -ldl -lpthread -lm"
     fi
     dap_libs="$gadap_lib $dap_libs $dap_extra_libs"
This syntax gives me what I need to build on our mac and linux boxes,  
but will probably be wrong for sunOS and others. Since we have checks  
for host-specific options, any platform-specific dap_extra_libs can  
be added as necessary.

Finally, I had wanted to include library version numbers in the 'q  
config' output, but that turned out to be tricky if the library did  
not contain a routine for spitting out a version string. Netcdf has  
nc_inq_libvers(), but many of the others simply have #define  
statements in the header files, but this was highly version-dependent  
and will cause problems if using system libraries instead of  
supplibs. I left some of that code in gacfg.c but other bits are  
commented out. If you guys have any ideas for how to do this, maybe  
there's a way to do it during the configuration process and populate  
GrADS's own config.h with strings like #define GA_ZLIB_VERSION  
1.2.3 ... then gacfg.c need only look in config.h to get that info.

Well, that's enough. If you've read this far, I thank you sincerely  
for your attention to these arcane details.
--Jennifer


--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Arlindo da S. <da...@al...>]

On Fri, Aug 8, 2008 at 11:25 AM, Jennifer Adams <jm...@co...> wrote:
> Dear All,
> I have been poring over the configure scripts. The whole thing got
> complicated quickly trying to prioritize supplibs v. system libraries and
> factor in the --with and --enable switches.

I believe you...

>  I quickly realized that it could
> take a really long time to get it all straightened out, and I don't have the
> right expertise to do that anyway. So, to simplify things for myself, I
> focused on getting it right when looking in SUPPLIBS, and will leave the
> dynamic checking in the capable hands of Pat and Arlindo and anyone else who
> contributed to those *.m4 macros. I am eager to put out 2.0.a3, so I going
> to release what I've done so far, and that will allow more time for you guys
> to make tweaks and adjustments at a more relaxed pace.

I'd need some guidance from you here. The build scripts in tag
grads-2_0_a2_oga-1 was my best attempt to build GrADS 2 with the
supplibs and the so-called "dynamic supplibs"; so far as I can tell
these were working, at least on darwin ppc/intel, Linux i686, x86_64,
and freebsd (based on passing the test suite).  I did this by adapting
the stuff Pat had done for v1.9.0-rc1, and he later on revised what I
did. In converting the v1.9 build I tried to cleanup as much legacy
stuff as I could, but perhaps I left some debris here or there.

Would I start tweaking it I would probably end up a place not too far
from the build scripts in 2.0.a2-oga.1 (there are minor C code changes
from COLA's v2.0.a2 but these are not related to the build.) I'll need
to understand which of features of the 2.0.a2-oga.1 build you object
so that I do not repeat the same mistake. Perhaps a face-to-face merge
exercise would be he most efficient way of getting this done.

> It should build with
> Arlindo's gigantic supplibs tarball unpacked and compiled, but you'll
> probably need to set an environment variable SUPPLIBS to point to it, since
> configure.ac will only look in the top directory and one level above for a
> directory named "supplibs".

The instructions on the wiki tell you to symlink the untarred supplibs
to the plain "supplibs" name, just as you require.

BTW, are you making use of pkgconfig where appropriate? The
supplibs-2.0.x are built with their own version of pkg-config and many
of the packages are instrumented to use it, e.g.,

gadap.pc
libcurl.pc
libdap.pc
libdapclient.pc
libdapserver.pc
libpng.pc
libsx.pc
libxml-2.0.pc

By using pkgconfig you get all the necessary flags for the build, and
the *.m4 macros take advantage of it.

> At this point, the biggest noticeable change is
> that 2.0.a3 will require subdirectories under supplibs/include/ like this:
> dap/
> gd/
> grib2/
> hdf/
> libpng12/
> netcdf/
> readline/
> udunits/
> zlib/

Is this list supposed to be comprehensive? Currently, the
supplibs-2.0.1 have the following structure under include/

X11/  <====== (neXawt)
curl/ <===== (needed for build portability)
gadap/ <=======
gd/
grib2c/  <===== (name change)
hdf/
jasper/ [grib2c]
jpeg/ [hdf]
libdap/ <===== (name change)
libnc-dap/  <======
libpng/ <==== (name change)
libsx/ <===== [athena]
libxml2/ <==== (needed for build portability [dap])
ncurses/  <====== (needed for portability [readline])
netcdf/
readline/
szlib/         <======= (needed for *reading* sziped files [hdf])
udunits/
zlib/

The "<====" above indicate discrepancies. We have built the
supplibs-2.0.1 on a number of platforms. If you could preserve the
names above it would not require us to go back have to rebuild all of
these again... Not a big deal, but kind of time consuming.

> The configure.ac algorithm is nominally like this:
> $use_feature=no
> if $with_feature != no
>    if (supplibs has required libraries and include files to support feature)
> ; then
>       $use_feature=yes
>    fi
>    if $use_feature=no (i.e., not found in supplibs) and
> $ga_dyn_supplibs=yes, then
>       use dynamic macros to find libraries
>    fi
> fi
> Forcing the use of dynamic libraries is achieved by not having a supplibs
> directory.
> Forcing the use of ONLY supplibs is achieved by using the configure switch
> "--disable-dyn-supplibs".
> The dynamic macros I copied almost verbatim from the opengrads m4 directory.
> I may have changed a name or two for clarity. This will be code that I am
> going to generally ignore and leave for you guys to support, since I will
> always be building from supplibs. But I would like for it to work, so please
> send me patches and changes and I will check them in.
> The --with-* arguments are trickier, since there are some that should be
> just yes/no and others that provide directory locations. The
> --with-package=yes/no switches that I used are:
>   --with-readline         command line editing
>   --with-printim          image output
>   --with-grib2            GRIB2 data format
>   --with-sdf              self-describing formats (HDF and NetCDF)
>   --with-dap              extra gradsdap executable, OPeNDAP-enabled
>   --with-gui              Athena X11 widget-based GUI
>  The remainders are in the dynamic *.m4 macros. I used --with-sdf to avoid
> conflicts with --with-hdf and --with-netcdf arguments in the dynamic
> macros.
> I generally avoid the -l<libname> syntax when using supplibs. The only
> exceptions are the -lncurses/-ltermcap thing with readline and linking with
> libdap and libnc-dap always requires a few extra libs.

pkgconfig usually takes care of these details.

> I put libcurl.a and
> libxml2.a in supplibs/lib, but there are still others, and these are not the
> same for every OS. The dynamic macros use the output from "ncdap-config
> --libs", but on my mac this left out some of the required ones so I have
> hard-coded a kludge:
>     if test "$is_darwin" = "yes" ; then
>       dap_extra_libs="-lssl -lcrypto -lpthread -liconv -lm"
>     else
>       dap_extra_libs="-lidn -lssl -lcrypto -ldl -lpthread -lm"
>     fi
>     dap_libs="$gadap_lib $dap_libs $dap_extra_libs"
> This syntax gives me what I need to build on our mac and linux boxes, but
> will probably be wrong for sunOS and others. Since we have checks for
> host-specific options, any platform-specific dap_extra_libs can be added as
> necessary.

Consider using pkgconfig. None better than the library developers to
know what is required for a given version; pkgconfig provides the
mechanism for them to convey this information.

> Finally, I had wanted to include library version numbers in the 'q config'
> output, but that turned out to be tricky if the library did not contain a
> routine for spitting out a version string. Netcdf has nc_inq_libvers(), but
> many of the others simply have #define statements in the header files, but
> this was highly version-dependent and will cause problems if using system
> libraries instead of supplibs. I left some of that code in gacfg.c but other
> bits are commented out. If you guys have any ideas for how to do this, maybe
> there's a way to do it during the configuration process and populate GrADS's
> own config.h with strings like #define GA_ZLIB_VERSION 1.2.3 ... then
> gacfg.c need only look in config.h to get that info.

The OpenGrADS supplibs have a tag corresponding to each supplib
version, say 2.0.1 (the one I use to build grads2). In the very least,
gacfg.c could report the supplibs version used for the build. I may
need to add a way for this version number to be retrieved...

For tracking the version of the individual packages, as long as the
packages are using autoconf (true for supplibs-2.0.x) you can get the
version number by parsing configure.ac/configure.in during the
supplibs build, e.g.,

% grep ^AC_INIT netcdf/configure.ac
AC_INIT([netCDF], [3.6.2], [su...@un...])

A simple script could added to the supplibs build to compile this
information in a simple text file, say "supplibs.h". Are you willing
to adopt (and help maintain) the supplibs-2.0.x?

> Well, that's enough. If you've read this far, I thank you sincerely for your
> attention to these arcane details.

This is the reality of autoconf.

   Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

It will be easier to discuss when 2.0.a3 is out, but ... here are  
examples of the kind of stuff I changed in configure.ac:
1) I noticed that the GA_CHECK_LIB routine only checked for the  
presence of a required library, but did not check whether it could  
actually be used in a program. So I changed all those to  
AC_CHECK_LIB, first making sure that the -I and -L flags were limited  
to the supplibs dir.
2) I avoid double checking for libraries that are required by more  
than one feature within the context of the with/without flags. The  
logic had been mostly in line with setting up the different 1.9  
exectuables, whereas now it's focused on enabling features in a  
single build. (Remember gradsdap will be phased out as soon as  
unidata netcdf 4 is opendap-ready.).
3) I added login to skip to dynamic macros if supplibs doesn't exist.
I am not trying to tear apart what you've already done, only trying  
to improve it.

As for the supplibs/include subdirectories, I renamed grib2->grib2c,  
and split dap into gadap and libnc-dap. But the libpng12 name comes  
from their source code, to distinguish from earlier versions, so I  
think we should leave that as is (they renamed the library too,  
libpng12.a).  The remainder of the include directories that weren't  
in my list may be essential for building all the supplibs in a  
coherent way, but they are not necessary for compiling GrADS -- I  
generally extract only what GrADS needs from a complete library  
installation and put that into my supplibs location. I should mention  
that I never build with GUI, so I don't know what's required there.

The pkgconfig files are a very user-friendly; I hadn't looked at them  
before. I was using executables like "ncdap-config --libs" which were  
giving me inconsistent results.

--Jennifer

On Aug 8, 2008, at 9:45 PM, Arlindo da Silva wrote:

> On Fri, Aug 8, 2008 at 11:25 AM, Jennifer Adams <jm...@co...>  
> wrote:
>> Dear All,
>> I have been poring over the configure scripts. The whole thing got
>> complicated quickly trying to prioritize supplibs v. system  
>> libraries and
>> factor in the --with and --enable switches.
>
> I believe you...
>
>>  I quickly realized that it could
>> take a really long time to get it all straightened out, and I  
>> don't have the
>> right expertise to do that anyway. So, to simplify things for  
>> myself, I
>> focused on getting it right when looking in SUPPLIBS, and will  
>> leave the
>> dynamic checking in the capable hands of Pat and Arlindo and  
>> anyone else who
>> contributed to those *.m4 macros. I am eager to put out 2.0.a3, so  
>> I going
>> to release what I've done so far, and that will allow more time  
>> for you guys
>> to make tweaks and adjustments at a more relaxed pace.
>
> I'd need some guidance from you here. The build scripts in tag
> grads-2_0_a2_oga-1 was my best attempt to build GrADS 2 with the
> supplibs and the so-called "dynamic supplibs"; so far as I can tell
> these were working, at least on darwin ppc/intel, Linux i686, x86_64,
> and freebsd (based on passing the test suite).  I did this by adapting
> the stuff Pat had done for v1.9.0-rc1, and he later on revised what I
> did. In converting the v1.9 build I tried to cleanup as much legacy
> stuff as I could, but perhaps I left some debris here or there.
>
> Would I start tweaking it I would probably end up a place not too far
> from the build scripts in 2.0.a2-oga.1 (there are minor C code changes
> from COLA's v2.0.a2 but these are not related to the build.) I'll need
> to understand which of features of the 2.0.a2-oga.1 build you object
> so that I do not repeat the same mistake. Perhaps a face-to-face merge
> exercise would be he most efficient way of getting this done.
>
>> It should build with
>> Arlindo's gigantic supplibs tarball unpacked and compiled, but you'll
>> probably need to set an environment variable SUPPLIBS to point to  
>> it, since
>> configure.ac will only look in the top directory and one level  
>> above for a
>> directory named "supplibs".
>
> The instructions on the wiki tell you to symlink the untarred supplibs
> to the plain "supplibs" name, just as you require.
>
> BTW, are you making use of pkgconfig where appropriate? The
> supplibs-2.0.x are built with their own version of pkg-config and many
> of the packages are instrumented to use it, e.g.,
>
> gadap.pc
> libcurl.pc
> libdap.pc
> libdapclient.pc
> libdapserver.pc
> libpng.pc
> libsx.pc
> libxml-2.0.pc
>
> By using pkgconfig you get all the necessary flags for the build, and
> the *.m4 macros take advantage of it.
>
>> At this point, the biggest noticeable change is
>> that 2.0.a3 will require subdirectories under supplibs/include/  
>> like this:
>> dap/
>> gd/
>> grib2/
>> hdf/
>> libpng12/
>> netcdf/
>> readline/
>> udunits/
>> zlib/
>
> Is this list supposed to be comprehensive? Currently, the
> supplibs-2.0.1 have the following structure under include/
>
> X11/  <====== (neXawt)
> curl/ <===== (needed for build portability)
> gadap/ <=======
> gd/
> grib2c/  <===== (name change)
> hdf/
> jasper/ [grib2c]
> jpeg/ [hdf]
> libdap/ <===== (name change)
> libnc-dap/  <======
> libpng/ <==== (name change)
> libsx/ <===== [athena]
> libxml2/ <==== (needed for build portability [dap])
> ncurses/  <====== (needed for portability [readline])
> netcdf/
> readline/
> szlib/         <======= (needed for *reading* sziped files [hdf])
> udunits/
> zlib/
>
> The "<====" above indicate discrepancies. We have built the
> supplibs-2.0.1 on a number of platforms. If you could preserve the
> names above it would not require us to go back have to rebuild all of
> these again... Not a big deal, but kind of time consuming.
>
>> The configure.ac algorithm is nominally like this:
>> $use_feature=no
>> if $with_feature != no
>>    if (supplibs has required libraries and include files to  
>> support feature)
>> ; then
>>       $use_feature=yes
>>    fi
>>    if $use_feature=no (i.e., not found in supplibs) and
>> $ga_dyn_supplibs=yes, then
>>       use dynamic macros to find libraries
>>    fi
>> fi
>> Forcing the use of dynamic libraries is achieved by not having a  
>> supplibs
>> directory.
>> Forcing the use of ONLY supplibs is achieved by using the  
>> configure switch
>> "--disable-dyn-supplibs".
>> The dynamic macros I copied almost verbatim from the opengrads m4  
>> directory.
>> I may have changed a name or two for clarity. This will be code  
>> that I am
>> going to generally ignore and leave for you guys to support, since  
>> I will
>> always be building from supplibs. But I would like for it to work,  
>> so please
>> send me patches and changes and I will check them in.
>> The --with-* arguments are trickier, since there are some that  
>> should be
>> just yes/no and others that provide directory locations. The
>> --with-package=yes/no switches that I used are:
>>   --with-readline         command line editing
>>   --with-printim          image output
>>   --with-grib2            GRIB2 data format
>>   --with-sdf              self-describing formats (HDF and NetCDF)
>>   --with-dap              extra gradsdap executable, OPeNDAP-enabled
>>   --with-gui              Athena X11 widget-based GUI
>>  The remainders are in the dynamic *.m4 macros. I used --with-sdf  
>> to avoid
>> conflicts with --with-hdf and --with-netcdf arguments in the dynamic
>> macros.
>> I generally avoid the -l<libname> syntax when using supplibs. The  
>> only
>> exceptions are the -lncurses/-ltermcap thing with readline and  
>> linking with
>> libdap and libnc-dap always requires a few extra libs.
>
> pkgconfig usually takes care of these details.
>
>> I put libcurl.a and
>> libxml2.a in supplibs/lib, but there are still others, and these  
>> are not the
>> same for every OS. The dynamic macros use the output from "ncdap- 
>> config
>> --libs", but on my mac this left out some of the required ones so  
>> I have
>> hard-coded a kludge:
>>     if test "$is_darwin" = "yes" ; then
>>       dap_extra_libs="-lssl -lcrypto -lpthread -liconv -lm"
>>     else
>>       dap_extra_libs="-lidn -lssl -lcrypto -ldl -lpthread -lm"
>>     fi
>>     dap_libs="$gadap_lib $dap_libs $dap_extra_libs"
>> This syntax gives me what I need to build on our mac and linux  
>> boxes, but
>> will probably be wrong for sunOS and others. Since we have checks for
>> host-specific options, any platform-specific dap_extra_libs can be  
>> added as
>> necessary.
>
> Consider using pkgconfig. None better than the library developers to
> know what is required for a given version; pkgconfig provides the
> mechanism for them to convey this information.
>
>> Finally, I had wanted to include library version numbers in the 'q  
>> config'
>> output, but that turned out to be tricky if the library did not  
>> contain a
>> routine for spitting out a version string. Netcdf has  
>> nc_inq_libvers(), but
>> many of the others simply have #define statements in the header  
>> files, but
>> this was highly version-dependent and will cause problems if using  
>> system
>> libraries instead of supplibs. I left some of that code in gacfg.c  
>> but other
>> bits are commented out. If you guys have any ideas for how to do  
>> this, maybe
>> there's a way to do it during the configuration process and  
>> populate GrADS's
>> own config.h with strings like #define GA_ZLIB_VERSION 1.2.3 ... then
>> gacfg.c need only look in config.h to get that info.
>
> The OpenGrADS supplibs have a tag corresponding to each supplib
> version, say 2.0.1 (the one I use to build grads2). In the very least,
> gacfg.c could report the supplibs version used for the build. I may
> need to add a way for this version number to be retrieved...
>
> For tracking the version of the individual packages, as long as the
> packages are using autoconf (true for supplibs-2.0.x) you can get the
> version number by parsing configure.ac/configure.in during the
> supplibs build, e.g.,
>
> % grep ^AC_INIT netcdf/configure.ac
> AC_INIT([netCDF], [3.6.2], [su...@un...])
>
> A simple script could added to the supplibs build to compile this
> information in a simple text file, say "supplibs.h". Are you willing
> to adopt (and help maintain) the supplibs-2.0.x?
>
>> Well, that's enough. If you've read this far, I thank you  
>> sincerely for your
>> attention to these arcane details.
>
> This is the reality of autoconf.
>
>    Arlindo
>
> -- 
> Arlindo da Silva
> da...@al...
>
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Arlindo da S. <da...@al...>]

On Mon, Aug 11, 2008 at 9:36 AM, Jennifer Adams <jm...@co...> wrote:

>
> It will be easier to discuss when 2.0.a3 is out, but ... here are examples
> of the kind of stuff I changed in configure.ac:
>

OK, I'll wait until a3 is out and then try to reconcile.


> As for the supplibs/include subdirectories, I renamed grib2->grib2c,
>

You mean, grib2c->grib2? This kind of renaming end up causing me a lot of
extra work for the supplibs-2.0.x, things like having to make changes to the
repository, build mechanism, etc.

BTW, are you using grib2c from the supplibs-2.0.x? I've added autoconf to it
and contributed to NCEP, but I am not sure if they have adopted it.


> and split dap into gadap and libnc-dap.
>

In the supplibs 2.0.x we already had 3 libraries:

gadap
libdap
libnc-dap

Have you actually tried a build using the pre-compiled supplibs-2.0.1?


http://sourceforge.net/project/showfiles.php?group_id=161773&package_id=241681

In v1.9.0-rc1, with pre-compiled supplibs, GrADS builds very simply out of
the tarball:


http://opengrads.org/wiki/index.php?title=Building_GrADS_v1.9_from_Sources

The idea is for people not to have to do any renaming, rearranging of the
the supplibs. I actually started working on a perl script that detects
people's hardware, then retrieves pre-compiled supplibs + grads sources
automatically, does the build and cleans up the mess. Very easy.

But the libpng12 name comes from their source code, to distinguish from
> earlier versions, so I think we should leave that as is (they renamed the
> library too, libpng12.a).
>

In the supplibs-2.0.x libpng12 is symlinked to libpng. In any case,
pkg-config takes care of it, better not to hardwire the name of the library,
include directory.


>  The remainder of the include directories that weren't in my list may be
> essential for building all the supplibs in a coherent way, but they are not
> necessary for compiling GrADS -- I generally extract only what GrADS needs
> from a complete library installation and put that into my supplibs location.
>
>

I'd say that these are necessary not only for building the supplibs
consistently but more importantly for getting *portable* GrADS binaries. For
example, using a system provided libcurl that depends on openssl does not
lead to portable binaries, I learned it the hard way (I believe your
binaries have this problem).  The advantage of having portable binaries is
that you can have a generic Linux i686 binary, rather than binaries for
specific distros.


> I should mention that I never build with GUI, so I don't know what's
> required there.
>

The supplibs-2.0.x should have all you need.

BTW, are you planing to include the pytests/ in a3?

  I look forward to seeing 2.0.a3.

    Thanks,

       Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] dynamic supplibs checking

[Jennifer A. <jm...@co...>]

The src and some binaries are out now. ftp://grads.iges.org/grads/2.0/

The supplibs/include directory names I used are like this:
gadap/  gd/  grib2c/  hdf/  libnc-dap/  libpng12/  netcdf/   
readline/  udunits/  zlib/

I have not used your pre-compiled supplibs. I believe the new code  
should work with them, but I'm sure some changes to configure.ac and  
Makefile.am will be required. For my pre-compiled builds, I rebuilt  
the curl library --without-libidn and --without-ssl and then rebuilt  
all the *dap*.a libraries around that. Hopefully they will be more  
portable. The python interface is not included; without any knowledge  
of python, I can't realistically support it.

>

--Jennifer



On Aug 12, 2008, at 1:16 AM, Arlindo da Silva wrote:

> On Mon, Aug 11, 2008 at 9:36 AM, Jennifer Adams <jm...@co...>  
> wrote:
>
> It will be easier to discuss when 2.0.a3 is out, but ... here are  
> examples of the kind of stuff I changed in configure.ac:
>
> OK, I'll wait until a3 is out and then try to reconcile.
>
> As for the supplibs/include subdirectories, I renamed grib2->grib2c,
>
> You mean, grib2c->grib2? This kind of renaming end up causing me a  
> lot of extra work for the supplibs-2.0.x, things like having to  
> make changes to the repository, build mechanism, etc.
>
> BTW, are you using grib2c from the supplibs-2.0.x? I've added  
> autoconf to it and contributed to NCEP, but I am not sure if they  
> have adopted it.
>
> and split dap into gadap and libnc-dap.
>
> In the supplibs 2.0.x we already had 3 libraries:
>
> gadap
> libdap
> libnc-dap
>
> Have you actually tried a build using the pre-compiled supplibs-2.0.1?
>
>    http://sourceforge.net/project/showfiles.php? 
> group_id=161773&package_id=241681
>
> In v1.9.0-rc1, with pre-compiled supplibs, GrADS builds very simply  
> out of the tarball:
>
>    http://opengrads.org/wiki/index.php? 
> title=Building_GrADS_v1.9_from_Sources
>
> The idea is for people not to have to do any renaming, rearranging  
> of the the supplibs. I actually started working on a perl script  
> that detects people's hardware, then retrieves pre-compiled  
> supplibs + grads sources automatically, does the build and cleans  
> up the mess. Very easy.
>
> But the libpng12 name comes from their source code, to distinguish  
> from earlier versions, so I think we should leave that as is (they  
> renamed the library too, libpng12.a).
>
> In the supplibs-2.0.x libpng12 is symlinked to libpng. In any case,  
> pkg-config takes care of it, better not to hardwire the name of the  
> library, include directory.
>
>  The remainder of the include directories that weren't in my list  
> may be essential for building all the supplibs in a coherent way,  
> but they are not necessary for compiling GrADS -- I generally  
> extract only what GrADS needs from a complete library installation  
> and put that into my supplibs location.
>
> I'd say that these are necessary not only for building the supplibs  
> consistently but more importantly for getting *portable* GrADS  
> binaries. For example, using a system provided libcurl that depends  
> on openssl does not lead to portable binaries, I learned it the  
> hard way (I believe your binaries have this problem).  The  
> advantage of having portable binaries is that you can have a  
> generic Linux i686 binary, rather than binaries for specific distros.
>
> I should mention that I never build with GUI, so I don't know  
> what's required there.
>
> The supplibs-2.0.x should have all you need.
>
> BTW, are you planing to include the pytests/ in a3?
>
>   I look forward to seeing 2.0.a3.
>
>     Thanks,
>
>        Arlindo
>
> -- 
> Arlindo da Silva
> da...@al...
> ---------------------------------------------------------------------- 
> ---
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/ 
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] dynamic supplibs checking

[Arlindo da S. <da...@al...>]

On Tue, Aug 12, 2008 at 10:00 AM, Jennifer Adams <jm...@co...> wrote:

> The src and some binaries are out now. ftp://grads.iges.org/grads/2.0/
>
> The supplibs/include directory names I used are like this:
> gadap/  gd/  grib2c/  hdf/  libnc-dap/  libpng12/  netcdf/  readline/
>  udunits/  zlib/
>
> I have not used your pre-compiled supplibs. I believe the new code should
> work with them, but I'm sure some changes to configure.ac and Makefile.am
> will be required.
>

If you give me a sneak preview of a3 I can make sure it builds with
supplibs-2.0.1. It would be much less work than patch and rebuild the
supplibs on all the different platforms. I rather spend the time providing
builds and supplibs for new platforms.



> For my pre-compiled builds, I rebuilt the curl library --without-libidn and
> --without-ssl and then rebuilt all the *dap*.a libraries around that.
> Hopefully they will be more portable. The python interface is not included;
> without any knowledge of python, I can't realistically support it.
>

The pytests/ are not the python interface. I just have a small part of it
(1 file) built in to support the tests. I consider it part of the build as
it allows you a straightforward way to test what you have built. Nothing
python gets installed or distributed, it is just an aid for the builder. It
is just a drop in and uses standard python libraries, no extra install
required (you need python 2.3 or later, pretty common these days).

  BTW, grads v2 is getting nice and stable stable. Isn't about time for a
"b3"? Some people simply would not touch alpha code. Alpha usually means
code that is unstable and with known bugs; you are passed this point.

     Arlindo

-- 
Arlindo da Silva
da...@al...