Re: [htdig-dev] to-do list for 3.1.6

[Joe R. J. <jj...@cl...>]

On Thu, 29 Nov 2001, Gilles Detillieux wrote:

> Date: Thu, 29 Nov 2001 11:55:08 -0600 (CST)
> From: Gilles Detillieux <gr...@sc...>
> To: Joe R. Jah <jj...@cl...>
> Cc: Geoff Hutchison <ghu...@ws...>,
     "ht://Dig developers list" <htd...@li...>
> Subject: Re: [htdig-dev] to-do list for 3.1.6
> 
> Wait a minute.  I'm almost positive that the problem on BSDi was not the
> use of the system's regex.h, but rather the use of the regex code that's
> bundled with htdig!  Can you do me a favour and have a look at what the
> value of HAVE_BROKEN_REGEX is in include/htconfig.h?  If it's #define'd,
> please try to #undef it and recompile/reinstall htdig, and let us know
> how that impacts digging time.  The selection of the regex code has to
> be an all or nothing thing.  If you use the bundled code, all source
> files that use regex.h should use the bundled one, but if you use the C
> library regex code, then all source files should use the system's regex.h.
> If you mix and match the two, you're likely to run into problems.
> 
> I think we need to fix htfuzzy/EndingsDB.cc to check the setting
> of HAVE_BROKEN_REGEX and use the appropriate header file.  Come to
> think of it, I think there's a problem with how HAVE_BROKEN_REGEX is
> handled in htlib/HtRegex.h too, because simply using #include <regex.h>
> doesn't guarantee that the compiler won't use the bundled one instead,
> as the Makefile.config file puts a -I../htlib in the compiler flags.
> I think to make this all work, we need to rename the bundled regex.h to
> something like htregex.h to avoid conflicts, as well as put some hooks
> in the bundled regex.c code to disable it all if you need to use the
> C library code instead.  What do you think, Geoff?
> 
> However, Joe, if you did remove both htlib/regex.c and htlib/regex.h
> as you said you did, then you should be safely using all the C library
> code, and not the bundled code, so it should be good.
> 
> I also had a look at the HAVE_BROKEN_REGEX on my Red Hat 4.2 system,
> and surprisingly it was defined.  I say surprisingly because previously
> I had tried to manually force it to use the C library regex, as Joe
> does by removing the bundled code and removing the reference to regex.o
> in htlib/Makefile, and that had caused htfuzzy and htsearch to crash.
> That was on 3.1.4 or 2.1.5, I think.  Anyway, 3.1.6 isn't crashing on this
> system, so I'd see this as further evidence that the HAVE_BROKEN_REGEX
> handling is not working.  So, we need to fix the usage of the flag.
> We also need to fix the test for this situation, because it should not
> be defining this flag on my RH 4.2 system.  Finally, I think we need a
> better name for it, as it implies that the C library regex is broken,
> when in fact the problem on BSDi systems is that the bundled regex code
> conflicts with the libraries in some way.
> 
> In the end, it might make sense to have a configure option to override
> the automatic test for this, because I'm not convinced it will work in
> all cases.  (However, to the best of my recollection, it is only BSDi
> systems that have a problem with the bundled regex code.)

Here is some statistics that may address your questions;)

___________________ 112501 + parsedate.0 + ssl.6 ___________________
Without any change.
htdig: Start digging:   Thu Nov 29 23:27:58 PST 2001
htmerge: Start merging: Thu Nov 29 23:33:35 PST 2001    337 seconds
htmerge: Total word count: 13160
htmerge: Total documents: 163
htmerge: Total size of documents (in K): 1904
____________________________________________________________________
___________________ 112501 + parsedate.0 + ssl.6 ___________________
remove reference to regex.o in htlib/Makefile

htdig: Start digging:   Thu Nov 29 23:42:15 PST 2001
htmerge: Start merging: Thu Nov 29 23:44:38 PST 2001    143 seconds
htmerge: Total word count: 13160
htmerge: Total documents: 163
htmerge: Total size of documents (in K): 1904
____________________________________________________________________
___________________ 112501 + parsedate.0 + ssl.6 ___________________
rm htlib/regex.*
remove reference to regex.o in htlib/Makefile
#undef HAVE_BROKEN_REGEX in include/htconfig.h

htdig: Start digging:   Thu Nov 29 22:22:32 PST 2001
htmerge: Start merging: Thu Nov 29 22:24:14 PST 2001	104 seconds
htmerge: Total word count: 13160
htmerge: Total documents: 163
htmerge: Total size of documents (in K): 1904
____________________________________________________________________
___________________ 112501 + parsedate.0 + ssl.6 ___________________
rm htlib/regex.*
remove reference to regex.o in htlib/Makefile
#define HAVE_BROKEN_REGEX in include/htconfig.h

htdig: Start digging:   Thu Nov 29 22:25:33 PST 2001
htmerge: Start merging: Thu Nov 29 22:27:12 PST 2001	 99 seconds
htmerge: Total word count: 13160
htmerge: Total documents: 163
htmerge: Total size of documents (in K): 1904
____________________________________________________________________
____________________ 092301 + Armstrong + ssl.4 ____________________
htdig: Start digging:   Fri Nov 30 00:18:06 PST 2001
htmerge: Start merging: Fri Nov 30 00:18:44 PST 2001     38 seconds
htmerge: Total word count: 13160
htmerge: Total documents: 163
htmerge: Total doc db size (in K): 1904
____________________________________________________________________

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...