htdig-dev — Developer Discussion for the ht://Dig project

Re: [htdig-dev] Re: Fwd: OS X and libtool feedback

[Jim C. <li...@yg...>]

On Thursday, May 29, 2003, at 07:33 PM, Lachlan Andrew wrote:

> I think that the problem on the compile farm was that the dynamic C++
> library doesn't actually *exist* -- not just that it isn't being

I checked another OS X box that I know uses a generic install straight 
off of an Apple CD (installed only a few weeks ago) and found that the 
dynamic library is not present. Apparently the dynamic version is not 
standard. I guess the copy I have must have come from one of the 
Developer Tool downloads, but given the way that it installs it still 
seems that it is not really intended for general use. Seems that we are 
probably fighting a losing battle on this one, at least for the time 
being. We can always add a FAQ telling people to use --disable-shared 
if it comes up a lot.

Jim

Re: [htdig-dev] Re: OS X

[Jim C. <li...@yg...>]

On Thursday, May 29, 2003, at 08:18 PM, Lachlan Andrew wrote:

> I just meant
>    (b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
> From a fresh install, just add "robots..." to your htdig.conf file,
> and then rundig.

I pulled a fresh copy last night (May 29), built with --disable-shared, 
added the robots_name attribute, and did a rundig. The dig ran to 
completion with no obvious errors. The databases are searchable. The 
db.words.db database ended up a little under 500 MB.

The only warnings were the regex related multiple definitions and the 
one warning involving lex. All of the 'make check' tests passed.

Jim

[htdig-dev] Re: HTDig on IA64 Linux

[Lachlan A. <lh...@us...>]

Greetings David,

The snapshot would be about a week old (judging from the=20
'wordlist_cache_dirty_level=3D40' suggestion).  Try setting =20
wordlist_cache_dirty_level=3D4  (the new default value) and it should=20
work.  If not, try it reducing it further (to a minimum of 1), and=20
let us know about it.

Thanks for the feedback!  (Once you get it working, it would be great=20
if you could run some tests for us, so we could list IA64 as a=20
supported platform.  No worries if you're not in a position to do=20
that.)

Lachlan

(It's good to see some other Melbournians interested in ht://Dig :)

On Fri, 30 May 2003 14:40, David Bannon wrote:

> 3.2.04 CVS snapshots - compiles and installs cleanly and indexes
> one of the virtual sites in question. But fails with the other.
> Specificially, it indexes for far too long (its only a 2Meg site,
> 10 or more minutes) and then produces an endless list of error
> messages like this :
>
> ..........
> Try setting  wordlist_cache_dirty_level=3D40  in configuration file
> WordDB: CDB___memp_cmpr_alloc: unexpected error from weakcmpr base
> WordDB: PANIC: Cannot allocate memory
> WordDB: PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
> WordDB: /opt/www/db/db.words.db: write failed for page 1668
> WordDB: Unable to allocate 8271 bytes from mpool shared region:
> Cannot allocate memory
> ..........
>
> Seems that its recursing, or perhaps going out and following links
> to other sites (it should not).
>
> Any suggestions of what course I should take with debugging ??
>
> Phone 61 03 9925 4733              Fax 61 03 9925 4647
> Mobile 0418 525687                 http://www.vpac.org
> Systems Manager, Victorian Partnership for Advanced Computing
> -------------------------------------------------------------
> ..... Humpty Dumpty was pushed !

[htdig-dev] HTDig on IA64 Linux

[David B. <D.B...@vp...>]

Hi Folks,

Wondering if any interest in my experience of running HTDig on IA64 Linux,
We have a dual Itanium II Box that I have been moving an existing web site
to. And run into some 'incidents'.

3.1.6 - It appears that there is an issue within the Berkley Database that
makes this unusable. It compiles and installs cleanly. But when I run
htdig -i I get a message about not being able to open/create the database
files. It is not an issue of diskspace or permissions. I traced it down as
far as the Berkely api db_open(). Most likely a data type size issue
relating to passed parameters, I suspect that the system is assuming 32
system because its linux. Hmm, started to lose interest.

3.2.03b - Tells me that it does not support IA64.

3.2.04 CVS snapshots - compiles and installs cleanly and indexes one of the
virtual sites in question. But fails with the other. Specificially, it
indexes for far too long (its only a 2Meg site, 10 or more minutes) and then
produces an endless list of error messages like this :

..........
Try setting  wordlist_cache_dirty_level=40  in configuration file
WordDB: CDB___memp_cmpr_alloc: unexpected error from weakcmpr base
WordDB: PANIC: Cannot allocate memory
WordDB: PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
WordDB: /opt/www/db/db.words.db: write failed for page 1668
WordDB: Unable to allocate 8271 bytes from mpool shared region: Cannot
allocate memory
..........

Seems that its recursing, or perhaps going out and following links to other
sites (it should not).

Any suggestions of what course I should take with debugging ??


-------------------------------------------------------------
David Bannon                       D.B...@vp...
Phone 61 03 9925 4733              Fax 61 03 9925 4647
Mobile 0418 525687                 http://www.vpac.org
Systems Manager, Victorian Partnership for Advanced Computing
-------------------------------------------------------------
..... Humpty Dumpty was pushed !

Re: [htdig-dev] Re: Fwd: OS X and libtool feedback

[Peter O'G. <pe...@po...>]

On Friday, May 30, 2003, at 10:33  AM, Lachlan Andrew wrote:
>
> I think that the problem on the compile farm was that the dynamic C++
> library doesn't actually *exist* -- not just that it isn't being
> found.  That means that it *should* actually still have worked on a
> system with that library, and a dynamic  zlib  (but as my email said,
> I'm not holding my breath).
>

That is correct. Apple's gcc does not ship a shared libstdc++ (although 
this may change with 10.3 and Apple's version of gcc-3.3 (when I built 
from apple darwin cvs I got a shared libstdc++). I was promised that 
10.3 would also include libtool-1.5, but who knows.

Any version of libtool < 1.5 will refuse to link a static archive into 
a shared library on darwin, this was "fixed" in 1.5 by setting the 
deplibs_check_method to pass_all.

What I've seen another project (gnucash) do is to append the entire 
contents of libtool.m4 to acinclude.m4 and check in a matching version 
of ltmain.sh to cvs. In their autogen script they run aclocal, 
automake, autoheader autoconf etc, but do not run libtoolize. This may 
also work for htdig.

libtool issues totally suck :(

Peter

Re: [htdig-dev] Re: OS X

[Lachlan A. <lh...@us...>]

I just meant
   (b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
=46rom a fresh install, just add "robots..." to your htdig.conf file,=20
and then rundig.

(The 700MB was getting confused with the test I did on my department's=20
site.  htdig.org only gives 400MB :)

Cheers,
Lachlan

On Thu, 29 May 2003 13:41, Jim Cole wrote:
> On Wednesday, May 28, 2003, at 04:22 AM, Lachlan Andrew wrote:
> > Excellent news!  Are you in a position to run the "big" dig at
> > some stage?  (The database file is about 700MB.)
>
> Sorry. What is the "big" dig? I suspect I can give that a shot once
> I figure out what it is :) I will pull a fresh copy of the code
> from CVS before starting another test.
>
> Jim

Re: [htdig-dev] Re: Fwd: OS X and libtool feedback

[Lachlan A. <lh...@us...>]

On Fri, 30 May 2003 00:05, Ted Stresen-Reuter wrote:
> (Pulling hair out of top of head)
> AHHHHHRRRGGGGG!!!

*sheepish grin*  I'm sorry for leaving CVS potentially broken last=20
night, and also very sorry if my email sounded dismissive of OS X=20
users' woes.  Needless to say, I was rather tired and emotional...

I have backed out the change, it works again on the compile farm. Four=20
tests fail because  httpd  doesn't work.  It used to, but it doesn't=20
on the other computers on the farm, so I assume it has just been=20
disabled...

It could well have worked properly with --disable-shared  (if the new=20
libtools doesn't need the hacks I put in earlier) but I was too tired=20
to test it.  Again, sorry for the frustration caused by the=20
exageration in my email.

I think that the problem on the compile farm was that the dynamic C++=20
library doesn't actually *exist* -- not just that it isn't being=20
found.  That means that it *should* actually still have worked on a=20
system with that library, and a dynamic  zlib  (but as my email said,=20
I'm not holding my breath).

Could someone who actually has a Mac try Peter's fix (which I think is=20
probably the correct one)?

Lachlan

[htdig-dev] Re: OS X and libtool feedback

[Peter O'G. <pe...@po...>]

>>
>> I applied Peter O'Gorman's patch, re-libtooled etc, and no luck :(
>>
>> It may be because the setup on the compile farm is rather messed up,
>> so others are welcome to try it, but I won't be holding my breath.
>>

Okay, I'm convinced, will look into it.

A question:
You re-libtooled using the glibtoolize on the compile farm OS X box? 
(if so, you get a broken libtool-1.4.2)


Peter

Re: [htdig-dev] Re: Fwd: OS X and libtool feedback

[Ted Stresen-R. <ted...@ma...>]

(Pulling hair out of top of head)

AHHHHHRRRGGGGG!!!

Ted Stresen-Reuter

On Thursday, May 29, 2003, at 08:53  AM, Lachlan Andrew wrote:

> Greetings all,
>
> I applied Peter O'Gorman's patch, re-libtooled etc, and no luck :(
>
> It may be because the setup on the compile farm is rather messed up,
> so others are welcome to try it, but I won't be holding my breath.
>
> The bad news is that I had to commit the change to get it onto the
> compile farm (their scp and sftp server is seems to be broken), and
> the regenerated files don't have the kludges to allow --enable-static
> --disable-shared  to work, so OS X support is back to zero.
>
> If anyone wants to re-do those patches, or to undo the latest commit,
> be my guest.  I'm going to bed...
>
> Lachlan
>   ZZZzzz...
>
> (The good news is that SunOS cc now works.)
>
> On Wed, 28 May 2003 23:52, Ted Stresen-Reuter wrote:
>>> From: "Peter O'Gorman" <pe...@po...>
>>>
>>> Well, after a bit of confusion as to why #include <iostream.h>
>>> would complain about memcpy and memcmp etc not being declared, I
>>> went to bed, woke up, had coffee, made the following change to
>>> acconfig.h, and it all compiled happily with libtool-1.5
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: eBay
> Get office equipment for less on eBay!
> http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
> _______________________________________________
> htdig-dev mailing list
> htd...@li...
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

[htdig-dev] Re: Fwd: OS X and libtool feedback

[Lachlan A. <lh...@us...>]

Greetings all,

I applied Peter O'Gorman's patch, re-libtooled etc, and no luck :(

It may be because the setup on the compile farm is rather messed up,=20
so others are welcome to try it, but I won't be holding my breath.

The bad news is that I had to commit the change to get it onto the=20
compile farm (their scp and sftp server is seems to be broken), and=20
the regenerated files don't have the kludges to allow --enable-static=20
--disable-shared  to work, so OS X support is back to zero.

If anyone wants to re-do those patches, or to undo the latest commit,=20
be my guest.  I'm going to bed...

Lachlan
  ZZZzzz...

(The good news is that SunOS cc now works.)

On Wed, 28 May 2003 23:52, Ted Stresen-Reuter wrote:
> > From: "Peter O'Gorman" <pe...@po...>
> >
> > Well, after a bit of confusion as to why #include <iostream.h>
> > would complain about memcpy and memcmp etc not being declared, I
> > went to bed, woke up, had coffee, made the following change to
> > acconfig.h, and it all compiled happily with libtool-1.5

Re: [htdig-dev] Re: OS X

[Jim C. <li...@yg...>]

On Wednesday, May 28, 2003, at 04:22 AM, Lachlan Andrew wrote:

> Excellent news!  Are you in a position to run the "big" dig at some
> stage?  (The database file is about 700MB.)

Sorry. What is the "big" dig? I suspect I can give that a shot once I 
figure out what it is :) I will pull a fresh copy of the code from CVS 
before starting another test.

Jim

[htdig-dev] SunOS cc problem

[Lachlan A. <lh...@us...>]

Greetings all,

The problem with  make check  on SunOS with native cc is the size of =20
off_t, the size of an offset in a file.  This seems to be related to=20
the  --enable-bigfile  configure option.  Does anyone remember=20
dealing with this issue before?

Thanks,
Lachlan

Re: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525

[Lachlan A. <lh...@us...>]

I agree -- win32 will be nice if the delay is up to a week or so, but=20
isn't worth delaying b5 too long for.

$0.02

Lachlan

On Thu, 29 May 2003 00:59, Gilles Detillieux wrote:

> As for the win32 support, I think it would be nice to include it if
> it won't delay the release too much.  If it's likely to entail huge
> code changes requiring lots more debugging and testing time, and
> will push the release back more than a couple weeks, my own feeling
> would be to hold off on this until b5 is out.  What do others
> think?

Re: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525

[Gilles D. <gr...@sc...>]

According to Neal Richter:
> Question:  I've got a bunch of native win32 support changes I've been
> sitting on (and am now reverifying).
> 
> Should I sit on them until we release 3.2.0b5??
> 
> What about fixes for memory leaks???
> 
> I'm back to a period here at work where I can work full-time on HtDig code
> itself to make it work natively with Win32....
> 
> Thanks.

Please don't sit on fixes for memory leaks, unless you have reason
to believe the fixes are system specific and will likely cause the
code to break on other systems.  3.2.0b5 isn't in a code freeze yet,
and even if it were we try to get bug fixes in as much as possible
in the pre-release period.  Fixing memory leaks before the code is
released is likely to benefit everyone, isn't it?

As for the win32 support, I think it would be nice to include it if
it won't delay the release too much.  If it's likely to entail huge
code changes requiring lots more debugging and testing time, and will
push the release back more than a couple weeks, my own feeling would
be to hold off on this until b5 is out.  What do others think?

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)

[htdig-dev] Fwd: OS X and libtool feedback

[Ted Stresen-R. <ted...@ma...>]

This is the reply I got from Peter O'Gorman when trying to compile 
htdig on OS X with shared libraries. I don't think he was working on 
the most recent snapshot, but I'd have to confirm that with him 
directly.

Hope this helps!

BTW, excellent work, Jim. You deserve some kind of reward... If you 
ever pass through Chicago, I'd like to buy you a drink.

Ted Stresen-Reuter

Begin forwarded message:

> From: "Peter O'Gorman" <pe...@po...>
> Date: Tue May 27, 2003  11:10:05  PM America/Chicago
> To: Ted Stresen-Reuter <ted...@ma...>
> Subject: Re: question about your modification to libtool
>
> Well, after a bit of confusion as to why #include <iostream.h> would 
> complain about memcpy and memcmp etc not being declared, I went to 
> bed, woke up, had coffee, made the following change to acconfig.h, and 
> it all compiled happily with libtool-1.5
>
> Thanks,
> Peter
>
> Reasons for this change:
> If we do #if HAVE_MEMCMP etc in the @TOP@ section of acinclude, it 
> will appear before the #define HAVE_MEMCMP.. a bad idea. So instead we 
> move the @BOTTOM@ tag up about 30 lines and all is well with the > world.
>
> diff -u -d -b -w -r1.6 acconfig.h
> --- acconfig.h  2002/12/31 07:59:02     1.6
> +++ acconfig.h  2003/05/28 00:40:24
> @@ -71,6 +71,7 @@
>  /* Define if we should use rxposix.h instead of regex.h */
>  #undef USE_RX
>
> +@BOTTOM@
>  /*
>   * Don't step on the namespace.  Other libraries may have their own
>   * implementations of these functions, we don't want to use their
> @@ -101,7 +102,7 @@
>  #define        vsnprintf       __db_Cvsnprintf
>  #endif
>
> -@BOTTOM@
> +
>
>  /*
>   * Big-file configuration.
>

[htdig-dev] Re: OS X

[Lachlan A. <lh...@us...>]

Great work, Jim!

On Wed, 28 May 2003 14:38, Jim Cole wrote:
> The most common warning is that /usr/include is explicitly provided

Yep, that breaks SunOS, and is fixed by the new zlib check (committed=20
yesterday).

> Another frequent warning is the following.
>     warning: redeclaration of C++ built-in type `wchar_t'
> I think this warning results Since /usr/include is no longer
> considered a system include directory,

Good point.  Could you try it again with the new  configure?

> A couple other warnings involve lex.
>
> conf_lexer.cxx: In function `int yylex()':
> conf_lexer.cxx:704: warning: label `find_rule' defined but not used
> /usr/include/gcc/darwin/3.1/g++-v3/streambuf: At top level:
> conf_lexer.cxx:1790: warning: `void* yy_flex_realloc(void*,
> unsigned int)'
>     defined but not used

I had been leaving them because this file is automatically generated=20
by flex.  I'll hack it for now, but does anyone know a clean way to=20
fix this?

> Finally, there are three link warnings (repeated multiple times)
> involving multiple definitions of _regcomp, _regexec, and _regfree.

Is there anything we can do about that, other than rename all of our=20
functions?  Suggestions, anyone?

> As for 'make check' the code now compiles (with the same warnings
> as above) and all tests pass. There is one possible glitch in that
> an attempt to find HtFileType fails.
> Checking after the fact, the file appears to be there.

It's put there by  make install  -- I haven't got around to setting=20
the path in  test/conf/htdig.conf  to be relative to the build=20
directory, rather than the install directory...

> The only other issue that I encountered involves htfuzzy.
> The 'Rejected' notice appears to be due to the blank line at the
> end of the synonyms file.

Yep, that's fixed too.

> I have tried a couple digs and have not encountered any crashes or
> noticeable database corruption; I do have compression enabled.

Excellent news!  Are you in a position to run the "big" dig at some=20
stage?  (The database file is about 700MB.)

Cheers,
Lachlan

[htdig-dev] OS X

[Jim C. <li...@yg...>]

Hi - Here is a summary of what I found when using the current CVS 
version (as of 05/25) under OS X.

Everything compiles, but there are a few warnings. The most common 
warning is due to the fact that /usr/include is explicitly provided via 
-I. This at least has the potential to create other problems. According 
to the GCC documentation...

If a standard system include directory, or a directory specified with 
-isystem, is also specified with -I, it will be searched only in the 
position requested by -I. Also, it will not be considered a system 
include directory. If that directory really does contain system 
headers, there is a good chance that they will break. For instance, if 
GCC's installation procedure edited the headers in /usr/include to fix 
bugs, -I/usr/include will cause the original, buggy headers to be found 
instead of the corrected ones. GCC will issue a warning when a system 
include directory is hidden in this way.

I think this only affects version 3.1, with explicit specification of 
system include directories being ignored in 3.2 and 3.3.

Another frequent warning is the following.

    warning: redeclaration of C++ built-in type `wchar_t'

I think this warning results from the problem introduced by explicitly 
specifying system include directories. Since /usr/include is no longer 
considered a system include directory, files in that directory are seen 
as redeclaring wchar_t. Well, that is my guess anyway.

A couple other warnings involve lex.

In file included from conf_lexer.cxx:24:
/usr/include/stdlib.h:77: warning: redeclaration of C++ built-in type 
`wchar_t'
conf_lexer.cxx: In function `int yylex()':
conf_lexer.cxx:704: warning: label `find_rule' defined but not used
/usr/include/gcc/darwin/3.1/g++-v3/streambuf: At top level:
conf_lexer.cxx:1790: warning: `void* yy_flex_realloc(void*, unsigned 
int)'
    defined but not used

Finally, there are three link warnings (repeated multiple times) 
involving multiple definitions of _regcomp, _regexec, and _regfree. 
They look something like

ld: warning multiple definitions of symbol _regcomp
../htlib/.libs/libht.a(regex.o) definition of _regcomp in section 
(__TEXT,__text)
/usr/lib/libSystem.dylib(regcomp.So) definition of _regcomp

As for 'make check' the code now compiles (with the same warnings as 
above) and all tests pass. There is one possible glitch in that an 
attempt to find HtFileType fails. The output generated by 'make check' 
includes

PASS: t_htmerge
PASS: t_htnet
sh: /Users/greyleaf/local/htdig_cvs/bin/HtFileType: No such file or 
directory
PASS: t_htdig_local
===================
All 14 tests passed

Checking after the fact, the file appears to be there.

The only other issue that I encountered involves htfuzzy. Processing of 
synonyms generates the following output.

htfuzzy/synonyms: 1500 willfull
htfuzzy/synonyms: 1510 woolie
htfuzzy/synonyms: Rejected line with less than 2 words:

htfuzzy/synonyms: 1519 worshipping
htfuzzy/synonyms: Done.
htfuzzy: Done.

The 'Rejected' notice appears to be due to the blank line at the end of 
the synonyms file.

Aside from the above (and the known issue with dynamic builds) things 
seem to be working reasonably well. I have tried a couple digs and have 
not encountered any crashes or noticeable database corruption; I do 
have compression enabled.

Jim

Re: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525

[Neal R. <ne...@ri...>]

Question:  I've got a bunch of native win32 support changes I've been
sitting on (and am now reverifying).

Should I sit on them until we release 3.2.0b5??

What about fixes for memory leaks???

I'm back to a period here at work where I can work full-time on HtDig code
itself to make it work natively with Win32....

Thanks.

On Sun, 25 May 2003, Geoff Hutchison wrote:

> STATUS of ht://Dig branch 3-2-x
>
> CHECKLIST FOR 3.2.0b5:
> * Add more items to checklist :-)
> * Must be able to
> 	(a) make check  and
> 	(b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
>   on all systems listed as "supported".
>   Systems tested so far:
>     - Mandrake 8.2, gcc 3.2 (lha, 21 May)
>     - FreeBSD 4.6, gcc 2.95.3 (lha, 23 May)
>     - Debian, Linux kernel 2.2.19, gcc 2.95.4 (lha, 23 May)
>   Partly tested:
>     - SunOS 5.8 = Solaris 2.8, gcc 3.1 (lha.  Makes check.  Big dig not tested)
>     - SunOS 5.8 = Solaris 2.8, Sun cc with g++ 3.1 (lha. Compiles, fails check)
>   To be tested:
>     - RedHat anyone?
>     - OS X anyone?
> * Latin-encodings (Gilles)
> * Check bugs listed in bug-tracker...
> * Polish release docs (Geoff)
>
>
>
>
>
> RELEASES:
>    3.2.0b5: Next release, June 2003???
>    3.2.0b4: "In progress" -- snapshots called "3.2.0b4" until prerelease.
>    3.2.0b3: Released:  22 Feb 2001.
>    3.2.0b2: Released:  11 Apr 2000.
>    3.2.0b1: Released:   4 Feb 2000.
>
> (Please note that everything added here should have a tracker PR# so
> we can be sure they're fixed. Geoff is currently trying to add PR#s for
> what's currently here.)
>
> SHOWSTOPPERS:
> * Mifluz database errors are a severe problem (PR#428295)
> -- Does Neal's new zlib patch solve this for now?
>
> KNOWN BUGS:
> * Odd behavior with $(MODIFIED) and scores not working with
>    wordlist_compress set but work fine without wordlist_compress.
>    (the date is definitely stored correctly, even with compression on
>     so this must be some sort of weird htsearch bug) PR#618737.
> * META descriptions are somehow added to the database as FLAG_TITLE,
>    not FLAG_DESCRIPTION. (PR#618738)
>    Can anyone reproduce this?  I can't! -- Lachlan
>
> PENDING PATCHES (available but need work):
> * Additional support for Win32.
> * Memory improvements to htmerge. (Backed out b/c htword API changed.)
> * Mifluz merge.
>
> NEEDED FEATURES:
> * Quim's new htsearch/qtest query parser framework.
> * File/Database locking. PR#405764.
>
> TESTING:
> * httools programs:
>   (htload a test file, check a few characteristics, htdump and compare)
> * Tests for new config file parser
> * Duplicate document detection while indexing
> * Major revisions to ExternalParser.cc, including fork/exec instead of popen,
>   argument handling for parser/converter, allowing binary output from an
>   external converter.
> * ExternalTransport needs testing of changes similar to ExternalParser.
>
> DOCUMENTATION:
> * List of supported platforms/compilers is ancient. (PR#405279)
> * Document all of htsearch's mappings of input parameters to config attributes
>    to template variables. (Relates to PR#405278.)
>   Should we make sure these config attributes are all documented in
>   defaults.cc, even if they're only set by input parameters and never
>   in the config file?
> * Split attrs.html into categories for faster loading.
> * Turn defaults.cc into an XML file for generating documentation and
>   defaults.cc.
> * require.html is not updated to list new features and disk space
>    requirements of 3.2.x (e.g. regex matching, database compression.)
>    PRs# 405280 #405281.
> * TODO.html has not been updated for current TODO list and
>    completions.
>    I've tried.  Someone "official" please check and remove this -- Lachlan
> * Htfuzzy could use more documentation on what each fuzzy algorithm
>    does. PR#405714.
> * Document the list of all installed files and default
>    locations. PR#405715.
>
> OTHER ISSUES:
> * Can htsearch actually search while an index is being created?
> * The code needs a security audit, esp. htsearch. PR#405765.
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> htdig-dev mailing list
> htd...@li...
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485

OS X compatible version of libtool (was: Re: [htdig-dev] Autoconf question)

[Ted Stresen-R. <ted...@ma...>]

Hi,

I just heard from one of the developers of libtool, Peter O'Gorman, who 
has been doing a lot of fixing to get libtool to work properly with OS 
X. In short, his input was: use version 1.5 of libtool. Judging by the 
number of changes to libtool that deal specifically with darwin (OS X) 
related issues, I'd say this is a promising path to follow. Peter is 
trying to build htdig from the most recent snapshots with libtool 1.5 
and will get back to me with the results.

Peter also said that you can tell what version of libtool we are using 
by looking "at the configure line". Specifically, he says: "Your 
developer can easily check if he is using libtool-1.5 by looking at the 
configure line:
checking how to recognise dependent libraries... pass_all (on 1.5) and
file_magic Mach-O dynamically linked shared library ( anything other 
than 1.5)"

A quick search of the htdig source would indicate that we are using a 
pre-1.5 version of libtool.

I hope this helps.

Ted Stresen-Reuter

On Tuesday, May 20, 2003, at 05:29  AM, Lachlan Andrew wrote:

> Greetings Gabriele,
>
> I wasn't running any of the other tools.  I just thought that, being
> the last step in the process you recommended, running  autoreconf  by
> itself should give the same files back.  When I run it twice in a
> row, it gives the the same  configure  file each time, but that file
> is different from the one in CVS.  Does that mean you didn't run
> autoreconf  when you updated the files three months ago?
>
> Lachlan
>
> On Mon, 19 May 2003 21:44, Gabriele Bartolini wrote:
>> Il lun, 2003-05-19 alle 12:41, Lachlan Andrew ha scritto:
>>> Am I right in thinking that running
>>>     autoreconf
>>> twice in the last step should make no difference?  When I run
>>> autoreconf (from Autoconf 2.57) the  configure  file is very
>>> different from the one in CVS.
>>
>> However, are u sure you did not change the config.sub and
>> config.guess files? What about aclocal.m4?
>>
>> So ... try to run the other tools separately. Which versions are
>> you using of autoconf (2.57, right?), automake and libtool?
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> htdig-dev mailing list
> htd...@li...
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

Re: [htdig-dev] zlib check breaks Solaris

[Lachlan A. <lh...@us...>]

Greetings Gabriele,

Yes, someone had reported that before.  The new test should fix that.

Cheers,
Lachlan

On Mon, 26 May 2003 17:52, Gabriele Bartolini wrote:
> when I run configure with just the 'with-zlib' option
> (without specifying any exact dir), I assume the default dirs are
> scanned. On the other hand, I get a weird result: the configure
> script goes searching for libraries in a 'yes/include' and
> 'yes/lib' directory (which prevents the library to be linked by
> libtool).
>    Anyone over there had the same problem? If I specify
> --with-zlib=3D/usr the installation goes on.

Re: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525

[Lachlan A. <lh...@us...>]

On Mon, 26 May 2003 20:39, J.E.J. op den Brouw wrote:
> Subject: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525
> > CHECKLIST FOR 3.2.0b5:
> > * Must be able to
> > (a) make check  and
> > (b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
> >   on all systems listed as "supported".
> Are these the conditions for a "well-functional" version?

I took the liberty of adding this section to the STATUS file, mainly=20
to promote discussion...

They seem essentially minimal conditions for us to be able to say a=20
platform is "supported" rather than "supported, but..."  If HP-UX can=20
be got to run but fails one of those tests, we can say exactly that=20
in the documentation.  Do you agree?  If you can think of any other=20
necessary conditions, please suggest them!

> >   To be tested:
> HP-UX 10.20 w/ GCC 2.8.1

Excellent :)  If you'd like a hand with that select(2) problem, I'd be=20
happy to look through your  config.log  file (but no promises I'll be=20
of any use).

Cheers,
Lachlan

Re: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525

[J.E.J. op d. B. <ht...@op...>]

Hi,

----- Original Message ----- 
From: "Geoff Hutchison" <ghu...@us...>
To: <htd...@li...>
Sent: Sunday, May 25, 2003 9:14 AM
Subject: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030525


> STATUS of ht://Dig branch 3-2-x
>
> CHECKLIST FOR 3.2.0b5:
> * Add more items to checklist :-)
> * Must be able to
> (a) make check  and
> (b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
>   on all systems listed as "supported".

Are these the conditions for a "well-functional" version?

>   Systems tested so far:
>     - Mandrake 8.2, gcc 3.2 (lha, 21 May)
>     - FreeBSD 4.6, gcc 2.95.3 (lha, 23 May)
>     - Debian, Linux kernel 2.2.19, gcc 2.95.4 (lha, 23 May)
>   Partly tested:
>     - SunOS 5.8 = Solaris 2.8, gcc 3.1 (lha.  Makes check.  Big dig not
tested)
>     - SunOS 5.8 = Solaris 2.8, Sun cc with g++ 3.1 (lha. Compiles, fails
check)
>   To be tested:
>     - RedHat anyone?
>     - OS X anyone?

HP-UX 10.20 w/ GCC 2.8.1

> * Latin-encodings (Gilles)
> * Check bugs listed in bug-tracker...
> * Polish release docs (Geoff)

Re: [htdig-dev] zlib check breaks Solaris

[Gabriele B. <g.b...@co...>]

Ciao guys,

   hopefully I will finally have a bit of time for testing the new
3.2.0b code, especially after all the great work done by Lachlan.

   I tried to run on my Debian Woody the configuration script, but I
encountered a 'small' problem with the zlib directory specification.

   Usually, when I run configure with just the 'with-zlib' option
(without specifying any exact dir), I assume the default dirs are
scanned. On the other hand, I get a weird result: the configure script
goes searching for libraries in a 'yes/include' and 'yes/lib' directory
(which prevents the library to be linked by libtool).

   Anyone over there had the same problem? If I specify --with-zlib=/usr
the installation goes on.

Ciao
-Gabriele

Il dom, 2003-05-25 alle 16:18, Lachlan Andrew ha scritto:
> Greetings all,
> 
> The current check for zlib (which explicitly adds -I/usr/include or 
> -I/usr/local/include if --with-zlib=... isn't specified) breaks the 
> compile farm's Solaris setting.  (/usr/include/stdargs.h is not gcc 
> compatible.)
> 
> Does anyone know
> (a) Why the default paths are added explicitly
> (b) Why AC_CHECK_LIB(z, inflateEnd) was run twice?
> 
> Cheers,
> Lachlan

-- 
Gabriele Bartolini - Web Programmer
Comune di Prato - Prato - Tuscany - Italy
g.b...@co... | http://www.comune.prato.it
> find bin/laden -name osama -exec rm {} ;

[htdig-dev] Mirror of HtDig at http://htdig.host1.com.au

[Campbell M. <cam...@ci...>]

Hi,

This is just to let you know that we have set up
a mirror of http://htdig.org/ (main site) at 
http://htdig.host1.com.au. Could you please
add us to your mirror list.

Thanks and regards,

Campbell McLeay
Systems Administrator
Host 1 Webhosting Australia
-- 

[htdig-dev] zlib check breaks Solaris

[Lachlan A. <lh...@us...>]

Greetings all,

The current check for zlib (which explicitly adds -I/usr/include or=20
-I/usr/local/include if --with-zlib=3D... isn't specified) breaks the=20
compile farm's Solaris setting.  (/usr/include/stdargs.h is not gcc=20
compatible.)

Does anyone know
(a) Why the default paths are added explicitly
(b) Why AC_CHECK_LIB(z, inflateEnd) was run twice?

Cheers,
Lachlan