htdig-dev — Developer Discussion for the ht://Dig project

Re: [htdig-dev] Re: New patch [was: new release of htdig]

[Jim C. <li...@yg...>]

Hi - I am still having problems with this patch applied. The dig makes 
it much farther than it did last time around, but compression still 
appears to be an issue. I receive the following a few times toward the 
end of the dig.

WordDB: CDB___memp_cmpr_read: unable to uncompress page at pgno = 246532
WordDB: PANIC: Input/output error
WordDB: CDB___memp_cmpr_read: unable to uncompress page at pgno = 246532
WordDB: PANIC: Input/output error

Then at the end of the dig, I see the following.

WordDB: CDB___memp_cmpr_read: unable to uncompress page at pgno = 246532
WordDB: PANIC: Input/output error
WordDBCursor::Get(17) failed DB_RUNRECOVERY: Fatal error, run database 
recovery

The referenced page number is the same for every error message. For 
this test, the default value was used for wordlist_page_size. Both 
wordlist_compress and wordlist_compress_zlib were set to true, and 
compression_level was set to 8. I used a fresh copy of the CVS code (as 
of yesterday) with dbase.patch2 applied.

Jim

On Sunday, April 27, 2003, at 02:27 AM, Lachlan Andrew wrote:

> OK, here is another patch...
>
> The problem was that I was using the counts of clean and dirty cache
> pages, but they were not recorded correctly in the old BDB 2.x
> (although that didn't show up in my initial tests...).  This patch
> contains the fix, copied from BDB 3.3.11.
>
> Thanks again for your help :)
>
> Lachlan
>
> On Sun, 27 Apr 2003 12:54, Jim Cole wrote:
>> I am still running into fatal problems with OS X. I no longer get
>> the segfault, but instead see the output shown below.
> <dbase.patch2>

[htdig-dev] synonyms

[Jim C. <li...@yg...>]

Using the current CVS code, htfuzzy generates the following.

htfuzzy/synonyms: 1510 woolie
htfuzzy/synonyms: Rejected line with less than 2 words:

htfuzzy/synonyms: 1519 worshipping
htfuzzy/synonyms: Done.
htfuzzy: Done.

The 'Rejected' line appears to be due to a blank line at the end of the 
synonyms file. It doesn't seem to hurt anything, but it is bound to 
raise questions if left as is.

Jim

[htdig-dev] Current Status as of snapshot 3.2.0b4-20030504

[Geoff H. <ghu...@us...>]

STATUS of ht://Dig branch 3-2-x

RELEASES:
   3.2.0b5: Next release, First quarter 2003???
   3.2.0b4: "In progress" -- snapshots called "3.2.0b4" until prerelease.
   3.2.0b3: Released:  22 Feb 2001.
   3.2.0b2: Released:  11 Apr 2000.
   3.2.0b1: Released:   4 Feb 2000.

(Please note that everything added here should have a tracker PR# so
we can be sure they're fixed. Geoff is currently trying to add PR#s for
what's currently here.)

SHOWSTOPPERS:
* Mifluz database errors are a severe problem (PR#428295)
-- Does Neal's new zlib patch solve this for now?

KNOWN BUGS:
* Odd behavior with $(MODIFIED) and scores not working with
   wordlist_compress set but work fine without wordlist_compress.
   (the date is definitely stored correctly, even with compression on
    so this must be some sort of weird htsearch bug) PR#618737.
* META descriptions are somehow added to the database as FLAG_TITLE,
   not FLAG_DESCRIPTION. (PR#618738)
   Can anyone reproduce this?  I can't! -- Lachlan

PENDING PATCHES (available but need work):
* Additional support for Win32.
* Memory improvements to htmerge. (Backed out b/c htword API changed.)
* Mifluz merge.

NEEDED FEATURES:
* Quim's new htsearch/qtest query parser framework.
* File/Database locking. PR#405764.

TESTING:
* httools programs: 
  (htload a test file, check a few characteristics, htdump and compare)
* Tests for new config file parser
* Duplicate document detection while indexing
* Major revisions to ExternalParser.cc, including fork/exec instead of popen,
  argument handling for parser/converter, allowing binary output from an
  external converter.
* ExternalTransport needs testing of changes similar to ExternalParser.

DOCUMENTATION:
* List of supported platforms/compilers is ancient. (PR#405279)
* Document all of htsearch's mappings of input parameters to config attributes
   to template variables. (Relates to PR#405278.)
  Should we make sure these config attributes are all documented in
  defaults.cc, even if they're only set by input parameters and never
  in the config file?
* Split attrs.html into categories for faster loading.
* Turn defaults.cc into an XML file for generating documentation and 
  defaults.cc.
* require.html is not updated to list new features and disk space
   requirements of 3.2.x (e.g. regex matching, database compression.)
   PRs# 405280 #405281.
* TODO.html has not been updated for current TODO list and
   completions.
   I've tried.  Someone "official" please check and remove this -- Lachlan
* Htfuzzy could use more documentation on what each fuzzy algorithm
   does. PR#405714.
* Document the list of all installed files and default
   locations. PR#405715.

OTHER ISSUES:
* Can htsearch actually search while an index is being created?
* The code needs a security audit, esp. htsearch. PR#405765.

Re: [htdig-dev] Re: New patch - feedback

[Jim C. <li...@yg...>]

On Wednesday, April 30, 2003, at 07:03 AM, Ted Stresen-Reuter wrote:

> On Wednesday, April 30, 2003, at 06:26  AM, Lachlan Andrew wrote:
>
>>> Lachlan: was this the error I was supposed to get?
>>
>> Umm... No.  The problem that I was trying to fix was one that Jim
>> reported, so he is the expert.  However, I got the impression that he
>> got it to run, but then it crashed or aborted at some point.  I was
>> expecting my patch to abort at the same point, with a more useful
>> error message.
>
> Hmm... well, ok. Let me know if there is something specific you need 
> me to try. Unfortunately, my time is very limited so debugging is 
> almost out of the question, but running a test here and there is not a 
> problem.

The problem for which the patch was intended involved database issues 
that popped up while indexing some test data provided by Lachlan. Info 
on the specifics is mostly likely lurking in the archives somewhere. 
The original intent was to attempt to reproduce a separate problem, but 
running the test under OS X turned up this new problem. I will be 
testing again with the newest patch sometime later tonight.

Jim

Re: [htdig-dev] Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

You know, I saw this mentioned somewhere and thought that that's what 
the purpose of the lib was but just to get some clarification, does 
libhtdig incorporate all the programs of the htdig project (htdig, 
htsearch, htpurge, etc...) or is it just the indexer? My guess is that 
it is everything. Also, does it integrate with external parsers the 
same way the standard programs do (via the .conf files)?

Also, although a personal search engine would be fun to develop, it's 
probably just a "wow, cool" kind of project rather than something 
someone actually needs... so, before I head down any new project paths, 
I think I'll try and finish something I've been working on for a while 
now that the htdig project would benefit more greatly from: splitting 
attrs.html.

Thanks.

Ted Stresen-Reuter

On Thursday, May 1, 2003, at 05:15  PM, Neal Richter wrote:

>
>>
>> On another note, I went to a WebObjects user group meeting last night
>> and learned of a very intriguing future for Mac OS X. Imagine 
>> embedding
>> Safari (or just the html rendering engine) inside other applications:
>> http://developer.apple.com/darwin/projects/webcore/
>>
>> I am considering putting a GUI on htdig for use as a personal search
>> engine (to index and find documents on a user's workstation) rendering
>> the results with the Safari rendering engine (KHTML). That way, no web
>> server is needed. We'll see if that ever goes anywhere...
>
>   This is the guts of what libhtdig & libhtdigphp is supposed to
> accomplish.  Of course libhtdigphp is supposed to be used with a
> webserver, but it works just fine from the command line as well..
> creating a PHP file that executes the php-wrapper calls for the 
> libhtdig APIs.
>
> 'php -e xxx.php' will execute php files from the command-line and work
> great as long as the php code does not depend on environment variables 
> set
> by Apache (like REMOTE_ADDR).
>
>   libhtdig itself could be linked into/against another C/C++ 
> application
> and be called to conduct the search & retrieve the results for display 
> however
> you want.  This is essentially the reason why the HtDig developers 
> voted
> on moving the entire codebase to LGPL.. so that anyone could use this
> library for 'information retrieval' applications.
>
>   libhtdig also contains extenstions that allow you add individual
> documents to the index outside of a web-crawl.  Just fill the structure
> appropriately and it will be indexed and returned on appropriate 
> searches.
>
>   I have not yet committed the libhtdigphp directory, due to issues 
> about
> 'configure' being general enough.  I'll commit it today and see what
> people think of the build process.
>
> Thanks.
>
> Neal Richter
> Knowledgebase Developer
> RightNow Technologies, Inc.
> Customer Service for Every Web Site
> Office: 406-522-1485
>
>
>
"How wonderful the world would be without Saddam and without Bush!"

[htdig-dev] 3.1.6 patch for regex support in restrict and exclude

[Gabriele B. <g.b...@co...>]

Ciao guys, 

   I was asked to perform more operations on restriction and exclusions
at search time and without any regex support, my boss requests cannot
easily be accomplished. I foresay that I have never handled regex
library before ...

   I understand that regex support for exclusions and restrictions are
part of the 3.2 code, but I think that it should not be too hard to
write a patch for the still stable release (which we use in our
production servers). 

   I was wondering: 

- if some of you has already worked on this (but I was not able to find
any patch in the archive for this) 
- if you have some hints to give me 

   Also because this morning I gave it a quick try, but I could not
succeed. Here is the small patch I applied (in attachment), but it looks
like all the URLs are discarded and I don't get any results back. Any
hints?

   Thank you and ciao, 

Ciao 
-Gabriele 
    
-- 
Gabriele Bartolini - Web Programmer
Comune di Prato - Prato - Tuscany - Italy
g.b...@co... | http://www.comune.prato.it
> find bin/laden -name osama -exec rm {} ;

Re: [htdig-dev] Re: New patch - feedback

[Gabriele B. <bar...@in...>]

>   I have not yet committed the libhtdigphp directory, due to issues about
>'configure' being general enough.  I'll commit it today and see what
>people think of the build process.

Great! I am really interested in this, Neal!

-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Prato, Tuscany, Italy
bar...@in... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno

Re: [htdig-dev] Re: New patch - feedback

[Neal R. <ne...@ri...>]

>
> On another note, I went to a WebObjects user group meeting last night
> and learned of a very intriguing future for Mac OS X. Imagine embedding
> Safari (or just the html rendering engine) inside other applications:
> http://developer.apple.com/darwin/projects/webcore/
>
> I am considering putting a GUI on htdig for use as a personal search
> engine (to index and find documents on a user's workstation) rendering
> the results with the Safari rendering engine (KHTML). That way, no web
> server is needed. We'll see if that ever goes anywhere...

  This is the guts of what libhtdig & libhtdigphp is supposed to
accomplish.  Of course libhtdigphp is supposed to be used with a
webserver, but it works just fine from the command line as well..
creating a PHP file that executes the php-wrapper calls for the libhtdig APIs.

'php -e xxx.php' will execute php files from the command-line and work
great as long as the php code does not depend on environment variables set
by Apache (like REMOTE_ADDR).

  libhtdig itself could be linked into/against another C/C++ application
and be called to conduct the search & retrieve the results for display however
you want.  This is essentially the reason why the HtDig developers voted
on moving the entire codebase to LGPL.. so that anyone could use this
library for 'information retrieval' applications.

  libhtdig also contains extenstions that allow you add individual
documents to the index outside of a web-crawl.  Just fill the structure
appropriately and it will be indexed and returned on appropriate searches.

  I have not yet committed the libhtdigphp directory, due to issues about
'configure' being general enough.  I'll commit it today and see what
people think of the build process.

Thanks.

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485

Re: [htdig-dev] Re: libtool... (Was: New patch - feedback)

[Gabriele B. <bar...@in...>]

Ciao Lachlan!

>  2003-03-21  Peter O'Gorman  <pe...@po...>
>  * libtool.m4 (darwin): Check compiler is apple gcc, add
>-single_module support so that dyloading c++ shared libraries will
>work.

I hadn't checked libtool recently. But dynlib were exactly the problem I 
encountered with libtool. I hope this would fix.

>That sounds like just the patch we need!  If someone tells me the
>steps (autoconf? automake? ???), I'll download and run 1.5 on the
>weekend.

I guess you only have to launch ( I am not pretty sure about it though - 
give a look at the man page):

libtoolize --copy inside the htdig source directory.

I don't think you need to run autoconf or automake either. However, in 
general, here are the steps I usually follow with ht://Check and - every 
death of pope (as we say in Italy - that is to say very rarely :-D ) - with 
ht://Dig.

You can find them here: 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/htcheck/htcheck/README.developers 
from the 'Browse CVS' section of SF for ht://Check.

Step 1 ( I think it is not necessary now)
=======

         aclocal

  Created macro file aclocal.m4 with definitions for libtool.


Step 2
=======

         libtoolize --copy

  Copies libtool files.


Step 3
=======

         autoconf

  Create the file 'configure'


Step 4
=======

         automake --foreign --add-missing --copy -v -f

Step 5
======

         autoreconf

Ciao ciao
-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Prato, Tuscany, Italy
bar...@in... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno

Re: [htdig-dev] Re: libtool... (Was: New patch - feedback)

[Ted Stresen-R. <ted...@ma...>]

Bravo and me too!

Thanks for scouring around out there. This does look like the answer...

Ted Stresen-Reuter
http://www.tedmasterweb.com/

On Wednesday, April 30, 2003, at 04:45  PM, Lachlan Andrew wrote:

> Looking through the ChangeLog for libtool 1.5 (Released 14 April):
>
>  2003-03-21  Peter O'Gorman  <pe...@po...>
>  * libtool.m4 (darwin): Check compiler is apple gcc, add
> -single_module support so that dyloading c++ shared libraries will
> work.
>
> That sounds like just the patch we need!  If someone tells me the
> steps (autoconf? automake? ???), I'll download and run 1.5 on the
> weekend.

[htdig-dev] Re: libtool... (Was: New patch - feedback)

[Lachlan A. <lh...@us...>]

Greetings again,

Looking through the ChangeLog for libtool 1.5 (Released 14 April):

 2003-03-21  Peter O'Gorman  <pe...@po...>
 * libtool.m4 (darwin): Check compiler is apple gcc, add=20
-single_module support so that dyloading c++ shared libraries will=20
work.

That sounds like just the patch we need!  If someone tells me the=20
steps (autoconf? automake? ???), I'll download and run 1.5 on the=20
weekend.

Cheers,
Lachlan


On Wed, 30 Apr 2003 22:42, Lachlan Andrew wrote:

> Could someone who knows the drill try regenerating the configuration
> files with  libtool 1.4?

Re: [htdig-dev] libtool... (Was: New patch - feedback)

[Gabriele B. <g.b...@co...>]

Dear mate Lachlan,

   sorry I've not been that much of help right now.

> I've never got the hang of libtool et al.  Could someone who knows the 
> drill try regenerating the configuration files with  libtool 1.4  or 
> the patched  1.3.5?  (Better yet, could someone post step-by-step 
> instructions?)

I already generated everything with:

autoconf 2.57, automake 1.6.3 and libtool 1.4.3

because I had already gone through these MacOSX issues (even though I
don't have a MacOSX) in february. I got it to compile without shared
libraries, as - IMHO - it is a problem regarding libtool more than an
application problem.

I had a chance to retry it today on cf.sourceforge.net (Darwin
ucf-cf-ppc-macosx-2 6.5 Darwin Kernel Version 6.5) and I got it compiled
(even right now).

I ran:
../htdig/configure --prefix=/home/users/a/an/angusgb/htdig
--disable-shared --enable-static

Are you sure you didn't run 'make distclean' before? I don't know why,
but if you don't erase the directory contents before, it keeps on
generating stuff with shared libs.

I remember - not exactly about the versions (I should have noted
everything somewhere!) - there were some compatibility problems also
with config.guess and config.sub files between automake and autoconf
too, which prevented the configure script to automatically detect some
information specifically related to the MacOSX world.

Also I found that libtool should have been upgraded to 1.4.x and ... I
did it.

Here is the entry in the ChangeLog:

==================
Thu Jan 30 19:28:32 CET 2003 Gabriele Bartolini
<an...@us...>

        * configure.in: used AC_LIBOBJ instead of deprecated LTLIBOBJS's
workaround
        * ltconfig: removed as not needed anymore since libtool 1.4
        * db/configure.in: added AC_CONFIG_AUX_DIR(../) for letting
automake know to use the main ltmain.sh file
        * configure, aclocal.m4, Makefile.in, */Makefile.in,
config.guess, config.sub,
        install-sh, ltmain.sh, missing, mkinstalldirs: re-generated by
autotools: aclocal, autoconf 2.57, automake 1.6.3 and libtool 1.4.3
        * db/aclocal.m4, db/configure, db/mkinstalldirs: ditto
==================

But ... I am not the best person to ask for autotools stuff at all!
Everytime I have to touch them my legs start shaking. :-P

Please let me know ... (however I am pretty busy now, as next week I
have a kinda conference about ht://Check).

Ciao
-Gabriele
-- 
Gabriele Bartolini - Web Programmer
Comune di Prato - Prato - Tuscany - Italy
g.b...@co... | http://www.comune.prato.it
> find bin/laden -name osama -exec rm {} ;

Re: [htdig-dev] Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

Hi again,

On Wednesday, April 30, 2003, at 06:26  AM, Lachlan Andrew wrote:

>> Lachlan: was this the error I was supposed to get?
>
> Umm... No.  The problem that I was trying to fix was one that Jim
> reported, so he is the expert.  However, I got the impression that he
> got it to run, but then it crashed or aborted at some point.  I was
> expecting my patch to abort at the same point, with a more useful
> error message.

Hmm... well, ok. Let me know if there is something specific you need me 
to try. Unfortunately, my time is very limited so debugging is almost 
out of the question, but running a test here and there is not a problem.
>
> I've now got an account on the compile farm, and I'm trying to get it
> to link properly...

I suspect the version of libtool is the problem too. The reason I 
suspect this is that in conversations with the author of libjpeg, I 
found similar issues in his conf files as well. I was able to get the 
library to compile by manually editing the conf files, but that's not 
the way to do it. He pointed out that they were generated with a really 
old version of libtool and that a newer version would probably solve 
the problem.
>
> It's good working with you :)
>
Same here. I actually feel a little bad that I haven't been able to 
complete the work I was doing on splitting attrs.html but I just 
haven't had the time (a couple of uninterrupted days with nothing more 
urgent to work on) so this is one way I can lend a hand.

On another note, I went to a WebObjects user group meeting last night 
and learned of a very intriguing future for Mac OS X. Imagine embedding 
Safari (or just the html rendering engine) inside other applications: 
http://developer.apple.com/darwin/projects/webcore/

I am considering putting a GUI on htdig for use as a personal search 
engine (to index and find documents on a user's workstation) rendering 
the results with the Safari rendering engine (KHTML). That way, no web 
server is needed. We'll see if that ever goes anywhere...

Ted Stresen-Reuter

[htdig-dev] Re: libtool...

[Lachlan A. <lac...@ip...>]

Oops.  Apologies to Gabriele for forgetting that he had already used=20
libtool 1.4...  There goes another theory...

Cheers,
Lachlan

On Wed, 30 Apr 2003 22:42, Lachlan Andrew wrote:
> Greetings all,
>
> This article may also be of help:
>   http://fink.sourceforge.net/doc/porting/libtool.php
> It looks like  libtool 1.3  doesn't support MacOS...
>
> I've never got the hang of libtool et al.  Could someone who knows
> the drill try regenerating the configuration files with  libtool
> 1.4  or the patched  1.3.5?  (Better yet, could someone post
> step-by-step instructions?)
>
> Thanks!
> Lachlan
>
> On Tue, 29 Apr 2003 22:53, Ted Stresen-Reuter wrote:
> > this article might be of help for the Undefined symbols issue...
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> htdig-dev mailing list
> htd...@li...
> https://lists.sourceforge.net/lists/listinfo/htdig-dev

[htdig-dev] libtool... (Was: New patch - feedback)

[Lachlan A. <lh...@us...>]

Greetings all,

This article may also be of help:
  http://fink.sourceforge.net/doc/porting/libtool.php
It looks like  libtool 1.3  doesn't support MacOS...

I've never got the hang of libtool et al.  Could someone who knows the=20
drill try regenerating the configuration files with  libtool 1.4  or=20
the patched  1.3.5?  (Better yet, could someone post step-by-step=20
instructions?)

Thanks!
Lachlan

On Tue, 29 Apr 2003 22:53, Ted Stresen-Reuter wrote:
> this article might be of help for the Undefined symbols issue...

[htdig-dev] Re: New patch - feedback

[Lachlan A. <lh...@us...>]

On Tue, 29 Apr 2003 22:53, Ted Stresen-Reuter wrote:
> this article might be of help for the Undefined symbols issue...
>
> http://developer.apple.com/techpubs/macosx/ReleaseNotes/
> CompilerTools.html

Yes, it does look relevant.

> Lachlan: was this the error I was supposed to get?

Umm... No.  The problem that I was trying to fix was one that Jim=20
reported, so he is the expert.  However, I got the impression that he=20
got it to run, but then it crashed or aborted at some point.  I was=20
expecting my patch to abort at the same point, with a more useful=20
error message.

I've now got an account on the compile farm, and I'm trying to get it=20
to link properly...

It's good working with you :)
Lachlan

Re: [htdig-dev] Re: Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

I don't know much about building software but this article might be of  
help for the Undefined symbols issue...

http://developer.apple.com/techpubs/macosx/ReleaseNotes/ 
CompilerTools.html

Lachlan: was this the error I was supposed to get?

Ted Stresen-Reuter

On Monday, April 28, 2003, at 10:37  PM, Jim Cole wrote:

> Hi - This problem has been around for a while under OS X. I tried  
> tracking down the cause a while back and didn't find anything obvious.  
> I will be taking another look as soon as I get a chance.
>
> Jim
>
> On Monday, April 28, 2003, at 09:04 PM, Ted Stresen-Reuter wrote:
>
>> Well, I tried the following ./configure followed by make test and it  
>> terminated as shown below:
>>
>> ./configure \
>> --enable-tests \
>> --prefix=/Library/htdig-3.2 \
>> --with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
>> --with-image-dir=/Library/WebServer/Documents/htdig/images \
>> --with-search-dir=/Library/WebServer/Documents/htdig/search \
>> --enable-static \
>> --disable-shared
>>
>> [... lots os stuff ...]
>> ld: Undefined symbols:
>> WordType::instance
>> make[2]: *** [testnet] Error 1
>> make[1]: *** [check-am] Error 2
>> make: *** [check-recursive] Error 1
>>
>> It didn't crash, that's for sure, but it does look like the make test  
>> failed. I didn't try make install and actually running it yet,  
>> thought I'd wait to hear back from the list.
>>
>> Ted
>>
>> On Monday, April 28, 2003, at 07:28  PM, Lachlan Andrew wrote:
>>
>>> Thanks for that, Ted. Yes, there is a small test suite.  You need to
>>>   ./configure --enable-tests  --whatever-you-usually-use
>>>   make check
>>> The bug was that this used to crash on OS X.  Could you confirm that
>>> your system crashes without the patch, like Jim's?  It would be great
>>> if you could also just test it on whatever web site you happen to
>>> have handy...
>>>
>>> I'm just testing a revised patch (attached), which reduces the
>>> performance penalty on other systems.  On OS X, I expect it to crash
>>> with a message asking you to add a parameter to the config file.  If
>>> you make that change, then it should work OK.  For the test suite,
>>> the config files are .../test/conf/htdig.{conf,conf2,conf3}.  Please
>>> let me know how it behaves.
>>>
>>> Thanks again!
>>> Lachlan
>>>
>>>
>>> On Tue, 29 Apr 2003 00:52, Ted Stresen-Reuter wrote:
>>>> worked like a charm... everything seems to work perfectly... do we
>>>> have a test suite? is there a specific bug we are looking to squash
>>>> with this patch? if so, how can i reproduce it?
>>>
>>> <dbase.patch3>
>> "How wonderful the world would be without Saddam and without Bush!"
>>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> htdig-dev mailing list
> htd...@li...
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>
>
"How wonderful the world would be without Saddam and without Bush!"

Re: [htdig-dev] Re: Re: New patch - feedback

[Jim C. <li...@yg...>]

Hi - This problem has been around for a while under OS X. I tried 
tracking down the cause a while back and didn't find anything obvious. 
I will be taking another look as soon as I get a chance.

Jim

On Monday, April 28, 2003, at 09:04 PM, Ted Stresen-Reuter wrote:

> Well, I tried the following ./configure followed by make test and it 
> terminated as shown below:
>
> ./configure \
> --enable-tests \
> --prefix=/Library/htdig-3.2 \
> --with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
> --with-image-dir=/Library/WebServer/Documents/htdig/images \
> --with-search-dir=/Library/WebServer/Documents/htdig/search \
> --enable-static \
> --disable-shared
>
> [... lots os stuff ...]
> ld: Undefined symbols:
> WordType::instance
> make[2]: *** [testnet] Error 1
> make[1]: *** [check-am] Error 2
> make: *** [check-recursive] Error 1
>
> It didn't crash, that's for sure, but it does look like the make test 
> failed. I didn't try make install and actually running it yet, thought 
> I'd wait to hear back from the list.
>
> Ted
>
> On Monday, April 28, 2003, at 07:28  PM, Lachlan Andrew wrote:
>
>> Thanks for that, Ted. Yes, there is a small test suite.  You need to
>>   ./configure --enable-tests  --whatever-you-usually-use
>>   make check
>> The bug was that this used to crash on OS X.  Could you confirm that
>> your system crashes without the patch, like Jim's?  It would be great
>> if you could also just test it on whatever web site you happen to
>> have handy...
>>
>> I'm just testing a revised patch (attached), which reduces the
>> performance penalty on other systems.  On OS X, I expect it to crash
>> with a message asking you to add a parameter to the config file.  If
>> you make that change, then it should work OK.  For the test suite,
>> the config files are .../test/conf/htdig.{conf,conf2,conf3}.  Please
>> let me know how it behaves.
>>
>> Thanks again!
>> Lachlan
>>
>>
>> On Tue, 29 Apr 2003 00:52, Ted Stresen-Reuter wrote:
>>> worked like a charm... everything seems to work perfectly... do we
>>> have a test suite? is there a specific bug we are looking to squash
>>> with this patch? if so, how can i reproduce it?
>>
>> <dbase.patch3>
> "How wonderful the world would be without Saddam and without Bush!"
>

Re: [htdig-dev] Re: Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

Well, I tried the following ./configure followed by make test and it 
terminated as shown below:

./configure \
--enable-tests \
--prefix=/Library/htdig-3.2 \
--with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
--with-image-dir=/Library/WebServer/Documents/htdig/images \
--with-search-dir=/Library/WebServer/Documents/htdig/search \
--enable-static \
--disable-shared

[... lots os stuff ...]
ld: Undefined symbols:
WordType::instance
make[2]: *** [testnet] Error 1
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1

It didn't crash, that's for sure, but it does look like the make test 
failed. I didn't try make install and actually running it yet, thought 
I'd wait to hear back from the list.

Ted

On Monday, April 28, 2003, at 07:28  PM, Lachlan Andrew wrote:

> Thanks for that, Ted. Yes, there is a small test suite.  You need to
>   ./configure --enable-tests  --whatever-you-usually-use
>   make check
> The bug was that this used to crash on OS X.  Could you confirm that
> your system crashes without the patch, like Jim's?  It would be great
> if you could also just test it on whatever web site you happen to
> have handy...
>
> I'm just testing a revised patch (attached), which reduces the
> performance penalty on other systems.  On OS X, I expect it to crash
> with a message asking you to add a parameter to the config file.  If
> you make that change, then it should work OK.  For the test suite,
> the config files are .../test/conf/htdig.{conf,conf2,conf3}.  Please
> let me know how it behaves.
>
> Thanks again!
> Lachlan
>
>
> On Tue, 29 Apr 2003 00:52, Ted Stresen-Reuter wrote:
>> worked like a charm... everything seems to work perfectly... do we
>> have a test suite? is there a specific bug we are looking to squash
>> with this patch? if so, how can i reproduce it?
>
> <dbase.patch3>
"How wonderful the world would be without Saddam and without Bush!"

[htdig-dev] Re: Re: New patch - feedback

[Lachlan A. <lh...@us...>]

Thanks for that, Ted. Yes, there is a small test suite.  You need to
  ./configure --enable-tests  --whatever-you-usually-use
  make check
The bug was that this used to crash on OS X.  Could you confirm that=20
your system crashes without the patch, like Jim's?  It would be great=20
if you could also just test it on whatever web site you happen to=20
have handy...

I'm just testing a revised patch (attached), which reduces the=20
performance penalty on other systems.  On OS X, I expect it to crash=20
with a message asking you to add a parameter to the config file.  If=20
you make that change, then it should work OK.  For the test suite,=20
the config files are .../test/conf/htdig.{conf,conf2,conf3}.  Please=20
let me know how it behaves.

Thanks again!
Lachlan


On Tue, 29 Apr 2003 00:52, Ted Stresen-Reuter wrote:
> worked like a charm... everything seems to work perfectly... do we
> have a test suite? is there a specific bug we are looking to squash
> with this patch? if so, how can i reproduce it?

Re: [htdig-dev] Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

worked like a charm... everything seems to work perfectly... do we have 
a test suite? is there a specific bug we are looking to squash with 
this patch? if so, how can i reproduce it?

Ted s-r

On Sunday, April 27, 2003, at 11:31  PM, Jim Cole wrote:

> Hi - You need to perform a static build. There is still an unresolved 
> issue involving dynamic builds of ht://Dig under OS X. For some reason 
> parts of the standard C++ library are not picked up when you perform a 
> dynamic build. I think the newer version of libtool used with the 3.2 
> branch is the current suspect. Try passing the following to configure.
>
>   --enable-static --disable-shared
>
> If I recall correctly you don't actually need to provide both, but if 
> you want to be thorough... ;)
>
> Jim
>
> On Sunday, April 27, 2003, at 09:46 PM, Ted Stresen-Reuter wrote:
>
>> patch, configure, make, make install all appeared to go fine, but 
>> then I got the following when trying to run htdig:
>>
>> % sudo bin/htdig -v /Library/Htdig-3.2.old/conf/htdig.conf
>> dyld: bin/htdig Undefined symbols:
>> __ZSt4cout
>> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
>> __ZTTSt14basic_ifstreamIcSt11char_traitsIcEE
>> __ZTVSt14basic_ifstreamIcSt11char_traitsIcEE
>> __ZTVSt9basic_iosIcSt11char_traitsIcEE
>> ___cxa_pure_virtual
>> __ZSt4cout
>> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
>> ___cxa_pure_virtual
>> __ZSt3cin
>> __ZSt4cerr
>> __ZSt4cout
>> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
>> Trace/BPT trap
>> %
>>
>> I used the following to configure the 4/27 snapshot:
>> ./configure \
>> --prefix=/Library/htdig-3.2 \
>> --with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
>> --with-image-dir=/Library/WebServer/Documents/htdig/images \
>> --with-search-dir=/Library/WebServer/Documents/htdig/search
>>
>> Am I missing something?
>
>
"How wonderful the world would be without Saddam and without Bush!"

Re: [htdig-dev] Re: New patch - feedback

[Jim C. <li...@yg...>]

Hi - You need to perform a static build. There is still an unresolved 
issue involving dynamic builds of ht://Dig under OS X. For some reason 
parts of the standard C++ library are not picked up when you perform a 
dynamic build. I think the newer version of libtool used with the 3.2 
branch is the current suspect. Try passing the following to configure.

   --enable-static --disable-shared

If I recall correctly you don't actually need to provide both, but if 
you want to be thorough... ;)

Jim

On Sunday, April 27, 2003, at 09:46 PM, Ted Stresen-Reuter wrote:

> patch, configure, make, make install all appeared to go fine, but then 
> I got the following when trying to run htdig:
>
> % sudo bin/htdig -v /Library/Htdig-3.2.old/conf/htdig.conf
> dyld: bin/htdig Undefined symbols:
> __ZSt4cout
> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
> __ZTTSt14basic_ifstreamIcSt11char_traitsIcEE
> __ZTVSt14basic_ifstreamIcSt11char_traitsIcEE
> __ZTVSt9basic_iosIcSt11char_traitsIcEE
> ___cxa_pure_virtual
> __ZSt4cout
> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
> ___cxa_pure_virtual
> __ZSt3cin
> __ZSt4cerr
> __ZSt4cout
> __ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
> Trace/BPT trap
> %
>
> I used the following to configure the 4/27 snapshot:
> ./configure \
> --prefix=/Library/htdig-3.2 \
> --with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
> --with-image-dir=/Library/WebServer/Documents/htdig/images \
> --with-search-dir=/Library/WebServer/Documents/htdig/search
>
> Am I missing something?

[htdig-dev] Re: New patch - feedback

[Ted Stresen-R. <ted...@ma...>]

patch, configure, make, make install all appeared to go fine, but then 
I got the following when trying to run htdig:

% sudo bin/htdig -v /Library/Htdig-3.2.old/conf/htdig.conf
dyld: bin/htdig Undefined symbols:
__ZSt4cout
__ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
__ZTTSt14basic_ifstreamIcSt11char_traitsIcEE
__ZTVSt14basic_ifstreamIcSt11char_traitsIcEE
__ZTVSt9basic_iosIcSt11char_traitsIcEE
___cxa_pure_virtual
__ZSt4cout
__ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
___cxa_pure_virtual
__ZSt3cin
__ZSt4cerr
__ZSt4cout
__ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_
Trace/BPT trap
%

I used the following to configure the 4/27 snapshot:
./configure \
--prefix=/Library/htdig-3.2 \
--with-cgi-bin-dir=/Library/WebServer/Documents/CGI-Executables \
--with-image-dir=/Library/WebServer/Documents/htdig/images \
--with-search-dir=/Library/WebServer/Documents/htdig/search

Am I missing something?

Ted Stresen-Reuter
(finally had a chance to get back to helping out here...)

On Sunday, April 27, 2003, at 03:27  AM, Lachlan Andrew wrote:

> OK, here is another patch...
>
> The problem was that I was using the counts of clean and dirty cache
> pages, but they were not recorded correctly in the old BDB 2.x
> (although that didn't show up in my initial tests...).  This patch
> contains the fix, copied from BDB 3.3.11.
>
> Thanks again for your help :)
>
> Lachlan
>
> On Sun, 27 Apr 2003 12:54, Jim Cole wrote:
>> I am still running into fatal problems with OS X. I no longer get
>> the segfault, but instead see the output shown below.
> <dbase.patch2>
"How wonderful the world would be without Saddam and without Bush!"

[htdig-dev] Re: b4 snapshot compile error

[Lachlan A. <lh...@us...>]

On Mon, 28 Apr 2003 03:42, Jim Gifford wrote:
> ./configure  \
> --with-zlib

That line is your problem, as I suggested last time.

The default behaviour is to use  zlib.  If the system can't find it,=20
then give  --with-zlib=3D/your/path/to/zlib  (without the final  /lib).

Thanks for pointing out that  ./configure  doesn't give diagnostics in=20
this case.  (Can anyone out there fix the  autoconf  files to make =20
--with-zlib  default to the appropriate directories, rather than=20
default to 'yes'?)

Regards,
Lachlan

> ----- Original Message -----
> From: "Lachlan Andrew" <lh...@us...>
>
> My current guess is that you specified  --something_or_other=3Dyes=20
> when it wanted  --something_or_other=3D/directory/ (e.g.
> --with-zlib=3Dyes).

[htdig-dev] Re: New patch [was: new release of htdig]

[Lachlan A. <lh...@us...>]

OK, here is another patch...

The problem was that I was using the counts of clean and dirty cache=20
pages, but they were not recorded correctly in the old BDB 2.x=20
(although that didn't show up in my initial tests...).  This patch=20
contains the fix, copied from BDB 3.3.11.

Thanks again for your help :)

Lachlan

On Sun, 27 Apr 2003 12:54, Jim Cole wrote:
> I am still running into fatal problems with OS X. I no longer get
> the segfault, but instead see the output shown below.