htdig-dev — Developer Discussion for the ht://Dig project

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Jesse op d. B. <ht...@op...>]

Lachlan Andrew wrote:

>Greetings all,
>
>Another thought:  When  make check  fails, try typing
>
>cd test
>g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o
>   -L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a
>   ../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a
>   ../db/.libs/libhtdb.a ../htlib/.libs/libht.a
>   ../htword/.libs/libhtword.a -lz
>make check
>
>Since you are using static libraries, the order of the libraries 
>matters.  The above  g++  command includes  libhtword.a  (in which 
>WordType::instance  should be defined) again at the end, just to be 
>sure.
>
>Out of interest, can you compile it using shared libraries?
>  
>
Still doesn't work.. Same error.
Shared libs won't work on UX. Not for ages......

--Jesse

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Jesse op d. B. <ht...@op...>]

Hi,

Lachlan Andrew wrote:

>Greetings Jesse,
>
>It's very good to hear that it compiles and runs!
>
>The  make check  problem is a mystery...
>Try
>   cd htword ; make clean ; cd .. ; make check
>If that doesn't work, could you please mail the results of
>    nm */.libs/*.a | grep WordType8instance
>and
>    nm htword/.libs/libhtword.a | grep WordType8instance
>?  Thanks.
>
>Lachlan
>
>  
>
Did: 

       cd htword ; make clean ; cd .. ; make check

Still compile error: 

g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o  -L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a ../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a ../db/.libs/libhtdb.a ../htlib/.libs/libht.a -lz
collect2: ld returned 1 exit status
/usr/ccs/bin/ld: Unsatisfied symbols:
   WordType::instance  (data)
make[2]: *** [testnet] Error 1
make[2]: Leaving directory `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test'
make: *** [check-recursive] Error 1
[msql@chaos htdig-3.2.0b4-20030914]$

The command:

   nm */.libs/*.a | grep WordType8instance

comes back with no output at all

using 

   nm */.libs/*.a | grep WordType

the output is:


[msql@chaos htdig-3.2.0b4-20030914]$ nm */.libs/*.a | grep WordType
Symbols from htlib/.libs/libht.a[HtWordType.o]:
_8WordType$instance |          |undef |data   |
Initialize__8WordTypeRC13Configuration|          |undef |code   |
_8WordType$instance |          |undef |data   |
GetWordType__C8WordList|         0|extern|entry  |$CODE$
Normalize__C8WordTypeR6String|          |undef |code   |
_$_8WordType        |          |undef |code   |
__8WordTypeRC13Configuration|          |undef |code   |
Symbols from htword/.libs/libhtword.a[WordType.o]:
Initialize__8WordTypeRC13Configuration|         0|extern|entry  |$CODE$
Instance__8WordType |         0|extern|entry  |$CODE$
IsChar__C8WordTypei |         0|extern|entry  |$CODE$
IsControl__C8WordTypei|         0|extern|entry  |$CODE$
IsDigit__C8WordTypei|         0|extern|entry  |$CODE$
IsStrictChar__C8WordTypei|         0|extern|entry  |$CODE$
NormalizeStatus__8WordTypei|         0|extern|entry  |$CODE$
Normalize__C8WordTypeR6String|         0|extern|entry  |$CODE$
StripPunctuation__C8WordTypeR6String|         0|extern|entry  |$CODE$
WordToken__C8WordTypeG6StringRi|         0|extern|entry  |$CODE$
_$_8WordType        |         0|extern|entry  |$CODE$
_8WordType$instance |1073741828|extern|data   |$DATA$
__8WordTypeRC13Configuration|         0|extern|entry  |$CODE$
_vt$8WordType       |1073741832|extern|data   |$DATA$
[msql@chaos htdig-3.2.0b4-20030914]$ 


Using:   nm */.libs/*.a | grep WordType\$instance


[msql@chaos htdig-3.2.0b4-20030914]$ nm */.libs/*.a | grep WordType\$instance
_8WordType$instance |          |undef |data   |
_8WordType$instance |          |undef |data   |
_8WordType$instance |1073741828|extern|data   |$DATA$
[msql@chaos htdig-3.2.0b4-20030914]$

Hope this helps

--Jesse

[htdig-dev] Re: Still having compile problems HP-UX 10.20

[Lachlan A. <lh...@us...>]

Greetings all,

On Sun, 21 Sep 2003 05:04, Ted Stresen-Reuter wrote:
> and here's the output from "nm */.libs/*.a | grep WordType8instance"
>
>           U __ZN8WordType8instanceE
>           U __ZN8WordType8instanceE
> 00000b60 s __GLOBAL__D__ZN8WordType8instanceE
> 00000b50 s __GLOBAL__I__ZN8WordType8instanceE
> 00000958 D __ZN8WordType8instanceE

That last line is the one that says  WordType::instance  is defined. =20
Jesse, do you get that on HP-UX?

Another thought:  When  make check  fails, try typing

cd test
g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o
   -L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a
   ../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a
   ../db/.libs/libhtdb.a ../htlib/.libs/libht.a
   ../htword/.libs/libhtword.a -lz
make check

Since you are using static libraries, the order of the libraries=20
matters.  The above  g++  command includes  libhtword.a  (in which=20
WordType::instance  should be defined) again at the end, just to be=20
sure.

Out of interest, can you compile it using shared libraries?

Cheers,
Lachlan

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Ted Stresen-R. <ted...@ma...>]

You are correct!

creating url
make MAKE="make" check-TESTS
PASS: t_wordkey
PASS: t_wordlist
PASS: t_wordskip
PASS: t_wordbitstream
PASS: t_search
PASS: t_htdb
PASS: t_rdonly
PASS: t_trunc
PASS: t_url
PASS: t_htdig
PASS: t_htsearch
PASS: t_htmerge
PASS: t_htnet
PASS: t_htdig_local
===================
All 14 tests passed
===================

Should the permissions thing be logged as a bug or is this a problem 
with my personal configuration of CVS?

Ted Stresen-Reuter

On Sunday, September 21, 2003, at 03:24  AM, Jim Cole wrote:

> On Saturday, September 20, 2003, at 01:04 PM, Ted Stresen-Reuter wrote:
>
>> Following the same instructions on Mac OS X produced the following 
>> output. Not sure if this is an error or not, though, because I don't 
>> know what all the tests are doing...
>>
>> creating url
>> make MAKE="make" check-TESTS
>> PASS: t_wordkey
>> PASS: t_wordlist
>> PASS: t_wordskip
>> PASS: t_wordbitstream
>> PASS: t_search
>> PASS: t_htdb
>> PASS: t_rdonly
>> PASS: t_trunc
>> ../test/test_prepare: /Users/tedsr/htdig/test/./t_url: Permission 
>> denied
>> ../test/test_prepare: exec: /Users/tedsr/htdig/test/./t_url: cannot 
>> execute: Undefined error: 0
>> FAIL: t_url
>
> This is due to the fact that the execute permissions on the t_url 
> script in the test directory are not being maintained. If you change 
> the permissions on that file (e.g. chmod 754 t_url) all tests 
> currently pass under OS X. I think something needs to be tweaked in 
> CVS to correct this problem.
>
> Jim
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> ht://Dig Developer mailing list:
> htd...@li...
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Jim C. <li...@yg...>]

On Saturday, September 20, 2003, at 01:04 PM, Ted Stresen-Reuter wrote:

> Following the same instructions on Mac OS X produced the following 
> output. Not sure if this is an error or not, though, because I don't 
> know what all the tests are doing...
>
> creating url
> make MAKE="make" check-TESTS
> PASS: t_wordkey
> PASS: t_wordlist
> PASS: t_wordskip
> PASS: t_wordbitstream
> PASS: t_search
> PASS: t_htdb
> PASS: t_rdonly
> PASS: t_trunc
> ../test/test_prepare: /Users/tedsr/htdig/test/./t_url: Permission 
> denied
> ../test/test_prepare: exec: /Users/tedsr/htdig/test/./t_url: cannot 
> execute: Undefined error: 0
> FAIL: t_url

This is due to the fact that the execute permissions on the t_url 
script in the test directory are not being maintained. If you change 
the permissions on that file (e.g. chmod 754 t_url) all tests currently 
pass under OS X. I think something needs to be tweaked in CVS to 
correct this problem.

Jim

[htdig-dev] Current Status as of snapshot 3.2.0b4-20030921

[Geoff H. <ghu...@us...>]

STATUS of ht://Dig branch 3-2-x

CHECKLIST FOR 3.2.0b5:
* Apply memory leak patches (Neal)
* Check bugs listed in bug-tracker...
* Polish release docs (Geoff)
* Must be able to
	(a) make check  and
	(b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
  on all systems listed as "supported".
  Systems tested so far:
    - Mandrake 8.2, gcc 3.2 (lha, 21 May)
    - FreeBSD 4.6, gcc 2.95.3 (lha, 23 May)
    - Debian, Linux kernel 2.2.19, gcc 2.95.4 (lha, 23 May)
    - SunOS 5.8 = Solaris 2.8, gcc 3.1 (lha, 25 May)
    - SunOS 5.8 = Solaris 2.8, Sun cc with g++ 3.1 (lha, 29 May)
    - OS X (Jim, 30 May)
  Partly tested:
    - RedHat 8 (Jim, 1 June.  make check requires tweaking for apache)
    - SunOS 5.8 = Solaris 2.8, gcc 2.95.2
	    (lha. Makes check minus apache, Digs small htdig.org. 27 May)
    - SunOS 5.8 = Solaris 2.8, Sun cc with g++ 2.95.2
	    (lha. Makes check minus apache, Digs small htdig.org. 2 June)
    - RedHat 7.3
	    (lha. Makes check minus apache. Digs small htdig.org. 25 May)
    - Alpha Debian
    	    (lha. Makes check minus apache. Digs small htdig.org. 25 May)
  To be tested:
    - HP-UX 10.20, gcc 2.8.1 (Jesse)
    - RedHat, other versions anyone?
    - RedHat AdvanceServer Itanium II (David Bannon)
  Known to have problems:
    - SGI/Irix 6.5.3 using SGI compilers
      <http://www.geocrawler.com/mail/msg.php3?msg_id=8025827&list=8825>





RELEASES:
   3.2.0b5: Next release, July 2003
   3.2.0b4: "In progress" -- snapshots called "3.2.0b4" until prerelease.
   3.2.0b3: Released:  22 Feb 2001.
   3.2.0b2: Released:  11 Apr 2000.
   3.2.0b1: Released:   4 Feb 2000.

(Please note that everything added here should have a tracker PR# so
we can be sure they're fixed. Geoff is currently trying to add PR#s for
what's currently here.)

SHOWSTOPPERS:
* Mifluz database errors are a severe problem (PR#428295)
-- Does Neal's new zlib patch solve this for now?

KNOWN BUGS:
* Odd behavior with $(MODIFIED) and scores not working with
   wordlist_compress set but work fine without wordlist_compress.
   (the date is definitely stored correctly, even with compression on
    so this must be some sort of weird htsearch bug) PR#618737.
* META descriptions are somehow added to the database as FLAG_TITLE,
   not FLAG_DESCRIPTION. (PR#618738)
   Can anyone reproduce this?  I can't! -- Lachlan
   Me either. Let's remove the PR. -Geoff

PENDING PATCHES (available but need work):
* Additional support for Win32. (Neal)
* Memory improvements to htmerge. (Backed out b/c htword API changed.)
* Mifluz merge.

NEEDED FEATURES:
* Quim's new htsearch/qtest query parser framework.
* File/Database locking. PR#405764.

TESTING:
* httools programs: 
  (htload a test file, check a few characteristics, htdump and compare)
* Tests for new config file parser
* Duplicate document detection while indexing
* Major revisions to ExternalParser.cc, including fork/exec instead of popen,
  argument handling for parser/converter, allowing binary output from an
  external converter.
* ExternalTransport needs testing of changes similar to ExternalParser.

DOCUMENTATION:
* List of supported platforms/compilers is ancient. (PR#405279)
* Document all of htsearch's mappings of input parameters to config attributes
   to template variables. (Relates to PR#405278.)
  Should we make sure these config attributes are all documented in
  defaults.cc, even if they're only set by input parameters and never
  in the config file?
* Split attrs.html into categories for faster loading.
* Turn defaults.cc into an XML file for generating documentation and 
  defaults.cc.
* require.html is not updated to list new features and disk space
   requirements of 3.2.x (e.g. regex matching, database compression.)
   PRs# 405280 #405281.
* TODO.html has not been updated for current TODO list and
   completions.
   I've tried.  Someone "official" please check and remove this -- Lachlan
* Htfuzzy could use more documentation on what each fuzzy algorithm
   does. PR#405714.
* Document the list of all installed files and default
   locations. PR#405715.

OTHER ISSUES:
* Can htsearch actually search while an index is being created?
* The code needs a security audit, esp. htsearch. PR#405765.

Re: [htdig-dev] Re: Is this a bug?

[Neal R. <ne...@ri...>]

I've got a fix for it.. a couple lines of code in the section that builds
the linked list of search results...

On Sat, 20 Sep 2003, Lachlan Andrew wrote:

> Yes, that does sound like a bug!  Well spotted, Neal!
>
> However, if reindexing obsoletes *removed* pages correctly, then I'd
> vote not to delay the release to fix the behaviour for pages which
> still exist but "shouldn't" be found.  Perhaps it would be best to
> file a bug report and fix it in 3.2.1...
>
> Opinions?
>
> Lachlan
>
> On Wed, 17 Sep 2003 08:07, Neal Richter wrote:
> > I'm using a fairly recent cvs snapshot...
> >
> >
> > 1) index a website
> >
> > 2) use search terms to return a given page.
> >
> > 3) Take links to that page out of the website
> >
> > 4) reindex site (don't use htdig -i)
> >
> > 5) repeat #2
> >
> > I would think that this page should be 'obsoleted' and not returned
> > after the reindex..
> >
> > htpurge does not touch these document since they haven't been
> > marked as obsolete...
>
> --
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> ht://Dig Developer mailing list:
> htd...@li...
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Ted Stresen-R. <ted...@ma...>]

Following the same instructions on Mac OS X produced the following 
output. Not sure if this is an error or not, though, because I don't 
know what all the tests are doing...

creating url
make MAKE="make" check-TESTS
PASS: t_wordkey
PASS: t_wordlist
PASS: t_wordskip
PASS: t_wordbitstream
PASS: t_search
PASS: t_htdb
PASS: t_rdonly
PASS: t_trunc
../test/test_prepare: /Users/tedsr/htdig/test/./t_url: Permission denied
../test/test_prepare: exec: /Users/tedsr/htdig/test/./t_url: cannot 
execute: Undefined error: 0
FAIL: t_url
PASS: t_htdig
PASS: t_htsearch
PASS: t_htmerge
PASS: t_htnet
PASS: t_htdig_local
====================
1 of 14 tests failed
====================
make[2]: *** [check-TESTS] Error 1
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1

and here's the output from "nm */.libs/*.a | grep WordType8instance"

          U __ZN8WordType8instanceE
          U __ZN8WordType8instanceE
00000b60 s __GLOBAL__D__ZN8WordType8instanceE
00000b50 s __GLOBAL__I__ZN8WordType8instanceE
00000958 D __ZN8WordType8instanceE

That's all for now...

Ted Stresen-Reuter

On Saturday, September 20, 2003, at 08:51  AM, Lachlan Andrew wrote:

> Greetings Jesse,
>
> It's very good to hear that it compiles and runs!
>
> The  make check  problem is a mystery...
> Try
>    cd htword ; make clean ; cd .. ; make check
> If that doesn't work, could you please mail the results of
>     nm */.libs/*.a | grep WordType8instance
> and
>     nm htword/.libs/libhtword.a | grep WordType8instance
> ?  Thanks.
>
> Lachlan
>
> On Thu, 18 Sep 2003 23:54, Jesse op den Brouw wrote:
>> Patch applied, line 156 commented out in the .h file:
>> configure runs fine,
>> make runs fine
>> make check breaks:
>>
>> g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o
>> -L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a
>> ../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a
>> ../db/.libs/libhtdb.a ../htlib/.libs/libht.a -lz
>> collect2: ld returned 1 exit status
>> /usr/ccs/bin/ld: Unsatisfied symbols:
>>    WordType::instance  (data)
>> make[2]: *** [testnet] Error 1
>> make[2]: Leaving directory
>> `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test
>> ' make[1]: *** [check-am] Error 2
>> make[1]: Leaving directory
>> `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test
>> ' make: *** [check-recursive] Error 1
>> [msql@chaos htdig-3.2.0b4-20030914]$
>>
>>
>> make install runs fine
>> rundig -v runs fine ( I used an institute's server instead of
>> www.htdig.org).
>> search works fine (so it seems)
>
> -- 
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> ht://Dig Developer mailing list:
> htd...@li...
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

[htdig-dev] Re: Is this a bug?

[Lachlan A. <lh...@us...>]

Yes, that does sound like a bug!  Well spotted, Neal!

However, if reindexing obsoletes *removed* pages correctly, then I'd=20
vote not to delay the release to fix the behaviour for pages which=20
still exist but "shouldn't" be found.  Perhaps it would be best to=20
file a bug report and fix it in 3.2.1...

Opinions?

Lachlan

On Wed, 17 Sep 2003 08:07, Neal Richter wrote:
> I'm using a fairly recent cvs snapshot...
>
>
> 1) index a website
>
> 2) use search terms to return a given page.
>
> 3) Take links to that page out of the website
>
> 4) reindex site (don't use htdig -i)
>
> 5) repeat #2
>
> I would think that this page should be 'obsoleted' and not returned
> after the reindex..
>
> htpurge does not touch these document since they haven't been
> marked as obsolete...

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Still having compile problems HP-UX 10.20

[Lachlan A. <lh...@us...>]

Greetings Jesse,

It's very good to hear that it compiles and runs!

The  make check  problem is a mystery...
Try
   cd htword ; make clean ; cd .. ; make check
If that doesn't work, could you please mail the results of
    nm */.libs/*.a | grep WordType8instance
and
    nm htword/.libs/libhtword.a | grep WordType8instance
?  Thanks.

Lachlan

On Thu, 18 Sep 2003 23:54, Jesse op den Brouw wrote:
> Patch applied, line 156 commented out in the .h file:
> configure runs fine,
> make runs fine
> make check breaks:
>
> g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o
> -L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a
> ../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a
> ../db/.libs/libhtdb.a ../htlib/.libs/libht.a -lz
> collect2: ld returned 1 exit status
> /usr/ccs/bin/ld: Unsatisfied symbols:
>    WordType::instance  (data)
> make[2]: *** [testnet] Error 1
> make[2]: Leaving directory
> `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test
>' make[1]: *** [check-am] Error 2
> make[1]: Leaving directory
> `/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test
>' make: *** [check-recursive] Error 1
> [msql@chaos htdig-3.2.0b4-20030914]$
>
>
> make install runs fine
> rundig -v runs fine ( I used an institute's server instead of
> www.htdig.org).
> search works fine (so it seems)

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Required pre-written search code

[Sunil R. <su...@ma...>]

Hi,
   We are looking for reusable search code which has below features
The search code must be configurable to search a set of static web pages
located on a number of web sites which are hosted on number of severs.
It should have capability to allow the user to search only certain subsets
of the sites for examples -
to search only one of the counties served by the site.
The search must be easily configurable so they can change the scope of the
search as the network of sites served grows.

Please let us know whether this think can implemented using your
code(ht://Dig system) or not?

If yes can you provide use demo version of it?
How much ht://Dig system will cost us?

Please let us know as soon as possible.

Thank you,
Regards,
Sunil Raskar
Project Leader
Nathan Ark Software Pvt. Ltd.

RE: [htdig-dev] Missing htconfig.h

[David B. <D.B...@vp...>]

Thanks Gabriele,

I've now run the full tests on HTDIG under RedHat Advanced Server 2.1 on
Itanium and can confirm that it passwd with flying colours!

Great work guys, its what open source is all about!

David

Re: [htdig-dev] Re: Still having compile problems HP-UX 10.20

[Jesse op d. B. <ht...@op...>]

Patch applied, line 156 commented out in the .h file:
configure runs fine,
make runs fine
make check breaks:

g++ -g -O2 -Wall -fno-rtti -fno-exceptions -o testnet testnet.o  
-L/opt/htdig/lib/zlib/lib ../htnet/.libs/libhtnet.a 
../htcommon/.libs/libcommon.a ../htword/.libs/libhtword.a 
../db/.libs/libhtdb.a ../htlib/.libs/libht.a -lz
collect2: ld returned 1 exit status
/usr/ccs/bin/ld: Unsatisfied symbols:
   WordType::instance  (data)
make[2]: *** [testnet] Error 1
make[2]: Leaving directory 
`/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory 
`/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/test'
make: *** [check-recursive] Error 1
[msql@chaos htdig-3.2.0b4-20030914]$


make install runs fine
rundig -v runs fine ( I used an institute's server instead of  
www.htdig.org).
search works fine (so it seems)

--Jesse

RE: [htdig-dev] Missing htconfig.h

[Gabriele B. <bar...@in...>]

Hi David,

>Hi Lachlan, I grabbed a new CVS and its perfect !

Sounds excellent!

>So its passing all my tests, but I'm sure you have some more serious tests,
>want me to run them ? If so, please let me know how.

I think that if you do just what Geoff says in his weekly snapshot message, 
it is more than enough:

<snip>
Must be able to
         (a) make check  and
         (b) index  www.htdig.org  using  "robotstxt_name: master-htdig"
</snip>

>Now, want some other platforms tested ? I can do AIX, HPUX (soon) and Apple
>Darwin if its any use to you.

Yep ... go on if you have the chance. I am sure Darwin has already be 
tested but if you have some time that'd be great!

Ciao and thanks again
-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Melbourne, Victoria, Australia
bar...@in... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno

RE: [htdig-dev] Missing htconfig.h

[David B. <D.B...@vp...>]

Hi Lachlan, I grabbed a new CVS and its perfect !

So its passing all my tests, but I'm sure you have some more serious tests,
want me to run them ? If so, please let me know how.

Thanks for the great work!

Now, want some other platforms tested ? I can do AIX, HPUX (soon) and Apple
Darwin if its any use to you.

David

> -----Original Message-----
> From: htd...@li...
> [mailto:htd...@li...]On Behalf Of Lachlan
> Andrew
> Sent: Thursday, September 18, 2003 12:25 AM
> To: D.B...@vp...
> Cc: htd...@li...; Gabriele Bartolini
> Subject: [htdig-dev] Missing htconfig.h
>
>
> Greetings David,
>
> Thanks for doing that testing.  The file .../include/htconfig.h  is in
> both CVS and the 7 September snapshot, and should be present.  It is
> possible that  ./configure  is removing it (it used to generate it
> from htconfig.h.in), or that the  -I  flags are not being set
> correctly on the Itanium.
>
> Could you please
> 1. Check if  .../include/htconfig.h  is indeed there.
> 2. If so, tell us the exact command which fails, including all flags,
> and the  ./configure  options you used.
> 3. If  .../include/htconfig.h  is not there:
> 3.a  Download/extract a fresh copy (from any time after 24 August)
> 3.b  Check that  include/htconfig.h  is there.  If not, let us know.
> 3.c  ./configure, and check if  .../include/htconfig.h  has gone.
> 3.d  If not, see if  make all  removes it.
> 3.e  If it is still there but  make  fails, go to 2 :)
>
> Thanks again,
> Lachlan
>
> > From: "David Bannon" <D.B...@vp...>
> >
> > I try it under Itanium RH Advanced server from time to time ....
> >
> > My (week or so ago) test still fails to compile with the same error
> > I posted
> > several weeks ago. I did not get any responses from developers then
> > so have not perused it.
> >
> > It fails with a message to the effect of :
> > Configuration.cc:20:22: htconfig.h: No such file or directory
> >
> > A quick read of the cvs indicates that htconfig.h has been removed,
> > then
> > reinstated etc. So I asked the developers if it should be there, in
> > which
> > case I should find out why its not there, or if it should not be
> > there, I'll
> > need to find our why the makefile is looking for it.
>
> --
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> ht://Dig Developer mailing list:
> htd...@li...
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev

[htdig-dev] Missing htconfig.h

[Lachlan A. <lh...@us...>]

Greetings David,

Thanks for doing that testing.  The file .../include/htconfig.h  is in=20
both CVS and the 7 September snapshot, and should be present.  It is=20
possible that  ./configure  is removing it (it used to generate it=20
from htconfig.h.in), or that the  -I  flags are not being set=20
correctly on the Itanium.

Could you please
1. Check if  .../include/htconfig.h  is indeed there.
2. If so, tell us the exact command which fails, including all flags,=20
and the  ./configure  options you used.
3. If  .../include/htconfig.h  is not there:
3.a  Download/extract a fresh copy (from any time after 24 August)
3.b  Check that  include/htconfig.h  is there.  If not, let us know.
3.c  ./configure, and check if  .../include/htconfig.h  has gone.
3.d  If not, see if  make all  removes it.
3.e  If it is still there but  make  fails, go to 2 :)

Thanks again,
Lachlan

> From: "David Bannon" <D.B...@vp...>
>
> I try it under Itanium RH Advanced server from time to time ....
>
> My (week or so ago) test still fails to compile with the same error
> I posted
> several weeks ago. I did not get any responses from developers then
> so have not perused it.
>
> It fails with a message to the effect of :
> Configuration.cc:20:22: htconfig.h: No such file or directory
>
> A quick read of the cvs indicates that htconfig.h has been removed,=20
> then
> reinstated etc. So I asked the developers if it should be there, in=20
> which
> case I should find out why its not there, or if it should not be=20
> there, I'll
> need to find our why the makefile is looking for it.

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Still having compile problems HP-UX 10.20

[Lachlan A. <lh...@us...>]

You might also need the attached patch to  HtCookie...

Lachlan

On Thu, 18 Sep 2003 00:11, Lachlan Andrew wrote:
> Greetings Jesse,
>
> Clutching at straws here...  What happens if you comment out line
> 156 of .../htlib/HtDateTime.h?
>
> Cheers,
> Lachlan
>
> On Tue, 16 Sep 2003 21:34, Jesse op den Brouw wrote:
> > This is still the problem (see add-on). What to choose: the int,
> > the int & or the const HtDateTime &
> > Hints please

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Still having compile problems HP-UX 10.20

[Lachlan A. <lh...@us...>]

Greetings Jesse,

Clutching at straws here...  What happens if you comment out line 156=20
of .../htlib/HtDateTime.h?

Cheers,
Lachlan

On Tue, 16 Sep 2003 21:34, Jesse op den Brouw wrote:

> This is still the problem (see add-on). What to choose: the int,
> the int & or the const HtDateTime &
> Hints please

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Release and phrase searching

[Lachlan A. <lh...@us...>]

Greetings Justyna,

Yes, the next production release will have phrase searching.

We *hope* to release it this month or next.  However, my personal=20
advice is not to wait until the official release.  The code is pretty=20
stable at the moment, and the release process is slow...

Cheers,
Lachlan


On Mon, 15 Sep 2003 15:34, Justyna Lubkowski wrote:

> The TODO list indicates that phrase searching is complete, but not
> in the current production release.  Could you let me know how soon
> there will be a new production release and if it will have phrase
> searching? Many thanks -
>
> Justyna Lubkowski
> 150 Lonsdale Street, Melbourne

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Is this a bug?

[Neal R. <ne...@ri...>]

I'm using a fairly recent cvs snapshot...


1) index a website

2) use search terms to return a given page.

3) Take links to that page out of the website

4) reindex site (don't use htdig -i)

5) repeat #2

I would think that this page should be 'obsoleted' and not returned
after the reindex..

htpurge does not touch these document since they haven't been marked as
obsolete...

Thanks.

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485

[htdig-dev] Still having compile problems HP-UX 10.20

[Jesse op d. B. <ht...@op...>]

Hi all,

This is still the problem (see add-on). What to choose: the int,
the int & or the const HtDateTime &
Hints please


HtFile.cc: In method `enum Transport::DocStatus HtFile::Request()':
HtFile.cc:280: call of overloaded `HtDateTime(int &)' is ambiguous
../htlib/HtDateTime.h:156: candidates are: HtDateTime::HtDateTime(int)
../htlib/HtDateTime.h:159:                 HtDateTime::HtDateTime(int &)
../htlib/HtDateTime.h:479:                 HtDateTime::HtDateTime(const 
HtDateTime &) <near match>
HtFile.cc:285: call of overloaded `HtDateTime(int &)' is ambiguous
../htlib/HtDateTime.h:156: candidates are: HtDateTime::HtDateTime(int)
../htlib/HtDateTime.h:159:                 HtDateTime::HtDateTime(int &)
../htlib/HtDateTime.h:479:                 HtDateTime::HtDateTime(const 
HtDateTime &) <near match>
HtFile.cc:307: call of overloaded `HtDateTime(int &)' is ambiguous
../htlib/HtDateTime.h:156: candidates are: HtDateTime::HtDateTime(int)
../htlib/HtDateTime.h:159:                 HtDateTime::HtDateTime(int &)
../htlib/HtDateTime.h:479:                 HtDateTime::HtDateTime(const 
HtDateTime &) <near match>
make[1]: *** [HtFile.lo] Error 1
make[1]: Leaving directory 
`/pers/www/msql/Projects/Htdig/BUILD320/htdig-3.2.0b4-20030914/htnet'
make: *** [all-recursive] Error 1



--Jesse

Re: [htdig-dev] Re: Incentives for finishing HtDig 3.2

[Neal R. <ne...@ri...>]

I rolled back the mp_alloc changes, and the Mac OS X people reported no
problems.

I just got back from vacation and will make every effort to redo the
memory leak fixes.

I am using both Insure++ (commercial) and Valgrind (open source).

As for the URL/TODO, no one really responded with outstanding tasks.

I'd vote for (2) below.  (1) will destroy the normalization of scores.

Thanks!

On Sun, 14 Sep 2003, Lachlan Andrew wrote:

> Greetings Neal,
>
> Thanks for your work in pushing 3.2 out.  What is the URL of the ToDo
> list?  As an alternative, you could modify the STATUS file in CVS,
> which is the basis of Geoff's weekly posts.
>
> The only ToDo I would add concerns backlink weights.  Below (and at
> <http://www.mail-archive.com/htd...@li.../msg01881.html>)
> is my original email from June, and Geoff's reply.  The basic problem
> is that the score based on a document is (sensibly) divided by the
> number of words in a document, but the score for links *to* the
> document isn't.  Before releasing 3.2, we should either
> (1) remove the division by document size or
> (2) change the weightings in  defaults.cc  to balance this.
> The potential disadvantage with (2) is breaking compatibility with old
> configurations.  Opinions?
>
> Cheers,
> Lachlan
>
> > The base score of documents I search for is typically 0.0001, while
> > the backlink factor is typically 2000.  Since these are added, the
> > weight given to the document itself is approximately zero!
> >
> >Does anyone know how this came about?
>
> Well, that makes some sense. We haven't "recalibrated" the scoring,
> though we trimmed out the whole "words in the front get higher score"
> bit. And since I assumed that somewhere along the 3.2 development,
> we'd add in some sort of "proximity weighting," I didn't really worry
> about it.
>
> As far as changing the weightings, I don't think anyone minds as long
> as it's explained up-front in release documentation. In particular,
> now that you don't have to reindex to change weightings, it's an easy
> change to your config file.
>
> -Geoff
>
>
>
> On Wed, 27 Aug 2003 00:04, Neal Richter wrote:
> > 	McGill University recently contacted the one of the HtDig Board
> > members to inquire about making some kind of financial arrangement
> > with HtDig to get 3.2 finished, tested and working with Phrase
> > Searching -- ie quoted strings.
> >
> > 	Please post your TODO list and I'll compile them and post them on
> > a web-page prioritized for release.  We can then have a short
> > debate and get to work.
> >
> > 	My personal opinion is that we limit the TODOs to the absolutely
> > necessary (ie satisfy Geoff's weekly status email) and get it
> > working and call it 3.2.  Everything else is a new release.
>
>
> --
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> ht://Dig Developer mailing list:
> htd...@li...
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485

Fwd: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030914

[Geoff H. <ghu...@ws...>]

Begin forwarded message:

> From: "David Bannon" <D.B...@vp...>
> Date: Sun Sep 14, 2003  5:38:32  PM America/Chicago
> To: <htd...@li...>
> Cc: "Geoff Hutchison" <ghu...@us...>
> Subject: RE: [htdig-dev] Current Status as of snapshot 3.2.0b4-20030914
> Reply-To: <D.B...@vp...>
>
>
>
> I try it under Itanium RH Advanced server from time to time ....
>
>>   To be tested:
>> .....
>>     - RedHat AdvanceServer Itanium II (David Bannon)
>
> My (week or so ago) test still fails to compile with the same error I 
> posted
> several weeks ago. I did not get any responses from developers then so 
> have
> not perused it.
>
> It fails with a message to the effect of :
> Configuration.cc:20:22: htconfig.h: No such file or directory
>
> A quick read of the cvs indicates that htconfig.h has been removed, 
> then
> reinstated etc. So I asked the developers if it should be there, in 
> which
> case I should find out why its not there, or if it should not be 
> there, I'll
> need to find our why the makefile is looking for it.
>
> I really don't have the time to waste going down wrong path when who 
> ever
> was playing there last can point me in the right direction.
>
> David
> -------------------------------------------------------------
> David Bannon                       D.B...@vp...
> Phone 61 03 9925 4733              Fax 61 03 9925 4647
> Mobile 0418 525687                 http://www.vpac.org
> Systems Manager, Victorian Partnership for Advanced Computing
> -------------------------------------------------------------
> ..... Humpty Dumpty was pushed !
>
>
>
>>   Known to have problems:
>>     - SGI/Irix 6.5.3 using SGI compilers
>>       
>> <http://www.geocrawler.com/mail/msg.php3?msg_id=8025827&list=8825>
>>
>>
>>
>>
>>
>> RELEASES:
>>    3.2.0b5: Next release, July 2003
>>    3.2.0b4: "In progress" -- snapshots called "3.2.0b4" until 
>> prerelease.
>>    3.2.0b3: Released:  22 Feb 2001.
>>    3.2.0b2: Released:  11 Apr 2000.
>>    3.2.0b1: Released:   4 Feb 2000.
>>
>> (Please note that everything added here should have a tracker PR# so
>> we can be sure they're fixed. Geoff is currently trying to add PR#s 
>> for
>> what's currently here.)
>>
>> SHOWSTOPPERS:
>> * Mifluz database errors are a severe problem (PR#428295)
>> -- Does Neal's new zlib patch solve this for now?
>>
>> KNOWN BUGS:
>> * Odd behavior with $(MODIFIED) and scores not working with
>>    wordlist_compress set but work fine without wordlist_compress.
>>    (the date is definitely stored correctly, even with compression on
>>     so this must be some sort of weird htsearch bug) PR#618737.
>> * META descriptions are somehow added to the database as FLAG_TITLE,
>>    not FLAG_DESCRIPTION. (PR#618738)
>>    Can anyone reproduce this?  I can't! -- Lachlan
>>    Me either. Let's remove the PR. -Geoff
>>
>> PENDING PATCHES (available but need work):
>> * Additional support for Win32. (Neal)
>> * Memory improvements to htmerge. (Backed out b/c htword API changed.)
>> * Mifluz merge.
>>
>> NEEDED FEATURES:
>> * Quim's new htsearch/qtest query parser framework.
>> * File/Database locking. PR#405764.
>>
>> TESTING:
>> * httools programs:
>>   (htload a test file, check a few characteristics, htdump and 
>> compare)
>> * Tests for new config file parser
>> * Duplicate document detection while indexing
>> * Major revisions to ExternalParser.cc, including fork/exec
>> instead of popen,
>>   argument handling for parser/converter, allowing binary output from 
>> an
>>   external converter.
>> * ExternalTransport needs testing of changes similar to 
>> ExternalParser.
>>
>> DOCUMENTATION:
>> * List of supported platforms/compilers is ancient. (PR#405279)
>> * Document all of htsearch's mappings of input parameters to
>> config attributes
>>    to template variables. (Relates to PR#405278.)
>>   Should we make sure these config attributes are all documented in
>>   defaults.cc, even if they're only set by input parameters and never
>>   in the config file?
>> * Split attrs.html into categories for faster loading.
>> * Turn defaults.cc into an XML file for generating documentation and
>>   defaults.cc.
>> * require.html is not updated to list new features and disk space
>>    requirements of 3.2.x (e.g. regex matching, database compression.)
>>    PRs# 405280 #405281.
>> * TODO.html has not been updated for current TODO list and
>>    completions.
>>    I've tried.  Someone "official" please check and remove this -- 
>> Lachlan
>> * Htfuzzy could use more documentation on what each fuzzy algorithm
>>    does. PR#405714.
>> * Document the list of all installed files and default
>>    locations. PR#405715.
>>
>> OTHER ISSUES:
>> * Can htsearch actually search while an index is being created?
>> * The code needs a security audit, esp. htsearch. PR#405765.
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> ht://Dig Developer mailing list:
>> htd...@li...
>> List information (subscribe/unsubscribe, etc.)
>> https://lists.sourceforge.net/lists/listinfo/htdig-dev
>
>

Re: [htdig-dev] configuration failed on Solaris 2.8

[Gabriele B. <bar...@in...>]

Hi dear Ziying,

    3.2.0b3 is actually very buggy and you should not rely on it. However, 
I replied to your bug posting just a few days ago, as you can see from: 
https://sourceforge.net/tracker/?func=detail&atid=104593&aid=799938&group_id=4593

    I could send you an e-mail because you sent an anonymous bug report.

    Please try with the latest snapshot code and let us know. 
http://htdig.sourceforge.net/files/snapshots/

Thank you,
-Gabriele

At 09.33 15/09/2003 -0400, Ziying Sherwin wrote:
>Dear Collegauges,
>
>We are trying to install htdig 3.2.0b3 on our SPARC/Solaris 2.8 platform 
>using
>gcc 3.3.  However, the configuration failed with the following error message:
>We have repeatedly tried to find others who have successfully built htdig 
>from
>source on our platform, but have found no one.  We also filed a bug report at:
>
>    http://sourceforge.net/tracker/?group_id=4593
>
>but still did not get any response.  This is a final summary of our problem,
>sent out as one last effort to find help and warn others who may be
>contemplating the same effort.  A description of the problem follows.
>
>Best Regards,
>Ziying Sherwin (sh...@nl...)
>Rick Rodgers (ro...@nl...)
>
>SUMMARY
>
>While compiling htdig 3.2.0b3 on Solaris 2.8 using gcc 3.3, we observe the
>following error message:
>
>[...copious similar material deleted here...]
>checking whether the C compiler (gcc ) is a
>cross-compiler... no
>checking whether we are using GNU C... (cached) yes
>checking whether gcc accepts -g... (cached) yes
>checking for ld used by GCC... (cached) /usr/ccs/bin/ld
>checking if the linker (/usr/ccs/bin/ld) is GNU ld...
>(cached) no
>checking for BSD-compatible nm... (cached)
>/usr/ccs/bin/nm -p
>checking whether ln -s works... (cached) yes
>loading cache ./config.cache within ltconfig
>ltconfig: you must specify a host type if you use
>                       `--no-verify'
>Try `ltconfig --help' for more information.
>configure: error: libtool configure failed
>
>We already specified the host type in the configuration option by:
>
>setenv CPPFLAGS "-Wno-deprecated"
>setenv CXXFLAGS "-Wno-deprecated"
>setenv CFLAGS "-O2 -I/depot/package/gcc_3.3/include/c++/3.3/backward"
>setenv LDFLAGS "-L/depot/package/gcc_3.3/lib"
>./configure \
>--prefix=/depot/package/htdig_3.2.0b3 \
>--with-config-dir=/depot/package/htdig_3.2.0b3/conf \
>--with-common-dir=/depot/package/htdig_3.2.0b3/share \
>--with-database-dir=/var/htdig_3.2.0b3/data \
>--with-cgi-bin-dir=/depot/package/htdig_3.2.0b3/cgi-bin \
>--with-image-dir=/depot/package/htdig_3.2.0b3/image \
>--with-search-dir=/depot/package/apache_2.0.46/htdocs/web_kit/htdig \
>--with-zlib=/depot/package/zlib_1.1.4 \
>--enable-bigfile \
>--host=solaris2.8
>
>
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>ht://Dig Developer mailing list:
>htd...@li...
>List information (subscribe/unsubscribe, etc.)
>https://lists.sourceforge.net/lists/listinfo/htdig-dev

--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Melbourne, Victoria, Australia
bar...@in... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno

[htdig-dev] configuration failed on Solaris 2.8

[Ziying S. <sh...@lh...>]

Dear Collegauges,

We are trying to install htdig 3.2.0b3 on our SPARC/Solaris 2.8 platform using 
gcc 3.3.  However, the configuration failed with the following error message:
We have repeatedly tried to find others who have successfully built htdig from 
source on our platform, but have found no one.  We also filed a bug report at:

   http://sourceforge.net/tracker/?group_id=4593

but still did not get any response.  This is a final summary of our problem, 
sent out as one last effort to find help and warn others who may be 
contemplating the same effort.  A description of the problem follows.

Best Regards,
Ziying Sherwin (sh...@nl...)
Rick Rodgers (ro...@nl...)

SUMMARY

While compiling htdig 3.2.0b3 on Solaris 2.8 using gcc 3.3, we observe the 
following error message:

[...copious similar material deleted here...]
checking whether the C compiler (gcc ) is a
cross-compiler... no
checking whether we are using GNU C... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for ld used by GCC... (cached) /usr/ccs/bin/ld
checking if the linker (/usr/ccs/bin/ld) is GNU ld...
(cached) no
checking for BSD-compatible nm... (cached)
/usr/ccs/bin/nm -p
checking whether ln -s works... (cached) yes
loading cache ./config.cache within ltconfig
ltconfig: you must specify a host type if you use
                      `--no-verify'
Try `ltconfig --help' for more information.
configure: error: libtool configure failed

We already specified the host type in the configuration option by:

setenv CPPFLAGS "-Wno-deprecated"
setenv CXXFLAGS "-Wno-deprecated"
setenv CFLAGS "-O2 -I/depot/package/gcc_3.3/include/c++/3.3/backward"
setenv LDFLAGS "-L/depot/package/gcc_3.3/lib"
./configure \
--prefix=/depot/package/htdig_3.2.0b3 \
--with-config-dir=/depot/package/htdig_3.2.0b3/conf \
--with-common-dir=/depot/package/htdig_3.2.0b3/share \
--with-database-dir=/var/htdig_3.2.0b3/data \
--with-cgi-bin-dir=/depot/package/htdig_3.2.0b3/cgi-bin \
--with-image-dir=/depot/package/htdig_3.2.0b3/image \
--with-search-dir=/depot/package/apache_2.0.46/htdocs/web_kit/htdig \
--with-zlib=/depot/package/zlib_1.1.4 \
--enable-bigfile \
--host=solaris2.8