htdig-dev — Developer Discussion for the ht://Dig project

[htdig-dev] htdig (3.1.6) search tips?

[Robert R. <ri...@li...>]

Ok, filling up the list.

could you add the resolution of  bug 165420 to the todo list on the webpage?

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=165420

Its about search tips, or rather the syntax of expressions.

Perhaps a FAQ entry on the syntax of search expressions?
Thanks

Robert

[htdig-dev] rundig again..

[Robert R. <ri...@li...>]

Hello again,

I know I am commenting on old code, but please have a look at:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=117887
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=139922

both revolve around the grep in rundig (for 3.1.6)

I have added the ^ in the debian package as the poster suggests

Robert

Re: [htdig-dev] Various things..

[Lachlan A. <lh...@us...>]

Greetings again,

I should have offered a revised  rundig  in the previous email.  See 
if the attached one suits your needs.  (It needs to be configured -- 
put it in  installdir/rundig  before running  ./configure)

FWIW, another danger of the previous version is that it has comments 
listing the "minimal" set of files which must be kept -- this list 
should now include the  .weakcmpr  file(s?).  It should probably not 
list the  .work  files, since they are automatically removed by 
rundig.

Cheers,
Lachlan

On Wed, 21 Jul 2004 10:17 pm, Robert Ribnitz wrote:
> I have therefore adapted the 'rundig' script of htdig-3.1.6, and
> attach it here..

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

Re: [htdig-dev] Various things..

[Lachlan A. <lh...@us...>]

Greetings Robert,

In 3.2.0b6,  rundig  automatically passes  -i  to htdig.  If it didn't 
in 3.1, that may account for some of the criticisms of its speed.  I 
have been meaning to make it optional, but didn't want to break any 
compatibility -- it looks like I don't have to worry about that.

Unfortunately, the role of  htmerge  has changed between 3.1 and 3.2 
-- after  htdig,  the script should now call  htpurge  instead, so 
I'm not sure if the 3.1.6 script will work as expected.

Cheers,
Lachlan

On Wed, 21 Jul 2004 10:17 pm, Robert Ribnitz wrote:
> I have a few observations:
>
> - it is possible to run htdig -i, meaning that the update is full
> rather than incremental ('initial').
> - if I use rundig -i, that option is passed to htdig (fine),  but
> also to htpurge and  htnotify  which error out, since they don't
> know any option -i
>
> I have therefore adapted the 'rundig' script of htdig-3.1.6, and
> attach it here..
>
> Please test..
>
> Robert

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Various things..

[Robert R. <ri...@li...>]

Hello,

I have pslit the packages for debian, into a htdig (V. 3.1.6 - with a 
few patches), and a htdig3.2. At the moment, they do striclty conlfict 
with each other. This is because I don't want to rename all the scripts 
form thdig to htdig3.2 etc, which would complicate an upgrade.

I know that the htdig-project is perthaps short on manpower, but 
nevertheless I have a few observations:

- it is possible to run htdig -i, meaning that the update is full rather 
than incremental ('initial').
- if I use rundig -i, that option is passed to htdig (fine),  but also 
to htpurge and  htnotify  which error out, since they don't know any 
option -i

I have therefore adapted the 'rundig' script of htdig-3.1.6, and attach 
it here..

Please test..

Robert

[htdig-dev] source code

[xaxx <xa...@id...>]

Is there anyone there?  All the site updates appear to end in 2002, and 
there does not appear to be a clear explanation of the search algorithm 
or the source code for the project anywhere on the site.

We have been using ht:/dig for our professional website and there are 
some very peculiar results which our webmaster cannot explain - largely 
because he does not understand the program.

I have been asked to unravel the problem, and I can be patient as 
needed, but not if there is no one minding the store anymore.

Please let me know how to get a copy of at least the htsearch module, 
and some decent documentation of the algorithm it implements.  Thanks,

Sincerely,

William A. Hoffman III

[htdig-dev] Basic question - ranking algorithm

[xaxx <xa...@id...>]

I'm new to ht:/dig , but have seen the results of using it on one 
website.  The thing is, the results are uneven on the website and 
similar word-counts and other factors result in reproducible but hugely 
variant scores (like 6.9344338 vs 2.2333322 with no visually 
recognizable differences in the pages or keywords).

In the past, when I've worked on projects, one of the ways for me to see 
where changes produce effects was to manually calculate a few results 
and compare them to the output of the compiled program.

However, while there is a page that outlines the weighting factors 
[Ranking pages and the use of Meta tags with ht://Dig], its description 
of the actual program is that the "htsearch... uses a complex rule to 
rank the pages."

Is there some source for the "complex rule" in pseudocode or some other 
simplified presentation format, that explains...better yet, demonstrates 
the complex rule?

Thanks for any help.  If the only answer is to download the htsearch 
module, I'm about to do that...

WAH

Re: Bug#244867: [htdig-dev] Status of htdig for Sarge..

[Robert R. <ri...@li...>]

Gilles Detillieux wrote:

>According to Robert Ribnitz:
>  
>
>>Hello Gabriele,
>>
>>Htdig does not depend on an external  libdb3 /libdb2 (since it uses its 
>>own version of it, already discussed). I was thinking about 
>>externalising libhtdig (and libhtdigphp). Forcibly, those libs would be 
>>different for the two htdig versions, and thus conflicting. To arrange 
>>the different db directories, or config file locations would be a minor 
>>issue.
>>    
>>
>
>libhtdig is part of ht://Dig 3.2, but not 3.1.6, so there shouldn't be
>an issue there.  The 3.1.6 code builds static libraries of libht.a,
>libcommon.a, libfuzzy.a and libdb.a, and links its binaries against
>these, but there shouldn't be any need to externalize any of them,
>so there shouldn't be any library conflicts.
>
>If you want to go to the effort, you could probably build 3.1.6 with an
>externalized libdb, rather than the bundled one.  However, 3.2 needs its
>own (patched) libdb, as you're already aware and have already addressed.
>So, that's not a new issue to deal with and wouldn't prevent packaging
>both 3.1.6 and 3.2.0b6 without conflicts.
>
>  
>
>>Both versions (minor issue) also depend on libnewt0, and conflict with 
>>libmifluz0 (ideally, both libmifluz0 and htdig externalise the 
>>conflicting set of libs and both depend on it. But since Mifluz is very 
>>similar to Ht://Dig, I don't expect people to install both on the same 
>>system.
>>
>>I'll do my best to get both packages into Sarge. At the moment, they 
>>conflict. If we can change the issue later on, all for the better...
>>
>>Robert
>>    
>>
>
>Again, only 3.2 uses mifluz.  3.1.6 doesn't, so there shouldn't be any
>conflict there either.  The only conflict might be that 3.1.6, 3.2, and
>the separate mifluz package all rely on slightly different versions of
>libdb.  What's libnewt0?  I didn't know either version depended on that.
>Is that a Debian addition to ht://Dig?
>  
>
After a first check it looksl ike I dont need that dependency (which I 
simply took as granted form the person packaging ht://Dig before me.
So the only (possible) point of conflict remaining is the name of the 
binariies, which is easy to change.  Will looki into it more deeply this 
week-end.

Robert

Re: [htdig-dev] Contrib Guides - update

[Gilles D. <gr...@sc...>]

According to Tony Howden:
> For the ht://dig webmaster, the contribs page for Guides has a broken 
> link at the top.
> 
> Search This! Searching Your Dynamic Site Using PHP3 and ht://Dig 
> <http://www.devshed.com/Server_Side/PHP/Search_This/page1.html> by Colin 
> Viebrock
> 
> This link goes to a default page on DevShed and misses the article by 
> Colin. A search of devshed for ht://dig reveals two related articles:
> 
> http://www.devshed.com/c/a/PHP/Search-This%21  - (this is the one that 
> was meant to be linked above)
> http://www.devshed.com/c/a/Administration/Site-Search-with-HTDIG    a 
> more recent article

Thanks.  I've updated the Guides page with these two links.

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)

Re: [htdig-dev] Htdig 3.1.6 for inclusion in Sarge...

[Gilles D. <gr...@sc...>]

According to Robert Ribnitz:
> the version of htdig 3.1.6 to include in Sarge will be the version 
> previously in Debian, plus all all patches for 3.1.6 with a later date 
> than the version on the website. Does it make sense to not patch up?

It certainly makes sense to include all bug fix patches from
ftp://ftp.ccsf.org/htdig-patches/3.1.6/, but not all patches there are bug
fixes.  Some of them may not be exactly what you want because they alter
the behaviour from what's documented and/or expected, and may be lacking
in documentation.  It would be a judgement call whether to include what
I'd consider to be optional patches such as AdjustableLoggingPatch.tar.gz,
Google-style.0, collections.0, cookie.0, cookies.0.gz, masking_noindex.0,
multiple-noindex.1, and ssl.12.  However, most other patches there are
pretty necessary bug fixes, or unobtrusive new features.

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)

Re: [htdig-dev] Status of htdig for Sarge..

[Gilles D. <gr...@sc...>]

According to Robert Ribnitz:
> Hello Gabriele,
> 
> Htdig does not depend on an external  libdb3 /libdb2 (since it uses its 
> own version of it, already discussed). I was thinking about 
> externalising libhtdig (and libhtdigphp). Forcibly, those libs would be 
> different for the two htdig versions, and thus conflicting. To arrange 
> the different db directories, or config file locations would be a minor 
> issue.

libhtdig is part of ht://Dig 3.2, but not 3.1.6, so there shouldn't be
an issue there.  The 3.1.6 code builds static libraries of libht.a,
libcommon.a, libfuzzy.a and libdb.a, and links its binaries against
these, but there shouldn't be any need to externalize any of them,
so there shouldn't be any library conflicts.

If you want to go to the effort, you could probably build 3.1.6 with an
externalized libdb, rather than the bundled one.  However, 3.2 needs its
own (patched) libdb, as you're already aware and have already addressed.
So, that's not a new issue to deal with and wouldn't prevent packaging
both 3.1.6 and 3.2.0b6 without conflicts.

> Both versions (minor issue) also depend on libnewt0, and conflict with 
> libmifluz0 (ideally, both libmifluz0 and htdig externalise the 
> conflicting set of libs and both depend on it. But since Mifluz is very 
> similar to Ht://Dig, I don't expect people to install both on the same 
> system.
> 
> I'll do my best to get both packages into Sarge. At the moment, they 
> conflict. If we can change the issue later on, all for the better...
> 
> Robert

Again, only 3.2 uses mifluz.  3.1.6 doesn't, so there shouldn't be any
conflict there either.  The only conflict might be that 3.1.6, 3.2, and
the separate mifluz package all rely on slightly different versions of
libdb.  What's libnewt0?  I didn't know either version depended on that.
Is that a Debian addition to ht://Dig?

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)

[htdig-dev] Htdig 3.1.6 for inclusion in Sarge...

[Robert R. <ri...@li...>]

Hello,

the version of htdig 3.1.6 to include in Sarge will be the version 
previously in Debian, plus all all patches for 3.1.6 with a later date 
than the version on the website. Does it make sense to not patch up?

Robert

[htdig-dev] Re: htdig-dev digest, Vol 1 #926 - 2 msgs

[Robert R. <ri...@li...>]

>
>--__--__--
>
>Message: 2
>Subject: Re: [htdig-dev] Status of htdig for Sarge..
>From: Gabriele Bartolini <g.b...@co...>
>To: Robert Ribnitz <ri...@li...>
>Cc: "ht://Dig - Dev" <htd...@li...>, 24...@bu...
>Organization: Comune di Prato
>Date: Tue, 13 Jul 2004 18:46:09 +0200
>
>Ciao Robert,
>
>   thank you again for your contribution!
>
>Il mar, 2004-07-13 alle 13:31, Robert Ribnitz ha scritto:
>  
>
>>As you know, there is a set of libraries htdig depends on. I don't think 
>>that externalising those libraries, as well as providing two separate 
>>versions of them is worth the trouble. Hence  a choice will have to be 
>>made. Either you rung the stable, proven version of htdig (3.1.6, as 
>>package 'htdig'), or you are out for helping the development team 
>>improve their efforts, or want to reach for new horizons, and install 
>>'htdig3.2'. In other words, the two versions will conflict with each other.
>>    
>>
>
>   The optimal would be to have them both. However, if this is not
>possible, your solution sounds good to me.
>
>   Would it be possible for you to remind me which external library[ies]
>would provide conflictuality between the 2 versions?
>
>Thank you again,
>-Gabriele
>  
>
Hello Gabriele,

Htdig does not depend on an external  libdb3 /libdb2 (since it uses its 
own version of it, already discussed). I was thinking about 
externalising libhtdig (and libhtdigphp). Forcibly, those libs would be 
different for the two htdig versions, and thus conflicting. To arrange 
the different db directories, or config file locations would be a minor 
issue.

Both versions (minor issue) also depend on libnewt0, and conflict with 
libmifluz0 (ideally, both libmifluz0 and htdig externalise the 
conflicting set of libs and both depend on it. But since Mifluz is very 
similar to Ht://Dig, I don't expect people to install both on the same 
system.

I'll do my best to get both packages into Sarge. At the moment, they 
conflict. If we can change the issue later on, all for the better...

Robert

Re: [htdig-dev] Status of htdig for Sarge..

[Gabriele B. <g.b...@co...>]

Ciao Robert,

   thank you again for your contribution!

Il mar, 2004-07-13 alle 13:31, Robert Ribnitz ha scritto:
> As you know, there is a set of libraries htdig depends on. I don't think 
> that externalising those libraries, as well as providing two separate 
> versions of them is worth the trouble. Hence  a choice will have to be 
> made. Either you rung the stable, proven version of htdig (3.1.6, as 
> package 'htdig'), or you are out for helping the development team 
> improve their efforts, or want to reach for new horizons, and install 
> 'htdig3.2'. In other words, the two versions will conflict with each other.

   The optimal would be to have them both. However, if this is not
possible, your solution sounds good to me.

   Would it be possible for you to remind me which external library[ies]
would provide conflictuality between the 2 versions?

Thank you again,
-Gabriele

[htdig-dev] Status of htdig for Sarge..

[Robert R. <ri...@li...>]

Hello,

After a discussion with my sponsor, it was decided to split the htdig 
package for Debian Sarge. There are people out there that rely on the 
indexing speed and on the fact of not having to rebuild their databases 
(I heard of cases of two to three diigit gigabyte figures of documents 
being indexed). Those folks want to rely on the stability and continuity 
of htdig 3.1.6, which will be available as package 'htdig' in Debian Sarge.

On the other hand, there are people who do not need to index that many 
documents or who like to benefit from the new features htdig 3.2.0 has 
introduced. Those people can live with the fact that htdig 3.2.0(b6) 
will run slower than the stable version, and that reindexing their 
databases is needed. For those people, a new package htdig3.2 has been 
created, with the first version entering debian being 3.20b6-1 (Based on 
3.2.0b6).

As you know, there is a set of libraries htdig depends on. I don't think 
that externalising those libraries, as well as providing two separate 
versions of them is worth the trouble. Hence  a choice will have to be 
made. Either you rung the stable, proven version of htdig (3.1.6, as 
package 'htdig'), or you are out for helping the development team 
improve their efforts, or want to reach for new horizons, and install 
'htdig3.2'. In other words, the two versions will conflict with each other.

In the hope of being able to satisfy most

Robert Ribnitz
(Debian Maintainer of htdig and htdig3.2)

[htdig-dev] [Task #87780] Misc URL related

[SourceForge.net <no...@so...>]

Task #87780 has been updated. 

Project: ht://Dig
Subproject: Testing 3.2
Summary: Misc URL related
Complete: 100%
Status: Open
Authority  : nealr
Assigned to: lha

Description: Use the documenation to test each one of these config verbs

# URLs

allow_double_slash

allow_virtual_hosts

case_sensitive

local_default_doc

remove_default_doc

common_url_parts

url_log

url_seed_score

server_aliases





Follow-Ups:

-------------------------------------------------------
Date: 2004-07-11 20:54
By: lha

Comment:
All incorporated into test suite except allow_virtual_hosts  and  url_log, which I have tested manually.  More testing/documentation update on url_log would be good.  (In particular, how should ht://Dig behave if the log file is there when it starts?)

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/pm/task.php?func=detailtask&project_task_id=87780&group_id=4593&group_project_id=32485

[htdig-dev] Re: Make check and htdig warnings

[Lachlan A. <lh...@us...>]

Greetings Joe,

Well spotted.  That shows the cause of the bug -- it's just in the 
test script, not ht://Dig itself.  Before I can fix it, I'll have to 
get the tests working with Apache2 (or install an old version).  The 
problem is that that particular dig is tesing both  url_rewrite_rules  
and  case_sensitive.  I'll have to separate those two tests.

Cheers,
Lachlan

On Sat, 10 Jul 2004 07:15 pm, Joe R. Jah wrote:
>
> No I masked the absolute path with tmp, but I think that also
> masked the cause of site4.html not being indexed;( The path to
> site4.html is turned to all lower case, although my OS is case
> sensitive;-/

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Current Status as of snapshot 3.2.0b6-20040711

[Geoff H. <ghu...@us...>]

STATUS of ht://Dig branch 3-2-x

RELEASES:
   3.2.0b6: Scheduled: 31 May 2004.
   3.2.0b5: Released:  10 Nov 2003.
   3.2.0b4: Cancelled.
   3.2.0b3: Released:  22 Feb 2001.
   3.2.0b2: Released:  11 Apr 2000.
   3.2.0b1: Released:   4 Feb 2000.

(Please note that everything added here should have a tracker PR# so
we can be sure they're fixed. Geoff is currently trying to add PR#s for
what's currently here.)

SHOWSTOPPERS:

KNOWN BUGS:
(none serious.
See <http://sourceforge.net/tracker/?atid=104593&group_id=4593&func=browse>.)

PENDING PATCHES (available but need work):
* Gilles's configuration parsing patches need testing before committing.
* Memory improvements to htmerge. (Backed out b/c htword API changed.)
* Mifluz merge.  (Is this still pending??)

NEEDED FEATURES:
* Quim's new htsearch/qtest query parser framework.
* File/Database locking. PR#405764.

TESTING:
* httools programs: 
  (htload a test file, check a few characteristics, htdump and compare)
* Tests for new config file parser
* Duplicate document detection while indexing
* Major revisions to ExternalParser.cc, including fork/exec instead of popen,
  argument handling for parser/converter, allowing binary output from an
  external converter.
* ExternalTransport needs testing of changes similar to ExternalParser.

DOCUMENTATION:
* List of supported platforms/compilers is ancient. (PR#405279)
* Document all of htsearch's mappings of input parameters to config attributes
   to template variables. (Relates to PR#405278.)
  Should we make sure these config attributes are all documented in
  defaults.cc, even if they're only set by input parameters and never
  in the config file?
* Split attrs.html into categories for faster loading.
* Turn defaults.cc into an XML file for generating documentation and 
  defaults.cc.
* require.html is not updated to list new features and disk space
   requirements of 3.2.x (e.g. regex matching, database compression.)
   PRs# 405280 #405281.
* Htfuzzy could use more documentation on what each fuzzy algorithm
   does. PR#405714.
* Document the list of all installed files and default
   locations. PR#405715.

OTHER ISSUES:
* Can htsearch actually search while an index is being created?
* The code needs a security audit, esp. htsearch. PR#405765.

[htdig-dev] Re: Make check and htdig warnings

[Joe R. J. <jj...@cl...>]

On Sat, 10 Jul 2004, Lachlan Andrew wrote:

> Date: Sat, 10 Jul 2004 17:54:38 +1000
> From: Lachlan Andrew <lh...@us...>
> To: Joe R. Jah <jj...@cl...>
> Cc: htd...@li...
> Subject: Re: Make check and htdig warnings
> 
> Greetings Joe,
> 
> Thanks for running that test.
> 
> It looks like you have compiled in the directory  /tmp/htdig-3.2.0b6 
> -- is that right?  If so, the 
> file  /tmp/htdig-3.2.0b6/test/htdocs/set1/site4.html  should exist, 
> so it is odd that it isn't being indexed.

No I masked the absolute path with tmp, but I think that also masked the
cause of site4.html not being indexed;( The path to site4.html is turned
to all lower case, although my OS is case sensitive;-/

In the attached file the path is /tmp/WWW/htdig/htdig-3.2.0b6, but the
output shows that gmake check tries to find site4.html in:

 /tmp/www/htdig/htdig-3.2.0b6/test/htdocs/set1/site4.html
      ^^^
      WWW

> Could you please try the attached test file?  To do that, either
> 1. Copy it to  .../test/t_htdig    and  "make check"
> or
> 2. Copy it to  .../test/t_htdig_v  and  "make TESTS=t_htdig_v check"
> (You can either keep the URL.cc file that I sent last time or replace 
> it with the normal one -- it won't matter.)
> 
> (By the way, I got one copy of each of your three emails -- all 
> straight from you not through the list.)

This time I have gzipped it, so it should get through the first time;)

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

> Thanks again for this,
> Lachlan
> 
> On Thu, 8 Jul 2004 05:12 pm, Joe R. Jah wrote:
> > Attached is the output of gmake check.
> 
> -- 
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Make check and htdig warnings

[Lachlan A. <lh...@us...>]

Greetings Joe,

Thanks for running that test.

It looks like you have compiled in the directory  /tmp/htdig-3.2.0b6 
-- is that right?  If so, the 
file  /tmp/htdig-3.2.0b6/test/htdocs/set1/site4.html  should exist, 
so it is odd that it isn't being indexed.

Could you please try the attached test file?  To do that, either
1. Copy it to  .../test/t_htdig    and  "make check"
or
2. Copy it to  .../test/t_htdig_v  and  "make TESTS=t_htdig_v check"
(You can either keep the URL.cc file that I sent last time or replace 
it with the normal one -- it won't matter.)

(By the way, I got one copy of each of your three emails -- all 
straight from you not through the list.)

Thanks again for this,
Lachlan

On Thu, 8 Jul 2004 05:12 pm, Joe R. Jah wrote:
> Attached is the output of gmake check.

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Lachlan A. <lh...@us...>]

Greetings Robert,

That is a good suggestion.  However, Gabriele is right that it is not 
as easy as it sounds...  It was your line of thinking that made me 
introduce the "store_phrases" attribute, with the aim of making the 
behaviour more like 3.1 when set to "false".  Unfortunately, it 
hardly changes the speed at all.  The only reason to keep is it that 
it reduces the database size a bit.

The only way to make the behaviour of 3.2 exactly like 3.1 is to make 
it 3.1.  One option would be to make a package which builds both 
versions -- that is almost what would be needed to get the speed back 
up to 3.1 levels.

Cheers,
Lachlan

On Thu, 8 Jul 2004 05:24 am, Robert Ribnitz wrote:
> the main feature that 3.2. will bring over 3.1 is to allow 'phrase
> searching', whicxh means that the indexing process will slow up to
> 75% (compared 3.2.0b6 to 3.1.6)
>
> I haven't looked at the code in depth, so I don't know how hard it
> would be to implement a config file option that would switch off
> that new behaviour, making 3.2.0b6 behave exactly like 3.1.6?

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Make check and htdig warnings (fwd)

[Joe R. J. <jj...@cl...>]

Greetings Lachlan,

This is the third time I am sending this message;) but this time I have
put the attachment on the patch archives instead:

 ftp://ftp.ccsf.org/htdig-patches/3.2.0b6/0Tests/gmake-check.output.0

Evidently it has some SPAM addresses in it that gets blocked somewhere in
the mail chain;-/  I am sorry Lachlan if your receiving triplicates;(

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

---------- Forwarded message ----------
Date: Thu, 8 Jul 2004 00:12:02 -0700 (PDT)
From: Joe R. Jah <jj...@cl...>
To: Lachlan Andrew <lh...@us...>
Cc: htd...@li...
Subject: Re: Make check and htdig warnings

On Wed, 7 Jul 2004, Lachlan Andrew wrote:

> Date: Wed, 7 Jul 2004 23:39:35 +1000
> From: Lachlan Andrew <lh...@us...>
> To: Joe R. Jah <jj...@cl...>
> Cc: htd...@li...
> Subject: Re: Make check and htdig warnings
> 
> Greetings Joe,
> 
> OK, this might take some playing...  Could you please copy the attached
> file to  htcommon/URL.cc  and send me the output?

Greetings Lachlan,

Attached is the output of gmake check.

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

> Unfortunately, I've recently upgraded to  Apache 2,  which isn't
> supported by the test suite :(   Does anyone else find
>   make TESTS=t_htdig check
> fails on any other platforms?
> 
> Thanks for your help, Joe!
> Lachlan
> 
> On Thu, 24 Jun 2004 03:45 am, Joe R. Jah wrote:
> > On Wed, 23 Jun 2004, Lachlan Andrew wrote:
> > > I think that the attached patch should fix it.  If so, I'll have
> > > to work out why  t_htdig_local  *was* working...
> >
> > No, it doesn't:
> 
> -- 
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Joe R. J. <jj...@cl...>]

On Wed, 7 Jul 2004, Gabriele Bartolini wrote:

> Date: Wed, 07 Jul 2004 21:44:41 +0200
> From: Gabriele Bartolini <an...@ti...>
> To: Robert Ribnitz <ri...@li...>, htd...@li...
> Subject: Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..
> 
> At 21.24 07/07/2004, Robert Ribnitz wrote:
> >Hello List,
> 
> Hi Robert,
> 
> >I haven't looked at the code in depth, so I don't know how hard it would 
> >be to implement a config file option that would switch off that new 
> >behaviour, making 3.2.0b6 behave exactly like 3.1.6?
> 
> Unfortunately it is not that easy. The database structure is very different 
> and it is not so easy to adapt it. I believe that it needs less time and 
> work to work on optimisation than porting forward the 3.1.x code and allow 
> a run-time different behaviour.
> 
> >Would that be worth a try (it could save me from packaging two different 
> >versions for Sarge release)?
> 
> Don't take me wrong. Having seen the progress of the code, I think it is 
> easier and faster to produce two different packages, maybe htdig and htdig31.
> 
> Ciao and thanks a lot for your contribution!
> -Gabriele

There is a config file attribute, store_phrases, in 3.2.0b6 that if set to
no it would save about 27% indexing time; it will not make it quite as
fast as 3.1.6, but it takes a great step towards it;)  That said, I am not
too fond of it because it has slightly slowed down regular indexing;(  and
I totally concur with Gabriele's reasoning.

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

[htdig-dev] Re: Make check and htdig warnings

[Joe R. J. <jj...@cl...>]

On Wed, 7 Jul 2004, Lachlan Andrew wrote:

> Date: Wed, 7 Jul 2004 23:39:35 +1000
> From: Lachlan Andrew <lh...@us...>
> To: Joe R. Jah <jj...@cl...>
> Cc: htd...@li...
> Subject: Re: Make check and htdig warnings
> 
> Greetings Joe,
> 
> OK, this might take some playing...  Could you please copy the attached
> file to  htcommon/URL.cc  and send me the output?

Greetings Lachlan,

Attached is the output of gmake check.

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

> Unfortunately, I've recently upgraded to  Apache 2,  which isn't
> supported by the test suite :(   Does anyone else find
>   make TESTS=t_htdig check
> fails on any other platforms?
> 
> Thanks for your help, Joe!
> Lachlan
> 
> On Thu, 24 Jun 2004 03:45 am, Joe R. Jah wrote:
> > On Wed, 23 Jun 2004, Lachlan Andrew wrote:
> > > I think that the attached patch should fix it.  If so, I'll have
> > > to work out why  t_htdig_local  *was* working...
> >
> > No, it doesn't:
> 
> -- 
> lh...@us...
> ht://Dig developer DownUnder  (http://www.htdig.org)

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Gabriele B. <an...@ti...>]

At 21.24 07/07/2004, Robert Ribnitz wrote:
>Hello List,

Hi Robert,

>I haven't looked at the code in depth, so I don't know how hard it would 
>be to implement a config file option that would switch off that new 
>behaviour, making 3.2.0b6 behave exactly like 3.1.6?

Unfortunately it is not that easy. The database structure is very different 
and it is not so easy to adapt it. I believe that it needs less time and 
work to work on optimisation than porting forward the 3.1.x code and allow 
a run-time different behaviour.

>Would that be worth a try (it could save me from packaging two different 
>versions for Sarge release)?

Don't take me wrong. Having seen the progress of the code, I think it is 
easier and faster to produce two different packages, maybe htdig and htdig31.

Ciao and thanks a lot for your contribution!
-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Prato, Toscana, Italia
an...@ti... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno