htdig-dev

[htdig-dev] Idea: Option to disable 'phrase indexing'..

[Robert R. <ri...@li...>]

Hello List,

the main feature that 3.2. will bring over 3.1 is to allow 'phrase 
searching', whicxh means that the indexing process will slow up to 75% 
(compared 3.2.0b6 to 3.1.6)

I haven't looked at the code in depth, so I don't know how hard it would 
be to implement a config file option that would switch off that new 
behaviour, making 3.2.0b6 behave exactly like 3.1.6?

Would that be worth a try (it could save me from packaging two different 
versions for Sarge release)?

What do you think, is there any speed (and acceptance) to be gained by 
such an option?

Robert

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Gabriele B. <an...@ti...>]

At 21.24 07/07/2004, Robert Ribnitz wrote:
>Hello List,

Hi Robert,

>I haven't looked at the code in depth, so I don't know how hard it would 
>be to implement a config file option that would switch off that new 
>behaviour, making 3.2.0b6 behave exactly like 3.1.6?

Unfortunately it is not that easy. The database structure is very different 
and it is not so easy to adapt it. I believe that it needs less time and 
work to work on optimisation than porting forward the 3.1.x code and allow 
a run-time different behaviour.

>Would that be worth a try (it could save me from packaging two different 
>versions for Sarge release)?

Don't take me wrong. Having seen the progress of the code, I think it is 
easier and faster to produce two different packages, maybe htdig and htdig31.

Ciao and thanks a lot for your contribution!
-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check 
maintainer
Current Location: Prato, Toscana, Italia
an...@ti... | http://www.prato.linux.it/~gbartolini | ICQ#129221447
 > "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The 
Inferno

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Joe R. J. <jj...@cl...>]

On Wed, 7 Jul 2004, Gabriele Bartolini wrote:

> Date: Wed, 07 Jul 2004 21:44:41 +0200
> From: Gabriele Bartolini <an...@ti...>
> To: Robert Ribnitz <ri...@li...>, htd...@li...
> Subject: Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..
> 
> At 21.24 07/07/2004, Robert Ribnitz wrote:
> >Hello List,
> 
> Hi Robert,
> 
> >I haven't looked at the code in depth, so I don't know how hard it would 
> >be to implement a config file option that would switch off that new 
> >behaviour, making 3.2.0b6 behave exactly like 3.1.6?
> 
> Unfortunately it is not that easy. The database structure is very different 
> and it is not so easy to adapt it. I believe that it needs less time and 
> work to work on optimisation than porting forward the 3.1.x code and allow 
> a run-time different behaviour.
> 
> >Would that be worth a try (it could save me from packaging two different 
> >versions for Sarge release)?
> 
> Don't take me wrong. Having seen the progress of the code, I think it is 
> easier and faster to produce two different packages, maybe htdig and htdig31.
> 
> Ciao and thanks a lot for your contribution!
> -Gabriele

There is a config file attribute, store_phrases, in 3.2.0b6 that if set to
no it would save about 27% indexing time; it will not make it quite as
fast as 3.1.6, but it takes a great step towards it;)  That said, I am not
too fond of it because it has slightly slowed down regular indexing;(  and
I totally concur with Gabriele's reasoning.

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

Re: [htdig-dev] Idea: Option to disable 'phrase indexing'..

[Lachlan A. <lh...@us...>]

Greetings Robert,

That is a good suggestion.  However, Gabriele is right that it is not 
as easy as it sounds...  It was your line of thinking that made me 
introduce the "store_phrases" attribute, with the aim of making the 
behaviour more like 3.1 when set to "false".  Unfortunately, it 
hardly changes the speed at all.  The only reason to keep is it that 
it reduces the database size a bit.

The only way to make the behaviour of 3.2 exactly like 3.1 is to make 
it 3.1.  One option would be to make a package which builds both 
versions -- that is almost what would be needed to get the speed back 
up to 3.1 levels.

Cheers,
Lachlan

On Thu, 8 Jul 2004 05:24 am, Robert Ribnitz wrote:
> the main feature that 3.2. will bring over 3.1 is to allow 'phrase
> searching', whicxh means that the indexing process will slow up to
> 75% (compared 3.2.0b6 to 3.1.6)
>
> I haven't looked at the code in depth, so I don't know how hard it
> would be to implement a config file option that would switch off
> that new behaviour, making 3.2.0b6 behave exactly like 3.1.6?

-- 
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)