Re: [htdig-dev] Re: Residual database errors

[Neal R. <ne...@ri...>]

On Wed, 26 Feb 2003, Lachlan Andrew wrote:

> Greetings all,
>
> Just for the record:
> 1)  The  -i  option doesn't remove the  _weakcmpr  file.
>     Neal, what effect will that have?
> 2)  I've just run  htdig  on an existing database *without*  -i  and
>     it also complained about  weakcmpr  problems.
>     (I've forgotten whether I ran  htpurge  after the first run, so
>     I'm running it again without it.)
> 3)  There is still a (different) problem with pagesize 32k.  The
>     htdig  ran OK, but the second  htpurge  complained near the end.

  #1 is easy to fix.

  Note that there is no word_db_weakcmp config variable....

Changes near htdig.cc:279

    const String  word_filename = config->Find("word_db");
    const String  word_weakcmp_filename = word_filename;
    word_weakcmp_filename.append("_weakcmpr");

    if (initial)
    {
       unlink(word_filename);
       unlink(word_weakcmp_filename);
    }

#3
  What is htpurge being run for????  Isn't its used to remove entries from
the index?  I know that htpurge is called immediately after htdig in
rundig... my question is WHY???!!!

  How are you guys using it?

  What happens when you try and use it to remove URLs from the index,
and try to add more URLs after purging??

  An interesting test would be to establish two test datasets that are
exact duplicates of each other at different URLs on your server.

  %htdig -i URL1
  %htdig -i URL2

  This would access, expand and rewrite nearly every page in the WordDB.
If there are problems rewriting/expanding pages, they may show up.


Thanks!

Neal Richter
Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485