[htdig-dev] purge and re-index failing

[Christopher M. <chr...@mc...>]

Greetings htdig folks,

 Recently I've been trying to have htDig purge and re-index items (via a
trigger in Postgres). The purge seems to work as I no longer see the
item in the search results, however, when I try to re-index, I cannot
bring the page back in unless I do a full index. I've just installed
3.2.0b5 hoping that this would help, but no luck. Here's some output
from my command line attempts to get it to work:


[root@lovelace bin]# ./htpurge -c /www/htdig/install/conf/ads.conf -u http://newfind.mcgill.ca/indexes/ads/?AdsID=10266860

[root@lovelace bin]# echo 'http://newfind.mcgill.ca/indexes/ads/?AdsID=1026860' | ./htdig - -s -v -m -c /www/htdig/install/conf/ads.conf

ht://dig Start Time: Thu Nov 13 16:36:02 2003

New server: newfind.mcgill.ca, 80
0:11472:0:http://newfind.mcgill.ca/indexes/ads/?AdsID=1026860:  (changed)  size = 660
htdig: Run complete
htdig: 1 server seen:
htdig:     newfind.mcgill.ca:80 1 document

HTTP statistics
===============
 Persistent connections    : Yes
 HEAD call before GET      : Yes
 Connections opened        : 2
 Connections closed        : 1
 Changes of server         : 0
 HTTP Requests             : 3
 HTTP KBytes requested     : 0.442383
 HTTP Average request time : 0 secs
 HTTP Average speed        : inf KBytes/secs

ht://dig End Time: Thu Nov 13 16:36:03 2003

So although this thing has been purged and re-entered, it no longer
shows up in the query results. Also, it seems that the dbs aren't being
updated after the htperge and htdig. Again more output from my konsole
(note the moddates and filesizes - also, the filesize of db.docdb
doesn't change between the purge and re-index):

[root@lovelace bin]# ls -ltr /www/htdig/install/var/ads
total 1584
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.excerpts.work
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.docs.index.work
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.docdb.work
-rw-r--r--    1 root     root        16384 Nov 13 16:14 db.words.db_weakcmpr
-rw-r--r--    1 root     root       619520 Nov 13 16:36 db.words.db
-rw-r--r--    1 root     root       655360 Nov 13 16:36 db.excerpts
-rw-r--r--    1 root     root       172032 Nov 13 16:36 db.docs.index
-rw-r--r--    1 root     root       344064 Nov 13 16:38 db.docdb

[root@lovelace bin]# ./htpurge -c /www/htdig/install/conf/ads.conf -u http://newfind.mcgill.ca/indexes/ads/?AdsID=1025825

[root@lovelace bin]# echo 'http://newfind.mcgill.ca/indexes/ads/?AdsID=1025825' | ./htdig - -s -v -m -c /www/htdig/install/conf/ads.conf

ht://dig Start Time: Thu Nov 13 17:05:14 2003

New server: newfind.mcgill.ca, 80
0:11475:0:http://newfind.mcgill.ca/indexes/ads/?AdsID=1025825:  (changed)  size = 336
htdig: Run complete
htdig: 1 server seen:
htdig:     newfind.mcgill.ca:80 1 document

HTTP statistics
===============
 Persistent connections    : Yes
 HEAD call before GET      : Yes
 Connections opened        : 2
 Connections closed        : 1
 Changes of server         : 0
 HTTP Requests             : 3
 HTTP KBytes requested     : 0.442383
 HTTP Average request time : 0 secs
 HTTP Average speed        : inf KBytes/secs

ht://dig End Time: Thu Nov 13 17:05:14 2003

[root@lovelace bin]# ls -ltr /www/htdig/install/var/ads
total 1584
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.excerpts.work
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.docs.index.work
-rw-r--r--    1 root     root        24576 Nov 13 13:35 db.docdb.work
-rw-r--r--    1 root     root        16384 Nov 13 16:14 db.words.db_weakcmpr
-rw-r--r--    1 root     root       619520 Nov 13 16:36 db.words.db
-rw-r--r--    1 root     root       655360 Nov 13 16:36 db.excerpts
-rw-r--r--    1 root     root       172032 Nov 13 16:36 db.docs.index
-rw-r--r--    1 root     root       344064 Nov 13 17:05 db.docdb


So, and info or help on this would be much appreciated.

Cheers,

Chris

-- 
Christopher Murtagh
Enterprise Systems Administrator
ISR / Web Communications Group 
McGill University
Montreal, Quebec
Canada

Tel.: (514) 398-3122
Fax:  (514) 398-2017