Re: [htdig-dev] Re: Residual database errors

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

On Sunday, February 23, 2003, at 06:21 AM, Lachlan Andrew wrote:

> If I run the attached  rundig  script, with  -v  and the attached
> .conf  script on the attached directory (51 copies of the attached
> file  hash) with an empty  .../var/htdig-crash1  directory, then all
> is well.  However, if I run it a *second* time, it gives the attached
> log  file.
>
> This is odd since the script uses  -i  which is supposed to ignore the
> contents of the directory.  (On another note, should  -i  also ignore
> the  db.log  file?  It currently doesn't.)
>
> Neal, can you (or anyone else) replicate this behaviour?

Hi - I was able to duplicate the problem on a machine running Red Hat 
8.0. The results of the second run were almost identical to what you 
log file shows. I was neither redirecting stderr nor paying attention 
when the errors started, so I didn't catch the page number of the first 
failure. I am in the process of repeating the experiment and will catch 
the page number this time around.

I tried the same thing on an OS X box, and htdig core dumps (segfault) 
early on during the first pass with rundig. The core file indicates 
failure in a vm_allocate call, however the backtrace shows the call to 
be 3000+ CDB___* calls deep. The entry point from the backtrace is show 
below. I can provide the full backtrace if anyone wants to see it.

Jim

#3249 0x00054bc4 in CDB___bam_c_put (dbc_orig=0x19e9660, 
key=0xbfffdd30, data=0xbfffdd50, flags=15) at bt_cursor.c:925
#3250 0x0003c7b0 in CDB___db_put (dbp=0x19e9904, txn=0x19e9660, 
key=0xa99640, data=0xbfffdd30, flags=27170944) at db_am.c:508
#3251 0x00025298 in WordList::Put(WordReference const&, int) 
(this=0xbffff1f0, arg=@0xbfffdd30, flags=1) at WordDB.h:126
#3252 0x0001eeac in HtWordList::Flush() (this=0xbffff1f0) at 
../htword/WordList.h:118
#3253 0x00020c9c in DocumentRef::AddDescription(char const*, 
HtWordList&) (this=0x548ab90, d=0x11bd98 "", words=@0xbffff1f0) at 
DocumentRef.cc:512
#3254 0x0000bac8 in Retriever::got_href(URL&, char const*, int) 
(this=0xbffff140, url=@0x1ae8d30, description=0x5446880 "QPainter", 
hops=1) at Retriever.cc:1496
#3255 0x00005c84 in HTML::do_tag(Retriever&, String&) (this=0xbba9b0, 
retriever=@0xbffff140, tag=@0xbbaa24) at ../htlib/htString.h:45
#3256 0x00005120 in HTML::parse(Retriever&, URL&) (this=0xbba9b0, 
retriever=@0xbffff140, baseURL=@0x10000) at HTML.cc:414
#3257 0x00009a24 in Retriever::RetrievedDocument(Document&, String 
const&, DocumentRef*) (this=0xbffff140, doc=@0xa99950, url=@0x10000, 
ref=0xbb9ad0) at Retriever.cc:818
#3258 0x000094d0 in Retriever::parse_url(URLRef&) (this=0xbffff140, 
urlRef=@0x1b730c0) at ../htcommon/URL.h:51
#3259 0x00008b28 in Retriever::Start() (this=0xbffff140) at 
Retriever.cc:432
#3260 0x0000f988 in main (ac=5, av=0xbffff9c4) at htdig.cc:338
#3261 0x0000266c in _start (argc=5, argv=0xbffff9c4, envp=0xbffff9dc) 
at /SourceCache/Csu/Csu-45/crt.c:267
#3262 0x000024ec in start () at 
/usr/include/gcc/darwin/3.1/g++-v3/streambuf:129