[Lurker-users] Debugging signal 11 program faults

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi everyone,

I have been running Lurker for quite awhile now to handle message
archives at <http://marc.free.net.ph/>. I had originally been using XFS,
then shifted to ext3, and then back.

The Lurker database I was previously using was built when the system was
running ext3, and seemed to be ok. Recently I purged the database to add
some "back-issues" of the linux-xfs mailing list to the archives, and
now cannot rebuild the database. Lurker keeps getting a signal 11
program fault, traps it, and syncs the database to disk. This continues
to happen when I rerun lurkerd, whether I purge the database or let it
resume where it left off.

I am using the lurkerd_0.1f-5 Debian package. I rebuilt the package with
only one change: I added "-g" to the CFLAGS to allow gdb to do tracing.
I could not run lurkerd from gdb, because it would just exit normally
without doing anything. So I started lurkerd the standard Debian way,
which makes it run as the www-data user, then attached to this process
manually using gdb, also as the www-data user.

First I got a lot of messages[1] about the attachment process which
seemed to work fine, although there was a permission denied error at
mbox.c line 220 for which I have a backtrace[2]. I made gdb continue,
and it took awhile, seemingly doing its real work. Then I hit the
SIGSEGV[3], at line 172 of indexer.c, for which a backtrace is also
available[4].

These have been completely reproducible lately. Without purging the
database, I ran lurkerd again then attached to the running process using
gdb. After awhile, the process ran into SIGSEGV again at line 172 of
indexer.c, information[5] on which I am providing in case it can help.

I have checked, and the mboxes lurker is reading are readable by
www-data. Please let me know what other information I may provide to
help in the debugging of this problem that is keeping me from providing
the archives using lurker.

Thank you very much for your time.

 --> Jijo

[1] Messages during attachment:

    This GDB was configured as "i386-linux"...
    Attaching to program: /usr/sbin/lurkerd, process 5768
    Reading symbols from /usr/lib/libst.so.1...done.
    Loaded symbols for /usr/lib/libst.so.1
    Reading symbols from /usr/lib/libc-client.so.2003debian...done.
    Loaded symbols for /usr/lib/libc-client.so.2003debian
    Reading symbols from /lib/libm.so.6...done.
    Loaded symbols for /lib/libm.so.6
    Reading symbols from /usr/lib/i686/cmov/libssl.so.0.9.7...done.
    Loaded symbols for /usr/lib/i686/cmov/libssl.so.0.9.7
    Reading symbols from /lib/libc.so.6...done.
    Loaded symbols for /lib/libc.so.6
    Reading symbols from /lib/libpam.so.0...done.
    Loaded symbols for /lib/libpam.so.0
    Reading symbols from /lib/libdl.so.2...done.
    Loaded symbols for /lib/libdl.so.2
    Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.7...done.
    Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.7
    Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
    Loaded symbols for /usr/lib/libgssapi_krb5.so.2
    Reading symbols from /usr/lib/libkrb5.so.3...done.
    Loaded symbols for /usr/lib/libkrb5.so.3
    Reading symbols from /usr/lib/libk5crypto.so.3...done.
    Loaded symbols for /usr/lib/libk5crypto.so.3
    Reading symbols from /lib/libcom_err.so.2...done.
    Loaded symbols for /lib/libcom_err.so.2
    Reading symbols from /lib/ld-linux.so.2...done.
    Loaded symbols for /lib/ld-linux.so.2
    Reading symbols from /lib/libresolv.so.2...done.
    Loaded symbols for /lib/libresolv.so.2
    Reading symbols from /usr/lib/gconv/ISO8859-1.so...done.
    Loaded symbols for /usr/lib/gconv/ISO8859-1.so
    0x08052dc3 in my_mbox_find_special (k=0x806bbad "\nFrom ", 
        s=0x41100000 "gin except root.\n> Removing the \"*\" tells the system that the password is disabled for that login.\n> I see saw this as a security problem if I removed the \"*\" so I tried to login\n> as \"+\" - I wouldn't"..., 
       e=0x41100ae0 "13:32:16 -0500\nFrom: Alan Cox <al...@cy...>\nMessage-Id: <199...@sn...>\nSubject: Re: psaux / watchdog combination problem ..\nTo: dr...@rf... (Dan McLaughlin)\nDat"...) at mbox.c:220
    220     mbox.c: Permission denied.
            in mbox.c

[2] Backtrace of permission denied error in mbox.c line 220:

    (gdb) bt
    #0  0x08052dc3 in my_mbox_find_special (k=0x806bbad "\nFrom ", 
        s=0x41100000 "gin except root.\n> Removing the \"*\" tells the system that the password is disabled for that login.\n> I see saw this as a security problem if I removed the \"*\" so I tried to login\n> as \"+\" - I wouldn't"..., 
        e=0x41100ae0 "13:32:16 -0500\nFrom: Alan Cox <al...@cy...>\nMessage-Id: <199...@sn...>\nSubject: Re: psaux / watchdog combination problem ..\nTo: dr...@rf... (Dan McLaughlin)\nDat"...) at mbox.c:220
    #1  0x08054530 in lu_mbox_map_message (mbox=0x8073bdc, msg=0x8073c00, start=2554592) at mbox.c:1040
    #2  0x08052fc7 in my_mbox_process_mbox (mbox=0x8073bdc, list=0x8073028, stamp=826782874) at mbox.c:314
    #3  0x08054030 in my_mbox_watch (arg=0x0) at mbox.c:836
    #4  0x4001ab02 in st_thread_create () from /usr/lib/libst.so.1
    #5  0xbffffa84 in ?? ()
    #6  0x00000000 in ?? ()

[3] SIGSEGV at indexer.c:

    Program received signal SIGSEGV, Segmentation fault.
    0x08050501 in my_indexer_traverse (in=0x403f2dc0, body=0x80b6090) at indexer.c:172
    172     indexer.c: Permission denied.
            in indexer.c

[4] Backtrace of permission denied at indexer.c line 172:

    (gdb) bt
    #0  0x08050501 in my_indexer_traverse (in=0x403f2dc0, body=0x80b6090) at indexer.c:172
    #1  0x0805057f in my_indexer_traverse (in=0x403f2dc0, body=0x80b6a08) at indexer.c:195
    #2  0x08050baa in lu_indexer_message (body=0x403f2dc0, stamp=852695640, reply_id=0x403f2b50 "") at indexer.c:443
    #3  0x08053634 in my_mbox_process_mbox (mbox=0x8073c28, list=0x8073028, stamp=852695640) at mbox.c:496
    #4  0x08054030 in my_mbox_watch (arg=0x0) at mbox.c:836
    #5  0x4001ab02 in st_thread_create () from /usr/lib/libst.so.1
    #6  0x4001aa2a in st_thread_create () from /usr/lib/libst.so.1
    #7  0x00000000 in ?? ()

[5] SIGSEGV at indexer.c after rerunning gdb with the salvaged db from
    the previous run:

    Program received signal SIGSEGV, Segmentation fault.
    0x08050501 in my_indexer_traverse (in=0x403f2dc0, body=0x80a2288) at indexer.c:172
    172     indexer.c: Permission denied.
            in indexer.c
    (gdb) bt
    #0  0x08050501 in my_indexer_traverse (in=0x403f2dc0, body=0x80a2288) at indexer.c:172
    #1  0x0805057f in my_indexer_traverse (in=0x403f2dc0, body=0x80a2b90) at indexer.c:195
    #2  0x08050baa in lu_indexer_message (body=0x403f2dc0, stamp=856902135, reply_id=0x403f2b50 "") at indexer.c:443
    #3  0x08053634 in my_mbox_process_mbox (mbox=0x8073c28, list=0x8073028, stamp=856902135) at mbox.c:496
    #4  0x08054030 in my_mbox_watch (arg=0x0) at mbox.c:836
    #5  0x4001ab02 in st_thread_create () from /usr/lib/libst.so.1
    #6  0x4001aa2a in st_thread_create () from /usr/lib/libst.so.1
    #7  0x00000000 in ?? ()

-- 
Federico Sevilla III  : http://jijo.free.net.ph      : When we speak of free
Network Administrator : The Leather Collection, Inc. : software we refer to
GnuPG Key ID          : 0x93B746BE                   : freedom, not price.