Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#99 bogofilter crashes upon processing dir with spam mails

closed
nobody
None
5
2006-11-19
2006-11-14
Michael Gerdau
No

Hi !

I can reliably crash the current stable release
(1.1.1) as downloaded today from the sf projectpage.

Here is how I can reproduce it:
The attached tarball contains a dir named spamtest
holding 26 msgs. I have it untarred in my home dir
and then issue

bogofilter -vvv -s -B ~/spamtest

which gives
Speicherzugriffsfehler
(presumably segmentation fault or something similar)

The msgs
1163324890.6552.8OJNc:2,S
1163325287.6552.wRGQC:2,S
1163325288.6552.H1bHp:2,S
seem to be the culprit -- any of these added to my
complete spamfolder (some 4000 msgs) makes bogofilter
crash. The attached dir is the smallest testcase I
had found, i.e. removing more msgs usually results in
bogofilter working as advertised.

I'm on SuSE 10.1, i686

mgd@Hamiller:~/ftp/mail/bogofilter> bogofilter -V
bogofilter version 1.1.1
Database: Sleepycat Software: Berkeley DB 4.3.29:
(April 23, 2006) AUTO-XA
Copyright (C) 2002-2003 Eric S. Raymond, Adrian Otto,
Gyepi Sam.
Copyright (C) 2002-2006 David Relson, Matthias
Andree, Greg Louis

bogofilter comes with ABSOLUTELY NO WARRANTY. This
is free software, and
you are welcome to redistribute it under the General
Public License. See
the COPYING file with the source distribution for
details.

Discussion

  • Michael Gerdau
    Michael Gerdau
    2006-11-14

    tarball with msgs to reproduce the crash

     
    Attachments
  • David Relson
    David Relson
    2006-11-14

    Logged In: YES
    user_id=30510

    Michael,

    You appear to have encountered a known, patched parsing
    problem that only affects bogofilter on a 64 bit processor.
    What hardware are you running?

    There are two ways to get the fix. You can update from CVS
    or you can download the lexer patch file, i.e.
    patch.1107.lexer_v3_l.txt, and patch your source. Once you
    have updated source, build bogofilter, test, and let me know
    what happens.

    Thank you.

    David

     
  • Michael Gerdau
    Michael Gerdau
    2006-11-14

    Logged In: YES
    user_id=841025

    I'm _not_ on a 64 bit processor but am using a P4 (that's why I wrote i686 in my initial report).

    It is a genuine Mobile Intel Pentium 4, cpu family 15, model 2, stepping 9

    Extracted the source from the src rpm and applied patch.1107.lexer_v3_l.txt.
    Rebuild the src rpm with the patched source and issued
    rpmbuild --rebuild bogofilter-1.1.1-2.src.rpm --target=i686

    This gives
    if
    gcc -DHAVE_CONFIG_H -I. -I. -I. -iquote../gnugetopt -iquote../trio -I. -I../gsl/specfunc -I.. -DBOGOFILTER -O2 -g -m32 -march=i686 -mtune=i686 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -Wpointer-arith -ggdb -Wall -W -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Wbad-function-cast -Wcast-qual -Wcast-align -Wwrite-strings -Waggregate-return -Wmissing-declarations -Wmissing-format-attribute -Wnested-externs -fno-common -Wchar-subscripts -Wcomment -Wimplicit -Wsequence-point -Wreturn-type -Wno-system-headers -Wformat -MT
    lexer_v3.o -MD -MP -MF ".deps/lexer_v3.Tpo" -c -o lexer_v3.o lexer_v3.c; \ then mv -f ".deps/lexer_v3.Tpo" ".deps/lexer_v3.Po"; else rm -f ".deps/lexer_v3.Tpo"; exit 1; fi
    lexer_v3.c: In function ‘yylex’:
    ./lexer_v3.l:325: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
    lexer_v3.c: At top level:
    lexer_v3.c:3325: warning: no previous prototype for ‘yyget_in’
    lexer_v3.c:3333: warning: no previous prototype for ‘yyget_out’
    lexer_v3.c:3341: warning: no previous prototype for ‘yyget_leng’
    lexer_v3.c:3350: warning: no previous prototype for ‘yyget_text’
    lexer_v3.c:3364: warning: no previous prototype for ‘yyset_in’
    lexer_v3.c:3369: warning: no previous prototype for ‘yyset_out’
    lexer_v3.c:3374: warning: no previous prototype for ‘yyget_debug’
    lexer_v3.c:3379: warning: no previous prototype for ‘yyset_debug’
    lexer_v3.c:3393: warning: no previous prototype for ‘yylex_destroy’

    and later during link:
    gcc -DBOGOFILTER -O2 -g -m32 -march=i686 -mtune=i686 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -Wpointer-arith -ggdb -Wall -W -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Wbad-function-cast -Wcast-qual -Wcast-align -Wwrite-strings -Waggregate-return -Wmissing-declarations -Wmissing-format-attribute -Wnested-externs -fno-common -Wchar-subscripts -Wcomment -Wimplicit -Wsequence-point -Wreturn-type -Wno-system-headers -Wformat -o
    bogofilter bogofilter.o main.o libbogofilter.a strlcpy.o strlcat.o /usr/lib/libdb.so libbf_gsl.a -lm
    libbogofilter.a(lexer.o): In function `yyinit':
    /usr/src/packages/BUILD/bogofilter-1.1.1/src/lexer.c:294: undefined reference to `yylineno'
    libbogofilter.a(lexer.o): In function `lexer_display_buffer':
    /usr/src/packages/BUILD/bogofilter-1.1.1/src/lexer.c:73: undefined reference to `yylineno'
    libbogofilter.a(lexer.o): In function `yy_get_new_line':
    /usr/src/packages/BUILD/bogofilter-1.1.1/src/lexer.c:114: undefined reference to `yylineno'
    libbogofilter.a(lexer.o): In function `skip_folded_line':
    /usr/src/packages/BUILD/bogofilter-1.1.1/src/lexer.c:262: undefined reference to `yylineno'
    collect2: ld returned 1 exit status
    make[3]: *** [bogofilter] Fehler 1

    flex --version returns "flex 2.5.31"

    I have manually verified that the patch had been correctly applied.
    Anything I have forgotten ?

    Best,
    Michael

     
  • Michael Gerdau
    Michael Gerdau
    2006-11-16

    Logged In: YES
    user_id=841025
    Originator: YES

    Hi !

    I have further reduced my testcase.
    It now only holds 5 msgs. After expanding the tarball

    bogofilter -vvv -s -B ~/spamtest

    shows the crash on my machine.

    Best,
    Michael

     
  • Michael Gerdau
    Michael Gerdau
    2006-11-16

    Minimized testcase with 5 msgs

     
  • David Relson
    David Relson
    2006-11-18

    Logged In: YES
    user_id=30510
    Originator: NO

    Interesting... I can trim your directory to the message 1163324890.6552.8OJNc:2,S which is 16,380 char and reproduce the problem. The message has a line that's 14,534 chars long and seems to be the root of the problem. The line can be trimmed by 800 characters, but not 900, and the problem still happens. More later ...

     
  • David Relson
    David Relson
    2006-11-19

    Logged In: YES
    user_id=30510
    Originator: NO

    Michael,

    The crash is caused by a buffer overflow in flex's support code. A bug report has been entered in flex's bug tracking system. I can send you the evidence if you want it.

    David

     
  • David Relson
    David Relson
    2006-11-19

    • status: open --> closed
     
  • Michael Gerdau
    Michael Gerdau
    2006-11-19

    Logged In: YES
    user_id=841025
    Originator: YES

    Hi David,

    I did suspect something like a buffer overflow because the specifics
    of the message 1163324890.6552.8OJNc:2,S did suggest something along
    that line but since I know next to nothing about the internals of
    bogofilter I was I no position to raise such a claim.
    [read: I'm easily convinced you can proove it :-]

    With a bug report entered for flex I'm confident the problem will be
    gone in foreseeable time. For the interim I have a workaround in
    that I delete the offending msgs.

    Thanks for tracking that down, best wishes,
    Michael