#116 Mail crashes bogofilter

v1.0_(example)
open
crash (1)
1
2014-07-17
2014-02-23
No

Running attached mail through bogofilter crashes, see attachments. This is the one and only message in >>100.000 I have which causes this issue.

glibc detected bogofilter: realloc(): invalid next size:
0x00000000018b6ea0 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71e16)[0x7fc85c6bee16]
/lib/libc.so.6(+0x77a2c)[0x7fc85c6c4a2c]
/lib/libc.so.6(realloc+0xf0)[0x7fc85c6c4d40]
bogofilter[0x40b34e]
bogofilter[0x410a29]
bogofilter[0x406305]
bogofilter[0x402cf5]
bogofilter[0x404c99]
bogofilter[0x402ebc]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7fc85c66bc8d]
bogofilter[0x402a09]
======= Memory map: ========
00400000-00443000 r-xp 00000000 09:02
3629819 /home/pi/bin/Bogo/bin/bogofilter
00643000-00647000 rw-p 00043000 09:02
3629819 /home/pi/bin/Bogo/bin/bogofilter
00647000-0064a000 rw-p 00000000 00:00 0
01889000-018d0000 rw-p 00000000 00:00
0 [heap]
7fc854000000-7fc854021000 rw-p 00000000 00:00 0
7fc854021000-7fc858000000 ---p 00000000 00:00 0
7fc85b7e0000-7fc85b7f6000 r-xp 00000000 09:02
3514370 /lib/libgcc_s.so.1
7fc85b7f6000-7fc85b9f5000 ---p 00016000 09:02
3514370 /lib/libgcc_s.so.1
7fc85b9f5000-7fc85b9f6000 rw-p 00015000 09:02
3514370 /lib/libgcc_s.so.1
7fc85b9f6000-7fc85b9f8000 r-xp 00000000 09:02
1393964 /usr/lib/gconv/ISO8859-1.so
7fc85b9f8000-7fc85bbf7000 ---p 00002000 09:02
1393964 /usr/lib/gconv/ISO8859-1.so
7fc85bbf7000-7fc85bbf8000 r--p 00001000 09:02
1393964 /usr/lib/gconv/ISO8859-1.so
7fc85bbf8000-7fc85bbf9000 rw-p 00002000 09:02
1393964 /usr/lib/gconv/ISO8859-1.so
7fc85bbf9000-7fc85bc05000 r-xp 00000000 09:02
3514422 /lib/libnss_files-2.11.3.so
7fc85bc05000-7fc85be04000 ---p 0000c000 09:02
3514422 /lib/libnss_files-2.11.3.so
7fc85be04000-7fc85be05000 r--p 0000b000 09:02
3514422 /lib/libnss_files-2.11.3.so
7fc85be05000-7fc85be06000 rw-p 0000c000 09:02
3514422 /lib/libnss_files-2.11.3.so
7fc85be06000-7fc85be10000 r-xp 00000000 09:02
3516413 /lib/libnss_nis-2.11.3.so
7fc85be10000-7fc85c00f000 ---p 0000a000 09:02
3516413 /lib/libnss_nis-2.11.3.so
7fc85c00f000-7fc85c010000 r--p 00009000 09:02
3516413 /lib/libnss_nis-2.11.3.so
7fc85c010000-7fc85c011000 rw-p 0000a000 09:02
3516413 /lib/libnss_nis-2.11.3.so
7fc85c011000-7fc85c026000 r-xp 00000000 09:02
3516403 /lib/libnsl-2.11.3.so
7fc85c026000-7fc85c225000 ---p 00015000 09:02
3516403 /lib/libnsl-2.11.3.so
7fc85c225000-7fc85c226000 r--p 00014000 09:02
3516403 /lib/libnsl-2.11.3.so
7fc85c226000-7fc85c227000 rw-p 00015000 09:02
3516403 /lib/libnsl-2.11.3.so
7fc85c227000-7fc85c229000 rw-p 00000000 00:00 0
7fc85c229000-7fc85c230000 r-xp 00000000 09:02
3516398 /lib/libnss_compat-2.11.3.so
7fc85c230000-7fc85c42f000 ---p 00007000 09:02
3516398 /lib/libnss_compat-2.11.3.so
7fc85c42f000-7fc85c430000 r--p 00006000 09:02
3516398 /lib/libnss_compat-2.11.3.so
7fc85c430000-7fc85c431000 rw-p 00007000 09:02
3516398 /lib/libnss_compat-2.11.3.so
7fc85c431000-7fc85c448000 r-xp 00000000 09:02
3516394 /lib/libpthread-2.11.3.so
7fc85c448000-7fc85c647000 ---p 00017000 09:02
3516394 /lib/libpthread-2.11.3.so
7fc85c647000-7fc85c648000 r--p 00016000 09:02
3516394 /lib/libpthread-2.11.3.so
7fc85c648000-7fc85c649000 rw-p 00017000 09:02
3516394 /lib/libpthread-2.11.3.so
7fc85c649000-7fc85c64d000 rw-p 00000000 00:00 0
7fc85c64d000-7fc85c7a6000 r-xp 00000000 09:02
3516399 /lib/libc-2.11.3.so
7fc85c7a6000-7fc85c9a5000 ---p 00159000 09:02
3516399 /lib/libc-2.11.3.so
7fc85c9a5000-7fc85c9a9000 r--p 00158000 09:02
3516399 /lib/libc-2.11.3.so
7fc85c9a9000-7fc85c9aa000 rw-p 0015c000 09:02
3516399 /lib/libc-2.11.3.so
7fc85c9aa000-7fc85c9af000 rw-p 00000000 00:00 0
7fc85c9af000-7fc85ca2f000 r-xp 00000000 09:02
3516412 /lib/libm-2.11.3.so
7fc85ca2f000-7fc85cc2f000 ---p 00080000 09:02
3516412 /lib/libm-2.11.3.so
7fc85cc2f000-7fc85cc30000 r--p 00080000 09:02
3516412 /lib/libm-2.11.3.so
7fc85cc30000-7fc85cc31000 rw-p 00081000 09:02
3516412 /lib/libm-2.11.3.so
7fc85cc31000-7fc85cda7000 r-xp 00000000 09:02
1385361 /usr/lib/libdb-4.8.so
7fc85cda7000-7fc85cfa6000 ---p 00176000 09:02
1385361 /usr/lib/libdb-4.8.so
7fc85cfa6000-7fc85cfab000 rw-p 00175000 09:02
1385361 /usr/lib/libdb-4.8.so
7fc85cfab000-7fc85cfc9000 r-xp 00000000 09:02
3516395 /lib/ld-2.11.3.so
7fc85d140000-7fc85d147000 r--s 00000000 09:02
1394001 /usr/lib/gconv/gconv-modules.cache
7fc85d147000-7fc85d1b8000 rw-p 00000000 00:00 0
7fc85d1c4000-7fc85d1c8000 rw-p 00000000 00:00 0
7fc85d1c8000-7fc85d1c9000 r--p 0001d000 09:02
3516395 /lib/ld-2.11.3.so
7fc85d1c9000-7fc85d1ca000 rw-p 0001e000 09:02
3516395 /lib/ld-2.11.3.so
7fc85d1ca000-7fc85d1cb000 rw-p 00000000 00:00 0
7fffea790000-7fffea7a5000 rw-p 00000000 00:00
0 [stack]
7fffea7ff000-7fffea800000 r-xp 00000000 00:00
0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00
0 [vsyscall]

1 Attachments

Related

Bugs: #116

Discussion

  • Hi Boris,

    sorry to see there is trouble. While I cannot confirm the crash at this time, I see that valgrind complains about three issues in the lexer, so there is some hope I can debug this. It may take me a couple of days to debug, though.

    Thank you for providing the relevant input!

    Cheers,
    Matthias

     
    • assigned_to: Matthias Andree
     
  • Boris, taking a first glance, I need to know a bit more information:

    • which bogofilter version are you using?

    • did your lexer_v3.c file get regenerated on your computer by running flex, or was the file used unaltered from the tarball? If it was generated on your computer, I'd need to know the flex version ("flex --version" prints it; if flex is not installed on your computer, that is fine).

    • how have you installed it, which ./configure options did you use, what Linux variant and version are you on?

    • do you happen to have sufficient debug information in bogofilter so that addr2line can translate the addresses you got from glibc to code lines?

    • what options do you use on bogofilter's command line, if any?

     
  • bogofilter version 1.2.4
    Database: Berkeley DB 4.8.30: (April 9, 2010) AUTO-XA

    I did compile bogofilter myself, though I do not see that that uses flex.

    env ./configure --prefix=$WHERE --sysconfdir=$WHERE
    $WHERE is simple the local path (user installation)

    Linux 2.6.32-5-amd64 #1 SMP Sun Sep 23 10:07:46 UTC 2012 x86_64 GNU/Linux

    The error comes without any command line options, yet there is ~/.bogofilter.cf:
    ham_cutoff=0.0
    spamicity_tags=Spam, Ham
    spamicity_formats=%0.3f, %0.3f
    header_format=%h: %c, spamicity=%p, version=%v
    db_cachesize=3
    timestamp=0
    robx=0.499
    min_dev=0.3
    robs=0.1
    spam_cutoff=0.5
    min-token-len=1
    multi-token-count=1

    HTH, please let me know if there is more I can tell you.

    pi

     
  • Boris, I cannot reproduce the issue. All I can trigger is a lexer read of uninitialized memory.

    If you had built bogofilter from an official tarball, it should not run flex as part of the build because there was a prebuilt lexer_v3.c in the tarball.

    I would require a proper backtrace; you might also try installing an up-to-date valgrind and run bogofilter or bogolexer (if that suffices to trigger the bug) on the problematic mail. valgrind should print a backtrace. You might also try filtering your error messages from glib through addr2line and see if that fills in the source code lines.

    Make sure you do not use "make install-strip".

    Without a proper backtrace, I will unfortunately be unable to debug this.

     
  • You will need an executable compiled and installed with at least line number information. That should be the default if you build yourself.

    Then addr2line -a -f -p -e /path/to/bogofilter 0x40b34e
    should print the line number for the given address - try this with all bogofilter and libc 0x... addresses from your backtrace, if that gives you useful information, paste the info and you're done. You can give multiple addresses on the command line.

    For a different issue I provoked I get this output, for example:

    $ addr2line -p -a -f -e src/bogolexer 0x40592C 0x40A7BD
    0x000000000040592c: yylex at /path/to/src/lexer_v3.c:2469
    0x000000000040a7bd: parse_new_token at /path/to/src/token.c:206

    If you get question marks, that does not help...

     
    • addr2line does not know -a and -p, yet:

      $ addr2line -f -e ~/bin/bogofilter 0x40b34e 0x410a29 0x406305 0x402cf5
      0x404c99 0x402ebc 0x402a09
      yy_get_next_buffer
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/lexer_v3.c:3185
      parse_new_token
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/token.c:206
      collect_words
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/collect.c:50
      bogofilter
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/bogofilter.c:99
      bogomain
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/bogomain.c:69
      main
      /home/pi/build-bogofilter/bogofilter-1.2.4/src/main.c:33
      _start
      ??:0

      Matthias Andree wrote:

      You will need an executable compiled and installed with at least
      line number information. That should be the default if you build
      yourself.

      Then addr2line -a -f -p -e /path/to/bogofilter 0x40b34e
      should print the line number for the given address - try this with
      all bogofilter and libc 0x... addresses from your backtrace, if that
      gives you useful information, paste the info and you're done. You
      can give multiple addresses on the command line.

      For a different issue I provoked I get this output, for example:

      $ addr2line -p -a -f -e src/bogolexer 0x40592C 0x40A7BD
      0x000000000040592c: yylex at /path/to/src/lexer_v3.c:2469
      0x000000000040a7bd: parse_new_token at /path/to/src/token.c:206

      If you get question marks, that does /not/ help...


      [bugs:#116] http://sourceforge.net/p/bogofilter/bugs/116/ Mail
      crashes bogofilter

      Status: open
      Labels: crash
      Created: Sun Feb 23, 2014 04:36 PM UTC by Boris 'pi' Piwinger
      Last Updated: Mon Feb 24, 2014 10:11 PM UTC
      Owner: Matthias Andree

      Running attached mail through bogofilter crashes, see attachments.
      This is the one and only message in >>100.000 I have which causes
      this issue.

      /glibc detected / bogofilter: realloc(): invalid next size:
      0x00000000018b6ea0 ***
      ======= Backtrace: =========
      /lib/libc.so.6(+0x71e16)[0x7fc85c6bee16]
      /lib/libc.so.6(+0x77a2c)[0x7fc85c6c4a2c]
      /lib/libc.so.6(realloc+0xf0)[0x7fc85c6c4d40]
      bogofilter[0x40b34e]
      bogofilter[0x410a29]
      bogofilter[0x406305]
      bogofilter[0x402cf5]
      bogofilter[0x404c99]
      bogofilter[0x402ebc]
      /lib/libc.so.6(__libc_start_main+0xfd)[0x7fc85c66bc8d]
      bogofilter[0x402a09]
      ======= Memory map: ========
      00400000-00443000 r-xp 00000000 09:02
      3629819 /home/pi/bin/Bogo/bin/bogofilter
      00643000-00647000 rw-p 00043000 09:02
      3629819 /home/pi/bin/Bogo/bin/bogofilter
      00647000-0064a000 rw-p 00000000 00:00 0
      01889000-018d0000 rw-p 00000000 00:00
      0 [heap]
      7fc854000000-7fc854021000 rw-p 00000000 00:00 0
      7fc854021000-7fc858000000 ---p 00000000 00:00 0
      7fc85b7e0000-7fc85b7f6000 r-xp 00000000 09:02
      3514370 /lib/libgcc_s.so.1
      7fc85b7f6000-7fc85b9f5000 ---p 00016000 09:02
      3514370 /lib/libgcc_s.so.1
      7fc85b9f5000-7fc85b9f6000 rw-p 00015000 09:02
      3514370 /lib/libgcc_s.so.1
      7fc85b9f6000-7fc85b9f8000 r-xp 00000000 09:02
      1393964 /usr/lib/gconv/ISO8859-1.so
      7fc85b9f8000-7fc85bbf7000 ---p 00002000 09:02
      1393964 /usr/lib/gconv/ISO8859-1.so
      7fc85bbf7000-7fc85bbf8000 r--p 00001000 09:02
      1393964 /usr/lib/gconv/ISO8859-1.so
      7fc85bbf8000-7fc85bbf9000 rw-p 00002000 09:02
      1393964 /usr/lib/gconv/ISO8859-1.so
      7fc85bbf9000-7fc85bc05000 r-xp 00000000 09:02
      3514422 /lib/libnss_files-2.11.3.so
      7fc85bc05000-7fc85be04000 ---p 0000c000 09:02
      3514422 /lib/libnss_files-2.11.3.so
      7fc85be04000-7fc85be05000 r--p 0000b000 09:02
      3514422 /lib/libnss_files-2.11.3.so
      7fc85be05000-7fc85be06000 rw-p 0000c000 09:02
      3514422 /lib/libnss_files-2.11.3.so
      7fc85be06000-7fc85be10000 r-xp 00000000 09:02
      3516413 /lib/libnss_nis-2.11.3.so
      7fc85be10000-7fc85c00f000 ---p 0000a000 09:02
      3516413 /lib/libnss_nis-2.11.3.so
      7fc85c00f000-7fc85c010000 r--p 00009000 09:02
      3516413 /lib/libnss_nis-2.11.3.so
      7fc85c010000-7fc85c011000 rw-p 0000a000 09:02
      3516413 /lib/libnss_nis-2.11.3.so
      7fc85c011000-7fc85c026000 r-xp 00000000 09:02
      3516403 /lib/libnsl-2.11.3.so
      7fc85c026000-7fc85c225000 ---p 00015000 09:02
      3516403 /lib/libnsl-2.11.3.so
      7fc85c225000-7fc85c226000 r--p 00014000 09:02
      3516403 /lib/libnsl-2.11.3.so
      7fc85c226000-7fc85c227000 rw-p 00015000 09:02
      3516403 /lib/libnsl-2.11.3.so
      7fc85c227000-7fc85c229000 rw-p 00000000 00:00 0
      7fc85c229000-7fc85c230000 r-xp 00000000 09:02
      3516398 /lib/libnss_compat-2.11.3.so
      7fc85c230000-7fc85c42f000 ---p 00007000 09:02
      3516398 /lib/libnss_compat-2.11.3.so
      7fc85c42f000-7fc85c430000 r--p 00006000 09:02
      3516398 /lib/libnss_compat-2.11.3.so
      7fc85c430000-7fc85c431000 rw-p 00007000 09:02
      3516398 /lib/libnss_compat-2.11.3.so
      7fc85c431000-7fc85c448000 r-xp 00000000 09:02
      3516394 /lib/libpthread-2.11.3.so
      7fc85c448000-7fc85c647000 ---p 00017000 09:02
      3516394 /lib/libpthread-2.11.3.so
      7fc85c647000-7fc85c648000 r--p 00016000 09:02
      3516394 /lib/libpthread-2.11.3.so
      7fc85c648000-7fc85c649000 rw-p 00017000 09:02
      3516394 /lib/libpthread-2.11.3.so
      7fc85c649000-7fc85c64d000 rw-p 00000000 00:00 0
      7fc85c64d000-7fc85c7a6000 r-xp 00000000 09:02
      3516399 /lib/libc-2.11.3.so
      7fc85c7a6000-7fc85c9a5000 ---p 00159000 09:02
      3516399 /lib/libc-2.11.3.so
      7fc85c9a5000-7fc85c9a9000 r--p 00158000 09:02
      3516399 /lib/libc-2.11.3.so
      7fc85c9a9000-7fc85c9aa000 rw-p 0015c000 09:02
      3516399 /lib/libc-2.11.3.so
      7fc85c9aa000-7fc85c9af000 rw-p 00000000 00:00 0
      7fc85c9af000-7fc85ca2f000 r-xp 00000000 09:02
      3516412 /lib/libm-2.11.3.so
      7fc85ca2f000-7fc85cc2f000 ---p 00080000 09:02
      3516412 /lib/libm-2.11.3.so
      7fc85cc2f000-7fc85cc30000 r--p 00080000 09:02
      3516412 /lib/libm-2.11.3.so
      7fc85cc30000-7fc85cc31000 rw-p 00081000 09:02
      3516412 /lib/libm-2.11.3.so
      7fc85cc31000-7fc85cda7000 r-xp 00000000 09:02
      1385361 /usr/lib/libdb-4.8.so
      7fc85cda7000-7fc85cfa6000 ---p 00176000 09:02
      1385361 /usr/lib/libdb-4.8.so
      7fc85cfa6000-7fc85cfab000 rw-p 00175000 09:02
      1385361 /usr/lib/libdb-4.8.so
      7fc85cfab000-7fc85cfc9000 r-xp 00000000 09:02
      3516395 /lib/ld-2.11.3.so
      7fc85d140000-7fc85d147000 r--s 00000000 09:02
      1394001 /usr/lib/gconv/gconv-modules.cache
      7fc85d147000-7fc85d1b8000 rw-p 00000000 00:00 0
      7fc85d1c4000-7fc85d1c8000 rw-p 00000000 00:00 0
      7fc85d1c8000-7fc85d1c9000 r--p 0001d000 09:02
      3516395 /lib/ld-2.11.3.so
      7fc85d1c9000-7fc85d1ca000 rw-p 0001e000 09:02
      3516395 /lib/ld-2.11.3.so
      7fc85d1ca000-7fc85d1cb000 rw-p 00000000 00:00 0
      7fffea790000-7fffea7a5000 rw-p 00000000 00:00
      0 [stack]
      7fffea7ff000-7fffea800000 r-xp 00000000 00:00
      0 [vdso]
      ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00
      0 [vsyscall]


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/bogofilter/bugs/116/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Bugs: #116

  • You can also try:

    sed 's/.*0x/0x/' | addr2line -p -a -f -e /home/pi/bin/Bogo/bin/bogofilter

    Then paste your backtrace, press enter, and press Ctrl-D to send bogolexer an end-of-input marker so it terminates. The output should be as in my previous comment.

     
  • Thanks, much better.

    Unfortunately you might be hitting one of the nasty corner cases in the current bogofilter tokenizer, and fixing those may require my rewriting major parts of the lexing (most importantly, separate concerns so we can parse input without ever pushing back nontrivial amounts of data or without rejecting rules). This isn't going to start in the next two weeks though.

     
  • Thanks for your help. In case it helps I have another testcase which also crashes.