#2711 Squirrelmail timeouts on email with bad headers

Produces PHP errors
closed-fixed
nobody
Folders (317)
5
2010-06-21
2010-04-14
Anonymous
No

If a user receives an email with bad headers syntax (typically a UTF8 character directly in Subject, From or To header without MIME encoded-word syntax (RFC 2047)), SquirrelMail 1.4.20 is unable to display the list of emails in the folder and it usually timeouts with php error "execution time exceeded 30s". I know these emails are against RFC standards, but these simply arrive sometimes, and it shouldn't block the user's entire mailbox. Squirrelmail 1.4.20-RC2 handles these improperly formatted email correctly, with no timeouts or slow-downs.

A typical email that causes this problem is attached.

Discussion

<< < 1 2 3 > >> (Page 2 of 3)
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-04-21

    Here is the output

    Array
    (
    [0] => pass
    [1] => auto
    [2] => wchar
    [3] => byte2be
    [4] => byte2le
    [5] => byte4be
    [6] => byte4le
    [7] => BASE64
    [8] => UUENCODE
    [9] => HTML-ENTITIES
    [10] => Quoted-Printable
    [11] => 7bit
    [12] => 8bit
    [13] => UCS-4
    [14] => UCS-4BE
    [15] => UCS-4LE
    [16] => UCS-2
    [17] => UCS-2BE
    [18] => UCS-2LE
    [19] => UTF-32
    [20] => UTF-32BE
    [21] => UTF-32LE
    [22] => UTF-16
    [23] => UTF-16BE
    [24] => UTF-16LE
    [25] => UTF-8
    [26] => UTF-7
    [27] => UTF7-IMAP
    [28] => ASCII
    [29] => EUC-JP
    [30] => SJIS
    [31] => eucJP-win
    [32] => SJIS-win
    [33] => CP51932
    [34] => JIS
    [35] => ISO-2022-JP
    [36] => ISO-2022-JP-MS
    [37] => Windows-1252
    [38] => Windows-1254
    [39] => ISO-8859-1
    [40] => ISO-8859-2
    [41] => ISO-8859-3
    [42] => ISO-8859-4
    [43] => ISO-8859-5
    [44] => ISO-8859-6
    [45] => ISO-8859-7
    [46] => ISO-8859-8
    [47] => ISO-8859-9
    [48] => ISO-8859-10
    [49] => ISO-8859-13
    [50] => ISO-8859-14
    [51] => ISO-8859-15
    [52] => ISO-8859-16
    [53] => EUC-CN
    [54] => CP936
    [55] => HZ
    [56] => EUC-TW
    [57] => BIG-5
    [58] => EUC-KR
    [59] => UHC
    [60] => ISO-2022-KR
    [61] => Windows-1251
    [62] => CP866
    [63] => KOI8-R
    [64] => KOI8-U
    [65] => ArmSCII-8
    [66] => CP850
    )

    I noticed new errors in the log, after I changed strtolower to mb_strtolower:

    [Thu Apr 22 00:09:38 2010] [error] [client 85.207.47.189] PHP Notice: Undefined variable: charset in /var/www/squirrelmail-1.4.20/functions/strings.php on line 1246, referer: https://webmail.jh-inst.cas.cz/stable/src/left_main.php
    [Thu Apr 22 00:09:38 2010] [error] [client 85.207.47.189] PHP Warning: mb_strtolower(): Unknown encoding "(null)" in /var/www/squirrelmail-1.4.20/functions/strings.php on line 1246, referer: https://webmail.jh-inst.cas.cz/stable/src/left_main.php

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-21

    Yes, you should revert your "fix", as it is broken.

    You can remove the "exit;" part of that debugging code and try again and verify that, with the patch that I sent the link to earlier, this list is only printed once before the message listing (or timeout). That is, strtolower() should only get called once, and the list you sent shows nothing unusual, so there is no reason why it would time out on this code.

    After that, adding step-by-step debug lines in sm_truncate_string() as I suggested earlier is the only other thing I can think of.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-04-22

    OK. I reverted strings.php to original, applied the patch from http://article.gmane.org/gmane.mail.squirrelmail.user/37485. Now I tried to analyze what's going on when the function sq_mb_list_encodings() (line 1175) is called.

    First thing I noticed is that the function is called once on the beginning of the right frame, and then twice for each email on the page between the buttons bar and the message listing, see this: http://img693.imageshack.us/img693/3884/screenshot20100422at121u.png

    Is it OK? Now I continue debugging the code.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-04-22

    The function sq_mb_list_encodings() is called in an infinite loop if there is an email with utf-8 encoded characters(s) in Subject, From or To headers. See this, each "*" represents one call of sq_mb_list_encodings().

    http://img153.imageshack.us/img153/3203/screenshot20100422at202.png

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-27

    Sorry for the delay. We really appreciate your help.

    As I said, I cannot reproduce the same thing you are seeing. Is this caused by the same message source that you shared previously?

    Where are you outputting the "*"? It would help if you could trace execution so we can see where the endless loop is being triggered. The most likely place it is happening is in sm_truncate_string(), so putting in some simple echo statements there could help.

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-27

    If it's the while loop in sm_truncate_string(), then this debug code could help (around line 190). Note that I've added to echo statements. All other code is unchanged:

    $entity_pos = -1;
    echo "<hr />STRING: $string<br />";
    while (($entity_pos = sq_strpos($string, '&', $entity_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {
    echo " ENTITY: $entity_pos -- $entity_end_pos<br />";
    $adjusted_max_chars += $entity_end_pos - $entity_pos;
    }

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-27

    OK, can't post code here without having spaces get munged. Based on 1.4.21[SVN] with the performance patch previously applied, the line numbers and code to add are:

    line 191
    echo "<hr />STRING: $string<br />";

    The line immediately AFTER that added line is this one (just for reference):
    while (($entity_pos = sq_strpos($string, '&', $entity_pos + 1)) !== FALSE

    The second one is:

    line 196
    echo " ENTITY: $entity_pos -- $entity_end_pos<br />";

    The line immediately AFTER that added line is this one (just for reference):
    $adjusted_max_chars += $entity_end_pos - $entity_pos;

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-27

    Now, if that is where the endless loop is happening, this would point to a problem in sq_strpos(). The output from the previous echo statements can be very helpful, but it may be more informative to see what sq_strpos() is doing with these poorly formed headers. I will next suggest some output to add to the sq_strpos() function. Line numbers assume that the last echo statements have not been removed.

    line 909
    echo "<hr />AUTO CHARSET SET TO $charset<br />";

    The line immediately BEFORE that added line is this one (just for reference):
    $charset = $sq_string_func_auto_charset;

    line 916
    echo "<hr />SQ_STRPOS CHARSET: $charset<br />";
    echo "SQ_STRPOS NEEDLE: $needle<br />";
    echo "SQ_STRPOS OFFSET: $offset<br />";
    echo "SQ_STRPOS HAYSTACK: $haystack<br />";

    The line immediately BEFORE that added line is this one (just for reference):
    $charset = strtolower($charset);

    line 945
    echo "NOT USING MBSTRING<br />";

    The line immediately BEFORE that added line is this one (just for reference):
    return mb_strpos($haystack, $needle, $offset, $charset);

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-04-27

    Finally, you can also try to see what mbstring is doing with these bad strings. Take that last line (about line 948) and change it from this:

    return mb_strpos($haystack, $needle, $offset, $charset);

    To this:

    { $ret = mb_strpos($haystack, $needle, $offset, $charset); echo "=== MBSTRING output is $ret<br />"; return $ret; }

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-04-28

    Hello,

    thank you. Yes, I am testing it with an empty folder including just one single email - the one I posted previously.

    With changes you suggested at Date: 2010-04-27 19:48:56 UTC, it looks like this

    http://img168.imageshack.us/img168/6752/screenshot20100428at120.png

    The line "ENTITY: 72 -- 76" is repeated mabe several thousand times, the bottom of the page looks like this - the message list is not displayed, script ends with php error execution time exceeded.

    http://img213.imageshack.us/img213/2503/screenshot20100428at122.png

    With your suggestions posted on Date: 2010-04-27 19:54:57 GMT I am getting this

    http://img97.imageshack.us/img97/7489/screenshot20100428at123.png

    the end of the page looks like this, script ends with php error execution time exceeded.

    http://img7.imageshack.us/img7/7489/screenshot20100428at123.png

    And finally, your suggestion posted on Date: 2010-04-27 20:05:43 GMT brings this result

    http://img11.imageshack.us/img11/5537/screenshot20100428at125.png

    the end of the page looks like this, script ends with php error execution time exceeded.

    http://img293.imageshack.us/img293/5537/screenshot20100428at125.png

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-06

    Unfortunatelly I am unable to analyze the results, is there anything I can help further with?

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-07

    I can reproduce it, too. I have squirrelmail version 1.4.20, php with mbstrings extension 5.2.13, the message says:
    Fatal error: Maximum execution time of 30 seconds exceeded in /usr/share/squirrelmail/functions/strings.php on line 1176
    what is
    if (! function_exists('mb_internal_encoding'))
    in function sq_mb_list_encodings.
    It's caused by only one non-ascii character in
    Subject: Newsletter n°11 du Dictionnaire de la Zone
    Only messages preceding this mail are displayed.

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-10

    Thanks for your continued help. Sorry for the delay (I am currently very busy).

    It would be helpful to see the output of all the patches/changes I suggested rolled into one (not separately), and especially to see all of the output up until it starts repeating/looping.

    That said, what you provided was certainly helpful. It is unclear to me why your mb_strpos() seems to be looking backward in the string - when given the offset of 73, it finds an entity at offset 72 (but it'd be helpful to see the combined output to verify this).

    One suggestion I can make that may or may not help would be to make this change: once again, in functions/strings.php, in the sm_truncate_string() function, find the while() loop. It looks like this (spacing probably munged):

    $entity_pos = -1;
    while (($entity_pos = sq_strpos($string, '&', $entity_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {

    Try making two changes, one to the first line and one to the second line:

    $entity_pos = $entity_end_pos = -1;
    while (($entity_pos = sq_strpos($string, '&', $entity_end_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {

    Does that help? If not, if you can make all of the debugging changes suggested below and show all of the output until the endless looping starts, that'd be appreciated.

    Thank you.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-13

    Thank you. The change you suggested helped, now it works.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-14

    This helped me (cf comment from: 2010-05-07 11:57:11 CEST) as well. Thank you very much, please consider including this patch into stable tree.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-18

    It looks like yet another problem occurs, although much less severe, there is warning displayed, but mailbox processing continues. Some kind of messages with faulty headers result in
    Warning: mb_strpos() [function.mb-strpos]: Offset not contained in string in /usr/share/squirrelmail/functions/strings.php on line 942
    the line is a return command in function sq_strpos:
    if (in_array($charset, sq_mb_list_encodings()))
    return mb_strpos($haystack, $needle, $offset, $charset);

    squirrelmail-1.4.20

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-22

    Sorry for the delay. Notifications for this tracker have been falling in my spam folder unfortunately. Regarding the PHP notices, again, I may need to see the full output from the suggested debugging changes below, but here is a suggestion that may fix the issue. Once again, in functions/strings.php, in the sm_truncate_string() function, find that same while() loop and change it to this (adding an extra clause on the beginning):

    while ($entity_end_pos < $actual_strlen
    && ($entity_pos = sq_strpos($string, '&', $entity_end_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)

    Thanks again for your help!

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-24

    Adding the line helped me, thanks, there is no more php warning. How would you expect full output? There were no more messages neither on the screen nor in the log, and php has error_reporting = E_ALL.
    Just to be sure (or to help to anyone else), the part in strings.php/sm_truncate_string() now looks like:

    if ($html_entities_as_chars)
    {
    while ($entity_end_pos < $actual_strlen
    && ($entity_pos = sq_strpos($string, '&', $entity_end_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {
    $adjusted_max_chars += $entity_end_pos - $entity_pos;
    }
    if ($actual_strlen <= $adjusted_max_chars)
    return $string;
    }

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-25

    Actually, I the first line in the while () loop might need to be this:

    while ($entity_end_pos + 1 < $actual_strlen

    (Note the added "+ 1")

    "Full output" means all the output produced by the debug code that I suggested earlier in this tracker. At the very least, can you please send the subject line for the message that causes this warning? It'd be best if you could simply export the full message source and send that, so nothing gets corrupted. I have not been able to reproduce the warning, and it'd be nice to be able to test this fix before I add it to SquirrelMail.

    Thanks for your help.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-27

    Thank you for your support. I has just added the "+ 1" and didn't notice anything wrong. When I put the old code back, the loop is in this part (detected by the echo lines suggested):
    while (($entity_pos = sq_strpos($string, '&', $entity_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {
    echo "5<br />";
    $adjusted_max_chars += $entity_end_pos - $entity_pos;
    }

    squirrelmail-1.4.20, language cs_CZ, charset utf8.
    The malformed headers often come from phpBB2.

    A minimum set of headers looks like:
    From tmp@domain.tld Sat May 1 20:24:26 2010
    Received: from minas.ics.cz (minas.ics.cz [1.2.4.40])
    by rosret.phil.cz (8.14.3/8.14.3) with ESMTP id o41IOP8k006601
    (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
    for <username@phil.cz>; Sat, 1 May 2010 20:24:26 +0200
    Received: from 81.mail-out.ovh.net (81.mail-out.ovh.net [8.9.182.117])
    by minas.ics.cz (8.13.8/8.13.8/SuSE Linux 0.8) with SMTP id o41IONMs018877
    for <username@phil.cz>; Sat, 1 May 2010 20:24:24 +0200
    Received: (qmail 21366 invoked by uid 0); 1 May 2010 17:57:51 -0000
    Received: from gw1.ovh.net (HELO 90plan.ovh.net) (2.251.189.201)
    by 81.mail-out.ovh.net with SMTP; 1 May 2010 17:21:07 -0000
    Received: by 90plan.ovh.net (Postfix, from userid 41989)
    id 2363F4508; Sat, 1 May 2010 19:20:59 +0200 (CEST)
    To: temp@phil.cz
    Subject: Newsletter n°11 du Dictionnaire de la Zone
    MIME-Version: 1.0
    Content-type: text/html; charset=iso-8859-1
    Content-transfer-encoding: 8bit
    Date: Sat, 01 May 2010 19:20:59 +0200
    Status: O
    X-IMAPbase: 1274943631 0000000001
    X-UID: 1

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-27

    Those are the headers that cause the PHP warning for the bad offset? Hmm. Can you please upload the raw message source as an attachment to this tracker? Feel free to zip it or tar it so I can be sure to get the source unchanged. Thanks a lot!

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2010-05-27

    I'm sorry, but I am unable to find any upload or attach button, neither in firefox nor konqueror, and sf.net documentation goes wrong today. The headers are almost right, just lines beginning with a lowercase letters should be indented.

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-27

    Under all the comments, there is a section called "Attached File", and under that, you can see "Add a file". Alternatively, feel free to send directly to me at paul@squirrelmail.org

    Thanks

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-27

    Example mail causes PHP notice

     
  • Paul Lesniewski

    Paul Lesniewski - 2010-05-27

    Thank you for the sample mail -- I have uploaded it to this tracker.

    I still cannot replicate the PHP notice regarding the bad offset, however. If you want to debug further for me, you could try changing the loop to look like this:

    echo "<hr />STRING: $string<br />ACTUAL STRLEN: $actual_strlen<br />";sm_print_r($string);
    while (($entity_pos = sq_strpos($string, '&', $entity_end_pos + 1)) !== FALSE
    && ($entity_end_pos = sq_strpos($string, ';', $entity_pos)) !== FALSE
    && $entity_pos <= $adjusted_max_chars)
    {
    echo " ENTITY: $entity_pos -- $entity_end_pos<br />".substr($string,$entity_pos, $entity_end_pos);
    $adjusted_max_chars += $entity_end_pos - $entity_pos;
    }

    Then add the debug lines suggested by me below on 2010-04-27 19:54:57 GMT (changes in the 900's) and 2010-04-27 20:05:43 GMT (line 948).

    But I appreciate all the time you've contributed and understand if this is too much to ask.

    Thanks again.

     
<< < 1 2 3 > >> (Page 2 of 3)

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks