#2711 Squirrelmail timeouts on email with bad headers

Produces PHP errors
closed-fixed
nobody
Folders (317)
5
2010-06-21
2010-04-14
Anonymous
No

If a user receives an email with bad headers syntax (typically a UTF8 character directly in Subject, From or To header without MIME encoded-word syntax (RFC 2047)), SquirrelMail 1.4.20 is unable to display the list of emails in the folder and it usually timeouts with php error "execution time exceeded 30s". I know these emails are against RFC standards, but these simply arrive sometimes, and it shouldn't block the user's entire mailbox. Squirrelmail 1.4.20-RC2 handles these improperly formatted email correctly, with no timeouts or slow-downs.

A typical email that causes this problem is attached.

Discussion

1 2 3 > >> (Page 1 of 3)
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-14

    Bad headers email - UTF8 chars directly in Subject

     
    Last edit: Anonymous 2014-03-14
  • I cannot reproduce. If you would, please try with a fresh snapshot of 1.4.21[SVN] from our downloads page. If the problem still happens, please show the exact error message and all of your language settings and current user language.

    You should also report this violation to the sending system's administrator, for they are the ones who should fix their software.

     
    • status: open --> pending-works-for-me
     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-14

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-14

    • status: pending-works-for-me --> open-works-for-me
     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-14

    I am able to reproduce the problem with fresh install of SM-1.4.21-SVN-20100414 with locales-SVN-20100414. However I was able to identify some configurations when this problem does not occur:

    Default Language | Default Charset | User Language | Status
    en_US | iso-8859-1 | English | OK
    en_US | iso-8859-1 | Czech | PROBLEM
    en_US | utf-8 | English | PROBLEM
    en_US | utf-8 | Czech | PROBLEM
    cs_CZ | iso-8859-2 | English | OK
    cs_CZ | iso-8859-2 | Czech | PROBLEM
    cs_CZ | utf-8 | English | PROBLEM
    cs_CZ | utf-8 | Czech | PROBLEM

    When SM freezes, apache2 produces high load on the server for about half minute:
    28267 www-data 40 0 133m 17m 4388 R 100 0.4 0:20.10 apache2

    SM does not display any PHP (nor any other) error, the message list just does not load and it does not list any messages, see attached screenshot.

    Thank you.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-14

    I just forgot to specify that SM-1.4.20-RC2 and all older versions work with all settings, including our usual setting:

    Default Language: cs_CZ
    Default Charset: utf-8
    User Language: Czech

     
    • status: open-works-for-me --> pending-works-for-me
     
  • I still cannot reproduce, possibly due to different locale setup. Please show the output of your src/configtest.php and what locales you have on your machine. Are you using standard gettext system or SquirrelMail's internal stand-in? Also, show all PHP mb_string configuration values - I suspect the problem is with mb_strlen() or mb_strpos() and possibly mismatched charset.

    Again, the REAL solution is to have the vendor of the offending client fix their software. Did you contact them?

    If you want to help debug further, since I cannot currently help, you can try first increasing your PHP logging verbosity, since the memory timeout you describe SHOULD in fact give you an error with some source code line numbers. Increase logging verbosity (error_reporting setting) and watch your log file if you don't have display_errors turned on. Another thing you can do is insert your own debug statements starting at about line 170 in functions/strings.php such as:

    echo "1<br />";

    And subsequent echo statements further down the code until you find where things are stalling out.

    Thank you

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-15

    Thank you. Configtest outputs are available at
    https://webmail.jh-inst.cas.cz/stable/src/configtest.php (1.4.20)
    https://webmail.jh-inst.cas.cz/testing/src/configtest.php (1.4.21-SVN-20100414)
    https://webmail.jh-inst.cas.cz/stable/rc2/configtest.php (1.4.20-RC2)

    I don't know about gettext, how can I determine which version are we using?

    mbstring configuration is php default, nothing changed, see https://webmail.jh-inst.cas.cz/phpinfo.php

    I know that the real problem is on the sender's side, I have contacted them all. But our mailserver receives 100.000 emails daily, and we have a few "bad headers" email every day. I cannot contact them all, and my users must be able to access their mailbox even when they receive a "bad headers" email.

    Thank you.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-15

    • status: pending-works-for-me --> open-works-for-me
     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-15

    I forgot again. Following locales are installed on the server

    cs_CZ ISO-8859-2
    cs_CZ.UTF-8 UTF-8
    en_US.ISO-8859-15 ISO-8859-15
    en_US.UTF-8 UTF-8

    Plus there is a typo in the address for configtest output of 1.4.20-RC2 (the last version that works for us), it is
    https://webmail.jh-inst.cas.cz/rc2/src/configtest.php

    Thank you.

     
  • I still cannot find any problems here. Can you please indicate how big the mailbox is where this is happening? Are there lots of messages? If you put the example message into a folder with just a few other messages, does the problem go away?

    Your original report, you say that the timeout coincides with a PHP error indicating the problem. You need to show the full text of that error message in order to help pinpoint the problem.

    You can also try the debugging lines that I suggested earlier, if you're comfortable with digging in the source. Also, please explain any language/encoding related settings you've made in SquirrelMail's configuration file.

    Thank you.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    Thank you again for your help. Currently no error is displayed in the web browser, but there are errors in the logs:

    PHP Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/squirrelmail-1.4.20/functions/strings.php on line 1245, referer: https://webmail.jh-inst.cas.cz/stable/src/left_main.php

    Will this help? Thank you!

     
  • 1) Please answer the other questions I asked.

    2) Line 1245 in vanilla 1.4.20 is this: function sq_lowercase_array_vals(&$val,$key) {
    It is unlikely that this is the timeout location. Please show what line 1245 looks like for you and explain why your copy of SquirrelMail is modified - show all modifications you've made.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    Sorry. The problem does not depend on the amount of messages in the folder, I can reproduce it with a single-message mailbox. I haven't modified the SM source files, I don't understand PHP.

    Lines 1245-1247 are

    function sq_lowercase_array_vals(&$val,$key) {
    $val = strtolower($val);
    }

    In 1.4.21-svn-20100414 the same problem occurs, PHP error is

    PHP Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/squirrelmail-1.4.21-svn-20100414/functions/strings.php on line 1182, referer: https://webmail.jh-inst.cas.cz/testing/src/left_main.php

    line 1182 in this version is

    array_walk($ret,'sq_lowercase_array_vals');

    So the problem really seems to be with the sq_lowercase_array_vals function?

     
  • Even if you put the email in a folder with nothing else, how big is the *INBOX* for your test account? Is it small, too?

    How busy is the server? What is its load? How many users are accessing it?

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    New account, just the one problematic email in INBOX. Server is running two quad-core Intel Xeons E5335 (8 cores together), average load 0.5. The server is not overloaded, the problem does not appear with SM 1.4.20-rc2 and older.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    Thank you. Unfortunatelly the patch did not help, just the error changed so it timeouts on another line in the PHP code:

    PHP Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/squirrelmail-1.4.20/functions/strings.php on line 884, referer: https://webmail.jh-inst.cas.cz/stable/src/left_main.php

    line 884 is

    function sq_strpos($haystack, $needle, $offset=0, $charset='auto')

     
  • Well, then I'm out of ideas. I simply cannot reproduce the issue. I suggest trying the debug methods I recommended below.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    I think it is the strtolower($val) function. It seems to have problems with UTF-8 encoded characters with accents. For UTF-8 it should be used mb_strtolower($val,$charset).

    I changed line 1246 from
    $val = strtolower($val);
    to
    $val = mb_strtolower($val,$charset);

    and now it works, problem solved :)

     
  • 1) It's not possible to force the use of mbstring functions for people who don't have that extension installed

    2) The only place sq_lowercase_array_vals() is used is in sq_mb_list_encodings(), where the array being operated on is just a list of mbstring encodings. That list of encodings should not include any non-ASCII characters, including unicode valies with accents, etc. Please show an array dump of $supported_encodings in your environment. Thank you.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here


    Anonymous
    2010-04-21

    Hello, as I wrote previously, I am not a programmer, and I don't understand almost anything in the php source code. Can you please specify what is "an array dump of $supported_encodings" and how can I get it for you? Thank you.

     
  • On about line 1181 (depending on the version), you should find:

    $ret = mb_list_encodings();

    AFTER that line, put this:

    sm_print_r($ret); exit;

    Thank you for all your help.

     
1 2 3 > >> (Page 1 of 3)