#2806 quoted printable text appears blank

open
nobody
None
5
2015-06-07
2013-03-29
Toth Attila MD
No

Newly introduced function left parts of headers and message body blank if quoted printable component is present. If I remove the patch, I

Related

Bugs: #2806

Discussion

1 2 3 4 > >> (Page 1 of 4)
  • Toth Attila MD
    Toth Attila MD
    2013-04-02

    I've upgraded from 1.4.22 because of php 5.4 (on Gentoo). I tried to use 1.4 SVN snapshots, but even the recent snapshot fails to display printed quotable fields. If a header contains printed quotable parts, it appears blank - either it is a subject or a sender for example. Hoovering over the sender field shows the email address, but not the name containing the printed quotable characters. In the message body, printed quotable sections are blank. Some parts of the message still appear, but large parts are omitted because of this phenomenon. Please let me know how I can help getting rid of this bug. If I enable PHP logging, it only put some non-relevant notices and warning generated by the database connection (pear). What else I should do to provide more information?

     
  • Toth Attila MD
    Toth Attila MD
    2013-04-02

    • summary: sm_encode_html_special_chars vs quoted printable --> quoted printable text appears blank
     
  • Aldaron
    Aldaron
    2013-06-03

    I also have this bug after upgrading my Linux distribution. It's outrageous. I have many users who need to be able to read their messages, and I live in a country where a significant portion of all messages use quoted-printable. Downgrading PHP is something I would seriously not want to do. Please, pretty please, give me a hint!

     
  • @dwokfur, thank you for the report, and most sincere apologies for not responding sooner (real life was intense in April). What you can do to help is forward a FULL copy of the message source for a few offending messages (be sure to remove any private info).

    @aldaron-j, please control your language. If you think we break things on purpose, and keep the secret to ourselves, then I don't know what to tell you. See above for how to be more constructive.

     
  • @dwokfur also, please complete the incomplete issue summary. Specifically, which patch did you find was causing the issue?

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-03

    @pdontthink: after some further testing, I have to say that I was wrong on the specific patch. But the symptoms are there. Unfortunately those emails are too personal to put it simply online. However I would send you some examples to help hunting down the problem. I try to contact you.

     
  • @jlehtira thank you for providing helpful details and collateral. However, I cannot reproduce the problem using your messages under today's SVN snapshot for version 1.4.23.

    Please provide relevant settings, including your user interface language selection and the configuration settings from config.php such as $default_charset, $lossy_encoding and $squirrelmail_default_language

    Other things that could have an impact on this might be gettext implementation, etc.

     
  • @dwokfur same goes for you. Thank you for getting some examples to me, but I can't reproduce the problem.

     
  • Thank you for your replies. I found out that setting $default_charset to iso-8859-1 fixes it, at least makes things appear normal again. Setting PHP:s default_charset to UTF-8 didn't seem to change anything. As the configure tool says the charset option is used only when default language is 'en_US', I didn't try changing it from that.

    $lossy_encoding was always false.

    Actually.. I put some random gibberish to Default Charset, and SquirrelMail acts the same way as with "utf-8". The config passes the ConfigTest regardless of the gibberish. If that means anything - might be hinting that some utf-8 something is not found?

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-04

    1.)
    Squirrelmail config snippet:
    $squirrelmail_default_language = 'hu_HU';
    $default_charset = 'utf-8';
    $lossy_encoding = false;

    gettext -V
    gettext (GNU gettext-runtime) 0.18.2

    mysql version: 5.5.31-MariaDB
    Web server: Apache 2.4.4

    2.)
    PHP 5.3:
    PHP Version: 5.3.25-pl0-gentoo

    ---
    GetText Support enabled

    ---
    mbstring
    Multibyte Support enabled
    Multibyte string engine libmbfl
    HTTP input encoding translation disabled

    mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.

    Multibyte (japanese) regex support enabled
    Multibyte regex (oniguruma) version 5.9.2

    Directive Local Value Master Value
    mbstring.detect_order auto auto
    mbstring.encoding_translation Off Off
    mbstring.func_overload 0 0
    mbstring.http_input auto auto
    mbstring.http_output UTF-8 UTF-8
    mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
    mbstring.internal_encoding UTF-8 UTF-8
    mbstring.language neutral neutral
    mbstring.strict_detection Off Off
    mbstring.substitute_character _ _

    ---
    iconv support enabled
    iconv implementation glibc
    iconv library version 2.17

    Directive Local Value Master Value
    iconv.input_encoding UTF-8 UTF-8
    iconv.internal_encoding UTF-8 UTF-8
    iconv.output_encoding UTF-8 UTF-8

    3.)
    PHP 5.4:
    PHP Version: 5.4.15-pl0-gentoo

    ---
    GetText Support enabled

    ---
    Multibyte Support enabled
    Multibyte string engine libmbfl
    HTTP input encoding translation disabled
    libmbfl version 1.3.2

    mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.

    Multibyte (japanese) regex support enabled
    Multibyte regex (oniguruma) version 5.9.2

    Directive Local Value Master Value
    mbstring.detect_order auto auto
    mbstring.encoding_translation Off Off
    mbstring.func_overload 0 0
    mbstring.http_input auto auto
    mbstring.http_output UTF-8 UTF-8
    mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
    mbstring.internal_encoding UTF-8 UTF-8
    mbstring.language neutral neutral
    mbstring.strict_detection Off Off
    mbstring.substitute_character _ _

    ---
    iconv support enabled
    iconv implementation glibc
    iconv library version 2.17

    Directive Local Value Master Value
    iconv.input_encoding UTF-8 UTF-8
    iconv.internal_encoding UTF-8 UTF-8
    iconv.output_encoding UTF-8 UTF-8

    Please let me know what else I can provide.

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-04

    Contrary to @jlehtira, setting $default_charset to iso-8859-1 or whatever value instead of utf-8 would not fix the symptoms. For exmaple I see only a single 'm' in the subject of sample message5.msg instead of telefonszám. The subject looks like this:
    Subject: =?iso-8859-2?Q?telefonsz=E1?= =?iso-8859-2?Q?m?=
    That was generated by hotmail.com!
    The body of message5.msg is completely empty.
    ---
    Content-Type: text/plain; charset="iso-8859-2"
    Content-Transfer-Encoding: quoted-printable

    Patient T=F3th Name tel.:<phonenumberremoved> =
    ---
    Where we have only a single quoted character represented using iso-8859-2. It displays fine using 1.4.22 along with PHP5.3. Unfortunately I cannot test each version of Squirrel with the same PHP...

     
  • @jlehtira Please try to give a more clear picture of exactly what variables you are referring to and what their values were when it was broken and when it was fixed. My $default_charset from config.php is 'utf-8' so that's not the only contributing factor. Please answer all the questions I posed to you earlier so we can try to get an accurate idea of what is causing this. Thank you.

     
  • @dwokfur you quoted two PHP setups; which are you using? Can you try briefly changing your $squirrelmail_default_language to 'en_US'?

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-05

    @pdontthink:
    1.4.22 runs only using PHP5.3, while I can test PHP5.4 only with 1.4.23SVN. I've provided both setups to show, that there are no significant differences regarding the two different versions of PHP used.
    The two squirrelmail configs are also the same.
    Problem persists after setting $squirrelmail_default_language to 'en_US'.
    What else I should try? Do you still cannot reproduce it?

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-05

    While switching back-and-forth between PHP5.3+squirrelmail-1.4.22 and PHP5.4+squirrelmail-1.4.23SVN, one of the users came up with this:
    "ERROR: Bad or malformed request.
    Query: SORT (ARRIVAL) ALL
    Server responded: Error in IMAP command UID SORT: Missing search parameters"
    The error message is misleading. The userprefs are stored in a mysql database on my host. And the cause is, that something gets messed up in the security_tokens of the user. Restoring the database solves the error message. Sorry for polluting this tracker...

     
  • What is your user language setting? Are you using other plugins? It might be best if you can create a new user, disable all plugins temporarily, and see if the problem persists in that account.

     
  • http://pastebin.com/D7uS8xkz is my config.php (URLs censored) when I have the problem. For me, the problem appears fixed with merely:

    $default_charset = 'iso-8859-1';

     
  • What language are your user accounts set to?

    Also, if anyone can share the output of configtest.php, that could be useful.

    @dwokfur, what IMAP server are you using?

     
  • Toth Attila MD
    Toth Attila MD
    2013-06-09

    @pdontthink: I'm using Dovecot. However the IMAP server stays the same. If I'm using a mail application to connect (mobile app), the emails appear just fine.
    I have both hu_HU and en_US users.
    I'll get back later with some results regarding a new user and the output of configtest. I have to admit, that the software runs on a production system and due to security restrictions I cannot just simply change anything I want...
    Setting default_charset is not enough in my case.

     
  • @pdontthink: I haven't set user accounts to any language, I don't even know how to do that. My config_local.php is empty (I suspect that matters), and my own account (that experiences the error), language is set to English.

    The output I get from configtest.php is in http://pastebin.com/cYawxzzY .

     
  • Paweł Tomulik
    Paweł Tomulik
    2013-06-10

    Hi,

    seems I had simillar problem, some email headers and contents were missing or displayed only partially.
    I've managed to work it around, by patching the new function sm_encode_html_specialhars() in functions/strings.php. I don't know whether my patch is correct , so use with care. For me it works well with utf-8 settings in $default_charset

    Update: the patch is currently marked as invalid.

     
    Last edit: Paweł Tomulik 2013-06-12
  • @ptomulik thanks for looking into it, but the solution is not to hard-code character sets. This will break installations/users that use non iso-8859-1.

     
  • The problem is that quoted-printable strings have their own encodings built-in, which need to be respected when decoded. Please test the attached patch named quoted_printable_fix-1.4.x.diff --- A patch for SM 1.5.2 is still needed (I have run out of time for now).

     
1 2 3 4 > >> (Page 1 of 4)