I've upgraded from 1.4.22 because of php 5.4 (on Gentoo). I tried to use 1.4 SVN snapshots, but even the recent snapshot fails to display printed quotable fields. If a header contains printed quotable parts, it appears blank - either it is a subject or a sender for example. Hoovering over the sender field shows the email address, but not the name containing the printed quotable characters. In the message body, printed quotable sections are blank. Some parts of the message still appear, but large parts are omitted because of this phenomenon. Please let me know how I can help getting rid of this bug. If I enable PHP logging, it only put some non-relevant notices and warning generated by the database connection (pear). What else I should do to provide more information?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I also have this bug after upgrading my Linux distribution. It's outrageous. I have many users who need to be able to read their messages, and I live in a country where a significant portion of all messages use quoted-printable. Downgrading PHP is something I would seriously not want to do. Please, pretty please, give me a hint!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@dwokfur, thank you for the report, and most sincere apologies for not responding sooner (real life was intense in April). What you can do to help is forward a FULL copy of the message source for a few offending messages (be sure to remove any private info).
@aldaron-j, please control your language. If you think we break things on purpose, and keep the secret to ourselves, then I don't know what to tell you. See above for how to be more constructive.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@pdontthink: after some further testing, I have to say that I was wrong on the specific patch. But the symptoms are there. Unfortunately those emails are too personal to put it simply online. However I would send you some examples to help hunting down the problem. I try to contact you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@jlehtira thank you for providing helpful details and collateral. However, I cannot reproduce the problem using your messages under today's SVN snapshot for version 1.4.23.
Please provide relevant settings, including your user interface language selection and the configuration settings from config.php such as $default_charset, $lossy_encoding and $squirrelmail_default_language
Other things that could have an impact on this might be gettext implementation, etc.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for your replies. I found out that setting $default_charset to iso-8859-1 fixes it, at least makes things appear normal again. Setting PHP:s default_charset to UTF-8 didn't seem to change anything. As the configure tool says the charset option is used only when default language is 'en_US', I didn't try changing it from that.
$lossy_encoding was always false.
Actually.. I put some random gibberish to Default Charset, and SquirrelMail acts the same way as with "utf-8". The config passes the ConfigTest regardless of the gibberish. If that means anything - might be hinting that some utf-8 something is not found?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.
Multibyte (japanese) regex support enabled
Multibyte regex (oniguruma) version 5.9.2
Directive Local Value Master Value
mbstring.detect_order auto auto
mbstring.encoding_translation Off Off
mbstring.func_overload 0 0
mbstring.http_input auto auto
mbstring.http_output UTF-8 UTF-8
mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
mbstring.internal_encoding UTF-8 UTF-8
mbstring.language neutral neutral
mbstring.strict_detection Off Off
mbstring.substitute_character _ _
---
iconv support enabled
iconv implementation glibc
iconv library version 2.17
Directive Local Value Master Value
iconv.input_encoding UTF-8 UTF-8
iconv.internal_encoding UTF-8 UTF-8
iconv.output_encoding UTF-8 UTF-8
3.)
PHP 5.4:
PHP Version: 5.4.15-pl0-gentoo
---
GetText Support enabled
---
Multibyte Support enabled
Multibyte string engine libmbfl
HTTP input encoding translation disabled
libmbfl version 1.3.2
mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.
Multibyte (japanese) regex support enabled
Multibyte regex (oniguruma) version 5.9.2
Directive Local Value Master Value
mbstring.detect_order auto auto
mbstring.encoding_translation Off Off
mbstring.func_overload 0 0
mbstring.http_input auto auto
mbstring.http_output UTF-8 UTF-8
mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
mbstring.internal_encoding UTF-8 UTF-8
mbstring.language neutral neutral
mbstring.strict_detection Off Off
mbstring.substitute_character _ _
---
iconv support enabled
iconv implementation glibc
iconv library version 2.17
Directive Local Value Master Value
iconv.input_encoding UTF-8 UTF-8
iconv.internal_encoding UTF-8 UTF-8
iconv.output_encoding UTF-8 UTF-8
Please let me know what else I can provide.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Contrary to @jlehtira, setting $default_charset to iso-8859-1 or whatever value instead of utf-8 would not fix the symptoms. For exmaple I see only a single 'm' in the subject of sample message5.msg instead of telefonszám. The subject looks like this:
Subject: =?iso-8859-2?Q?telefonsz=E1?= =?iso-8859-2?Q?m?=
That was generated by hotmail.com!
The body of message5.msg is completely empty.
---
Content-Type: text/plain; charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable
Patient T=F3th Name tel.:<phonenumberremoved> =
---
Where we have only a single quoted character represented using iso-8859-2. It displays fine using 1.4.22 along with PHP5.3. Unfortunately I cannot test each version of Squirrel with the same PHP...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@jlehtira Please try to give a more clear picture of exactly what variables you are referring to and what their values were when it was broken and when it was fixed. My $default_charset from config.php is 'utf-8' so that's not the only contributing factor. Please answer all the questions I posed to you earlier so we can try to get an accurate idea of what is causing this. Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@pdontthink:
1.4.22 runs only using PHP5.3, while I can test PHP5.4 only with 1.4.23SVN. I've provided both setups to show, that there are no significant differences regarding the two different versions of PHP used.
The two squirrelmail configs are also the same.
Problem persists after setting $squirrelmail_default_language to 'en_US'.
What else I should try? Do you still cannot reproduce it?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
While switching back-and-forth between PHP5.3+squirrelmail-1.4.22 and PHP5.4+squirrelmail-1.4.23SVN, one of the users came up with this:
"ERROR: Bad or malformed request.
Query: SORT (ARRIVAL) ALL
Server responded: Error in IMAP command UID SORT: Missing search parameters"
The error message is misleading. The userprefs are stored in a mysql database on my host. And the cause is, that something gets messed up in the security_tokens of the user. Restoring the database solves the error message. Sorry for polluting this tracker...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What is your user language setting? Are you using other plugins? It might be best if you can create a new user, disable all plugins temporarily, and see if the problem persists in that account.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@pdontthink: I'm using Dovecot. However the IMAP server stays the same. If I'm using a mail application to connect (mobile app), the emails appear just fine.
I have both hu_HU and en_US users.
I'll get back later with some results regarding a new user and the output of configtest. I have to admit, that the software runs on a production system and due to security restrictions I cannot just simply change anything I want...
Setting default_charset is not enough in my case.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@pdontthink: I haven't set user accounts to any language, I don't even know how to do that. My config_local.php is empty (I suspect that matters), and my own account (that experiences the error), language is set to English.
seems I had simillar problem, some email headers and contents were missing or displayed only partially.
I've managed to work it around, by patching the new function sm_encode_html_specialhars() in functions/strings.php. I don't know whether my patch is correct , so use with care. For me it works well with utf-8 settings in $default_charset
@ptomulik thanks for looking into it, but the solution is not to hard-code character sets. This will break installations/users that use non iso-8859-1.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The problem is that quoted-printable strings have their own encodings built-in, which need to be respected when decoded. Please test the attached patch named quoted_printable_fix-1.4.x.diff --- A patch for SM 1.5.2 is still needed (I have run out of time for now).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've upgraded from 1.4.22 because of php 5.4 (on Gentoo). I tried to use 1.4 SVN snapshots, but even the recent snapshot fails to display printed quotable fields. If a header contains printed quotable parts, it appears blank - either it is a subject or a sender for example. Hoovering over the sender field shows the email address, but not the name containing the printed quotable characters. In the message body, printed quotable sections are blank. Some parts of the message still appear, but large parts are omitted because of this phenomenon. Please let me know how I can help getting rid of this bug. If I enable PHP logging, it only put some non-relevant notices and warning generated by the database connection (pear). What else I should do to provide more information?
I also have this bug after upgrading my Linux distribution. It's outrageous. I have many users who need to be able to read their messages, and I live in a country where a significant portion of all messages use quoted-printable. Downgrading PHP is something I would seriously not want to do. Please, pretty please, give me a hint!
@dwokfur, thank you for the report, and most sincere apologies for not responding sooner (real life was intense in April). What you can do to help is forward a FULL copy of the message source for a few offending messages (be sure to remove any private info).
@aldaron-j, please control your language. If you think we break things on purpose, and keep the secret to ourselves, then I don't know what to tell you. See above for how to be more constructive.
@dwokfur also, please complete the incomplete issue summary. Specifically, which patch did you find was causing the issue?
@pdontthink: after some further testing, I have to say that I was wrong on the specific patch. But the symptoms are there. Unfortunately those emails are too personal to put it simply online. However I would send you some examples to help hunting down the problem. I try to contact you.
@dwokfur, thank you, I received your external emails.
jlehtira has kindly uploaded message samples and corresponding screen shots on tracker number 3614300 --
https://sourceforge.net/tracker/index.php?func=detail&aid=3614300&group_id=311&atid=100311
Big thanks to jlehtira
@jlehtira thank you for providing helpful details and collateral. However, I cannot reproduce the problem using your messages under today's SVN snapshot for version 1.4.23.
Please provide relevant settings, including your user interface language selection and the configuration settings from config.php such as $default_charset, $lossy_encoding and $squirrelmail_default_language
Other things that could have an impact on this might be gettext implementation, etc.
@dwokfur same goes for you. Thank you for getting some examples to me, but I can't reproduce the problem.
Thank you for your replies. I found out that setting $default_charset to iso-8859-1 fixes it, at least makes things appear normal again. Setting PHP:s default_charset to UTF-8 didn't seem to change anything. As the configure tool says the charset option is used only when default language is 'en_US', I didn't try changing it from that.
$lossy_encoding was always false.
Actually.. I put some random gibberish to Default Charset, and SquirrelMail acts the same way as with "utf-8". The config passes the ConfigTest regardless of the gibberish. If that means anything - might be hinting that some utf-8 something is not found?
1.)
Squirrelmail config snippet:
$squirrelmail_default_language = 'hu_HU';
$default_charset = 'utf-8';
$lossy_encoding = false;
gettext -V
gettext (GNU gettext-runtime) 0.18.2
mysql version: 5.5.31-MariaDB
Web server: Apache 2.4.4
2.)
PHP 5.3:
PHP Version: 5.3.25-pl0-gentoo
---
GetText Support enabled
---
mbstring
Multibyte Support enabled
Multibyte string engine libmbfl
HTTP input encoding translation disabled
mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.
Multibyte (japanese) regex support enabled
Multibyte regex (oniguruma) version 5.9.2
Directive Local Value Master Value
mbstring.detect_order auto auto
mbstring.encoding_translation Off Off
mbstring.func_overload 0 0
mbstring.http_input auto auto
mbstring.http_output UTF-8 UTF-8
mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
mbstring.internal_encoding UTF-8 UTF-8
mbstring.language neutral neutral
mbstring.strict_detection Off Off
mbstring.substitute_character _ _
---
iconv support enabled
iconv implementation glibc
iconv library version 2.17
Directive Local Value Master Value
iconv.input_encoding UTF-8 UTF-8
iconv.internal_encoding UTF-8 UTF-8
iconv.output_encoding UTF-8 UTF-8
3.)
PHP 5.4:
PHP Version: 5.4.15-pl0-gentoo
---
GetText Support enabled
---
Multibyte Support enabled
Multibyte string engine libmbfl
HTTP input encoding translation disabled
libmbfl version 1.3.2
mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.
Multibyte (japanese) regex support enabled
Multibyte regex (oniguruma) version 5.9.2
Directive Local Value Master Value
mbstring.detect_order auto auto
mbstring.encoding_translation Off Off
mbstring.func_overload 0 0
mbstring.http_input auto auto
mbstring.http_output UTF-8 UTF-8
mbstring.http_output_conv_mimetypes ^(text/|application/xhtml\+xml) ^(text/|application/xhtml\+xml)
mbstring.internal_encoding UTF-8 UTF-8
mbstring.language neutral neutral
mbstring.strict_detection Off Off
mbstring.substitute_character _ _
---
iconv support enabled
iconv implementation glibc
iconv library version 2.17
Directive Local Value Master Value
iconv.input_encoding UTF-8 UTF-8
iconv.internal_encoding UTF-8 UTF-8
iconv.output_encoding UTF-8 UTF-8
Please let me know what else I can provide.
Contrary to @jlehtira, setting $default_charset to iso-8859-1 or whatever value instead of utf-8 would not fix the symptoms. For exmaple I see only a single 'm' in the subject of sample message5.msg instead of telefonszám. The subject looks like this:
Subject: =?iso-8859-2?Q?telefonsz=E1?= =?iso-8859-2?Q?m?=
That was generated by hotmail.com!
The body of message5.msg is completely empty.
---
Content-Type: text/plain; charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable
Patient T=F3th Name tel.:<phonenumberremoved> =
---
Where we have only a single quoted character represented using iso-8859-2. It displays fine using 1.4.22 along with PHP5.3. Unfortunately I cannot test each version of Squirrel with the same PHP...
@jlehtira Please try to give a more clear picture of exactly what variables you are referring to and what their values were when it was broken and when it was fixed. My $default_charset from config.php is 'utf-8' so that's not the only contributing factor. Please answer all the questions I posed to you earlier so we can try to get an accurate idea of what is causing this. Thank you.
@dwokfur you quoted two PHP setups; which are you using? Can you try briefly changing your $squirrelmail_default_language to 'en_US'?
@pdontthink:
1.4.22 runs only using PHP5.3, while I can test PHP5.4 only with 1.4.23SVN. I've provided both setups to show, that there are no significant differences regarding the two different versions of PHP used.
The two squirrelmail configs are also the same.
Problem persists after setting $squirrelmail_default_language to 'en_US'.
What else I should try? Do you still cannot reproduce it?
While switching back-and-forth between PHP5.3+squirrelmail-1.4.22 and PHP5.4+squirrelmail-1.4.23SVN, one of the users came up with this:
"ERROR: Bad or malformed request.
Query: SORT (ARRIVAL) ALL
Server responded: Error in IMAP command UID SORT: Missing search parameters"
The error message is misleading. The userprefs are stored in a mysql database on my host. And the cause is, that something gets messed up in the security_tokens of the user. Restoring the database solves the error message. Sorry for polluting this tracker...
What is your user language setting? Are you using other plugins? It might be best if you can create a new user, disable all plugins temporarily, and see if the problem persists in that account.
http://pastebin.com/D7uS8xkz is my config.php (URLs censored) when I have the problem. For me, the problem appears fixed with merely:
$default_charset = 'iso-8859-1';
What language are your user accounts set to?
Also, if anyone can share the output of configtest.php, that could be useful.
@dwokfur, what IMAP server are you using?
@pdontthink: I'm using Dovecot. However the IMAP server stays the same. If I'm using a mail application to connect (mobile app), the emails appear just fine.
I have both hu_HU and en_US users.
I'll get back later with some results regarding a new user and the output of configtest. I have to admit, that the software runs on a production system and due to security restrictions I cannot just simply change anything I want...
Setting default_charset is not enough in my case.
@pdontthink: I haven't set user accounts to any language, I don't even know how to do that. My config_local.php is empty (I suspect that matters), and my own account (that experiences the error), language is set to English.
The output I get from configtest.php is in http://pastebin.com/cYawxzzY .
Hi,
seems I had simillar problem, some email headers and contents were missing or displayed only partially.
I've managed to work it around, by patching the new function sm_encode_html_specialhars() in functions/strings.php. I don't know whether my patch is correct , so use with care. For me it works well with utf-8 settings in $default_charset
Update: the patch is currently marked as invalid.
Last edit: Paweł Tomulik 2013-06-12
Sorry for previous post, I've sent the patch here:
https://sourceforge.net/tracker/?func=detail&aid=3614454&group_id=311&atid=300311
Update: the patch is currently marked as invalid.
Last edit: Paweł Tomulik 2013-06-12
@ptomulik thanks for looking into it, but the solution is not to hard-code character sets. This will break installations/users that use non iso-8859-1.
The problem is that quoted-printable strings have their own encodings built-in, which need to be respected when decoded. Please test the attached patch named quoted_printable_fix-1.4.x.diff --- A patch for SM 1.5.2 is still needed (I have run out of time for now).