Re: [SM-I18N] Serious problems with cyrillic encodings after switching to unicodeversion of SM.

[Tomas K. <to...@us...>]

> But I believe that before switching to utf-8 one could see the text
> normally
> when interface encoding was the same as letter encoding at least. Say,
> when browser's encoding is set to koi8-r and original letter uses
> koi8-r. Now it'll work only in case with letter written in utf-8.
> Frankly saying, it's very unusual encoding for mail nowadays...

People stay in their own countries and support of 256 symbols or less is
enough for them. How many times you've seen "This message contains
characters unsupported by your charset" question in Outlook Express? Not
often if system is set to Russian.

I live between Western Europe and Russia. Lithuanian users can use webmail
for Lithuanian only communications, but they can write emails to Russia or
Germany too. iso-8859-4, iso-8859-13 and windows-1257 does not support
Russian characters. Some German characters are not supported too.

If I know that there is charset that allows me to quote Russian jokes and
add Lithuanian comments about them, why shouldn't I use it. I might even
want to reply to those big5 emails that I get sometimes :).

> > > Is this a known bug? Is there a workaround?
> >
> > 1.5.0cvs contains code that allows replying/forwarding emails without
> > loosing 8bit symbols. Actually it is working better, when utf-8 is
> used in
> > translation. With koi8-r you don't loose symbols only when you reply
> to
> > Russian emails.
> >
> >
> http://www.topolis.lt/squirrelmail/patches/pending/stable-only/compose-decode/compose-decode2.diff.gz
> >
> > Code is experimental. My plan for this weekend is to fix header (to:,
> > from:, cc:, bcc:, subject:) decoding.
>
> We are playing with 1.5.0cvs already. Of course we need some time to
> take decision if we can use this, but at first glance it's working good,
> excluding headers :)

Line wrapping needs to be fixed.

Quick fix for headers:
If you remove all non-UTF-8 interface languages (it is easy to do in
1.5.0cvs), open src/compose.php, find lines in newMail, getforwardHeader,
getReplyCitation functions that contain

decodeHeader('some-variable',false,'true or false');
and change them to
decodeHeader('some-variable',true,'true or false');

same thing needs to be done in src/read_body.php SendMDN function.

It is possible that you need to change functions/mime.php function.

--- mime.php    12 Nov 2003 14:57:12 -0000      1.292
+++ mime.php    16 Nov 2003 09:15:40 -0000
@@ -619,7 +619,7 @@
             /* if the last chunk isn't an encoded string then put back
the space, otherwise don't */
             if ($iLastMatch !== $j) {
                 if ($htmlsave) {
-                    $ret .= ' ';
+                    $ret .= ' ';
                 } else {
                     $ret .= ' ';
                 }
@@ -665,7 +665,7 @@

         if (!$encoded) {
             if ($htmlsave) {
-                $ret .= ' ';
+                $ret .= ' ';
             } else {
                 $ret .= ' ';
             }

If you want to keep English or any other translation and use this fix -
change their encoding to utf-8. By changing decodeHeader function you
enable decoding of headers. It doesn't work correctly with other
translations in some cases. In Russia chances to get into such broken
situation is much bigger than in Western Europe.

This is just a quick hack, it won't go into SquirrelMail because I need to
enable header conversion only with utf-8 or with
is_conversion_safe($charset) = true.

Please note, that changing squirrelmail.po to different charset is not
enough, you need to fix functions/i18n.php too. And you will break some of
i18n fixes by doing that. If webmail interface uses non-Unicode charset,
it is not i18n compatible.

-- 
Tomas