#28 1.4.5 -- Special characters converted to "?" (Question mark)

closed
nobody
None
5
2005-12-19
2005-12-07
No

Squirrelmail v. 1.4.5

Special characters (in this case French characters such
as é, à, ç) as well as some single quote characters (')
are shown incorrectly in message view and displayed as
the question mark '?' character.

The same message viewed in Squirrelmail version 1.4.3a
didn't exhibit the problem.

Browsers and their encoding settings:
----

IE 6: tried UTF-8, Western European (ISO), Western
European (Windows), Auto-Select

Firefox 1.5: tried Western (ISO-8859-1), UTF-8 (even
worse), English (US-ASCII)

Squirrelmail language settings:
----

1. Default Language : en_US
2. Default Charset : iso-8859-1
3. Enable lossy encoding : true/false (no difference)

Tried with display languages: en_US, fr_FR

I would be willing to send a sample email reproducing
the problem.

Discussion

  • Logged In: YES
    user_id=1175083

    PHP version 4.2.2

     
    • labels: --> 102904
     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-08

    Logged In: YES
    user_id=225877

    Content-Type, Content-Transfer-Encoding and MIME-version
    headers from affected emails?

     
  • Logged In: YES
    user_id=1175083

    The email in question has none of those headers.

     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-08

    Logged In: YES
    user_id=225877

    Contact person that created email message and ask him or her
    to fix used mailer. Mailer violates email formating rules.
    If email does not have MIME-Version header, it should
    contain only standard 7bit us-ascii characters.

    Emails that use 8bit characters are formated according to
    rfc2045-2048. RFC 2045 requires MIME-Version header (see
    chapter 4). If 8bit characters are not encoded, same rfc
    requires Content-Transfer-Encoding: 8bit header (see chapter
    6, default header value is 7bit). Content-Type header should
    be used to set email character set information.

    You might be able to undo conversion to question marks, if
    you remove functions/decode/us_ascii.php file. This file
    contains us-ascii character set decoding function, which
    blocks all 8bit characters.

    Don't confuse functions/decode/us_ascii.php with
    functions/encode/us_ascii.php. SquirrelMail 1.4.5 might
    break, if you remove us-ascii encoding function.

     
  • Logged In: YES
    user_id=1175083

    What will be the impact of removing that file from production?

    We may have users using Squirrelmail in other languages.
    How will it impact the viewing of us-ascii (7bit) files in
    other language settings of SM?

    Is there a reason why this was added to Squirrelmail after
    1.4.3a?

     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-08

    Logged In: YES
    user_id=225877

    if file is removed, us-ascii mails are displayed undecoded.
    email should be displayed using same character set as the
    one that is used by selected translation.

    SquirrelMail 1.4.3 and older contained
    charset_decode_iso_8859_default() function which applied
    similar decoding to unsupported iso-8859-x character sets.
    In 1.4.4 decoding system was rewritten. Changes added
    automatic loading of decoding functions from
    functions/decode directory and removed last decoding
    functions from functions/i18n.php.
    charset_decode_iso_8859_default() function was moved to
    us_ascii.php and updated.

    Correct solution is to fix email format. If mailer does not
    provide character set information, email will be displayed
    correctly only when used charset matches the one that is
    used by selected translation. If such broken emails reach
    courier mta, they will be corrupted.

    P.S. could you update your user information. SourceForge
    switched to utf-8 and your last name is displayed incorrectly.

     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-12

    • labels: 102904 -->
     
  • Logged In: YES
    user_id=1175083

    Thanks for the solution. We ended up removing the us-ascii
    decode file. That suited our needs.

     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-19

    • status: open --> closed
     
  • Tomas Kuliavas
    Tomas Kuliavas
    2005-12-19

    Logged In: YES
    user_id=225877

    Issue is specific to broken email message. Message can't be
    displayed correctly, if it does not provide correct
    character set information.

    Closing bug tracker.