#1763 Some binary attachments are truncated after NULL

open
nobody
Compose (426)
5
2005-04-26
2005-04-26
No

Compose: Binary attachments containing a NULL
character and the string <HEAD> are sent truncated.

Binary attachements that contain in the first 256
characters the strings <HEAD> or <BODY> or <HTML>
are marked by SquirrelMail as "text/html", although they
contain also a Null character.

By the most of mailservers are all bytes from the first
Null character to the next line terminator (\r or \l)
stripped from the attachment of MIME type "text/html"
or these Null characters are converted to another
character (space or \0x80).

For example the file containing 7 bytes "\0<HEAD>" is
sent as a zero-length file.

In my opinion, all files that contain the a Null character
should be marked as "aplication/octet-stream" (if can’t
be found any more specific binary type), but never
should be marked as "text/html" or other "text/…". Text
files that only at the end of file have one or more added
Null character (e.g. some RTF) can still be marked as
the text.

= the stream been sent by SquirrelMail (bad) =
Content-Type: text/html; name="file..."
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="file..."

\0<head>
----boundary (the "\0" is a binary Null character)

= the data stored by the mailserver from SquirrelMail -
(bad) =
Content-Type: text/html; name="file..."
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="file..."

----boundary

= Netscape 7 (the same, but correct) =
Content-Type: application/octet-stream; name="file..."
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="file..."

ADxoZWFkPg==
----boundary

= Internet Explorer 6 (correct) =
Content-Type: application/octet-stream; name="file..."
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="file..."

=00<head>
----boundary

The real problem concerns the file format used by the
Czech Post Office (national) as an envelope of
cryptografic transferred data:

"<CPBF><HEAD>...some binary data...<DATA>...some
binary data..."

SquirrelMail - all versions 1.4.4 / 1.4.5-cvs / 1.5.1-cvs
Snapshots
PHP/4.3.3
Apache/2.0.47 (Fedora)

Discussion

  • Hynek Cernoch

    Hynek Cernoch - 2005-04-28

    Logged In: YES
    user_id=1265286

    The solution suggested in version 1.51 is not right.
    // remove NUL characters
    $body_part = str_replace("\0",'',$body_part);

    A binary file should remain exactly the same. In my opinion,
    the only solution is to mark the file as some MIME binary
    type.

    There are other 3 requiremens:
    1) Not only characters after NUL, but also the NUL character
    self should remain
    2) Characters \r (\0xD) should not be converted to \n (\0xA)
    3) The \r\n should not be converted to single \n
    All these changes are made by mailserver when the MIME
    type is bad (text).

    Also Quoted-Printable or Base64 encoded attachments are
    converted by mailserver to 8bit without encoding, when their
    MIME type is text and so are they corrupted. Therefore it
    would not be good solution to change the Content-Transfer-
    Encoding only.

     
  • Marc Groot Koerkamp

    Logged In: YES
    user_id=476981

    Okey, 2 things.
    1) The NUL replacements in squirrelmail are required because
    mail should not contain NUL characters.
    2) If as you described the DATA parts contains binary data
    then the encoding should be BASE64.

    So the real problem is that we initially assumed
    text/whatever mail never contains NUL characters and
    secondary we thought it always can be removed.

    The sollution probably is encoding attachements always as
    base64.

    But i have to look at this issue later on.

    Thnx for the info you provided.

     
  • Hynek Cernoch

    Hynek Cernoch - 2005-04-29

    Logged In: YES
    user_id=1265286

    ** RESOLVED **

    You are right. The change of encoding base64 is good
    enough for the mailserver to correctly save attachment in the
    mailbox, but some email client (outlook express) use to
    modify every received attachment of type "text/html".
    (Outlook adds the text
    "<!-- saved from url=(0022)http://internet.e-mail -->" and
    sometimes a line terminator at the end)

    Also it was is necessary to detect which the binary files were
    bad recognized by PHP

    I changed the function saveAttachedFiles in file
    src/compose.php
    The lines from comment "added by hynekcer" to "end
    hynekcer" were added:

    The original function was taken from CVS Revision 1.411
    cvs: squirrelmail/squirrelmail/src/compose.php

    - this is a very first function executed only if a new
    attachment is uploaded
    - only here is the type of attachment tested ($_FILES
    ['attachfile']['type'])
    => this was the best place to modify

     
  • Hynek Cernoch

    Hynek Cernoch - 2005-04-29

    bug fix for ./src/compose.php

     
  • Jonathan Angliss

    Logged In: YES
    user_id=620333

    I have a slight issue with this patch, and that is the fread
    of the whole attachment to process it. This becomes an
    issue on slightly larger attachments. I know you've limited
    this code to text/* type files only, but you could very
    easily have a 3MB richtext file which will cause havok with
    the memory on the server if you are sticking with PHP defaults.

     
  • Thijs Kinkhorst

    Thijs Kinkhorst - 2005-11-22

    Logged In: YES
    user_id=285765

    That's easy to resolve by looping over the file. The only
    thing I see with this is that it adds quite a performance
    hit on attaching files, especially relevant for larger
    installs. Therefore, I'd rather solve the problem that the
    filetype is detected incorrectly. How does that happen?

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks