#13 stop Exchange from converting text/plain parts to text/html

open
nobody
None
5
2010-09-03
2010-09-03
No

Stop Exchange from converting text/plain parts to text/html.

This unconditionally sets a plain text property on the whole message. Such a work-around forces preservation of all text/plain MIME parts and of simple text/plain messages. This depends on Exchange ignoring the property on non-text parts.

Tested only with DAV.

Discussion

  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-04

    This does not preserve the format=flowed parameter of the Content-Type header.

    As a result, replies made in Thunderbird will have "greater than" signs instead of quoting bars. The MUAs will unwrap paragraphs in quotations. Empty lines between quotations and subsequent paragraphs will be removed.

    I.e., instead of wrapped quotations with a bar on the left and a separate follow-up,

    | Aaaa bbb ccc ddd
    | ee ffff ggg hhh

    Awesome!

    MUAs will show the raw text in the quotation and eliminate empty lines preceding the follow-up,

    > Aaaa bbb ccc ddd ee ffff ggg hhh
    Awesome!

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-07

    I compared headers fetched by the Davmail IMAP layer from the OWA layer with the headers fetched by MS Outlook from the Exchange server. It appears that the OWA layer strips the "format=flowed" part as I can see it in the sent copy fetched by MS Outlook from the server.

    MS Outlook shows copies of my sent messages having the ms/tnef content-type with the binary content-transfer-encoding. I only get text/plain content-type with the base64 transfer encoding through OWA/davmail IMAP.

     
  • Mickael Guessant

    Very interesting, but I found a side effect: when you set messageFormat to 1 on the draft, html messages are also converted to text/plain.

    In addition, could you please give more details on the format=flowed issue ?

     
  • Mickael Guessant

    I tried to set different values on mailOverrideFormat depending on mail Content-Type to preserve html content with text only IMS and plaintext content with html IMS.

    Can you please check this ?

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-09

    As of rev. 1433, a text/plain message gets stored in the Sent folder and shows up in Inbox as text/html, beas64-encoded.

    I found 2 options with resembling names and property IDs in Microsoft documentation. They probably relate to different programming and data flow layers. Description of both options does not specify the exact meaning of their values. For example, it does not say whether a particular format name specifies a condition on which some action applies or if the name is part of unconditional action.

    PR_INETMAIL_OVERRIDE_FORMAT:
    0: default format/default encoding,
    1: MIME format/base64 encoding,
    2: Text format/UUEncode encoding
    3: Text format/BINHEX encoding
    http://msdn.microsoft.com/en-us/library/cc160675%28office.12%29.aspx

    PidTagInternetMailOverrideFormat:

    A complex bitmask that does not seem to be compatible with PR_INETMAIL_OVERRIDE_FORMAT values.
    http://msdn.microsoft.com/en-us/library/ee237479%28EXCHG.80%29.aspx

     
  • Mickael Guessant

    According to my tests, the message in Sent folder is still wrong, but the one received on an external address is right (text/plain preserved).

    And PidTagInternetMailOverrideFormat and PR_INETMAIL_OVERRIDE_FORMAT is the same property according to [ms-oxprops].pdf

    Will do some more tests with the bitmap definition.

    Outlook sets value 0x00160000
    ENCODING_PREFERENCE | ENCODING_MIME | BODY_ENCODING_TEXT_AND_HTML | INTERNET_FORMAT_DEFAULT

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-09

    I checked the messages in Inbox through Microsoft Outlook, and they looked as text/plain. I did not investigate their ms/tnef winmail.dat content to figure more details.

    The same message shows up as text/html, base64-encoded when fetched through the davmail IMAP / OWA gateway.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-09

    Oops, my words "looked like text/plain" proved to only reflect my fixed width font selection in Outlook's Tools/Options/Mail Formats/Fonts/International Fonts/Proportional font. I figured that after restoring that font option to a proportional font name. The messages showed up in the proportional font. I understand this may indicate that the messages had a content type other than "text/plain".

     
  • Mickael Guessant

    A new patch is available in subversion, seems to work fine in more cases.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-10

    I am afraid this will keep converting plain text messages to HTML if they have attachments.

    Also, I cannot figure out why all my messages show up in Outlook ms/tnef-encoded when they show in OWA/davmail IMAP as base64-encoded with either text/html or text/plain content type.

     
  • Mickael Guessant

    You are right, will need to handle more complex cases once simple ones are OK.

    About the tnef encoding, can you try to set the MAPI_SEND_NO_RICH_INFO flag ?

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-15

    I added setting the mailOverride property on plain text simple messages with the option you mentioned and without the "text and html" option. This still resulted in MS Outlook showing winmail.dat messages in its Internet headers box. The box I am referring to shows up after double-clicking the message, choosing View / Options...

    With MS Outlook, all messages except those saved in the Sent Items folder appear with ms/tnef content type, binary content type encoding.

    I also tried PUTting a message/rfc821 content type after reading an Evolution document,

    http://git.gnome.org/browse/evolution-exchange/tree/docs/sending-mail

    but this did not change anything other than the fact that the Sent copy showed with all headers in the body.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-15

    (Forgot to tell that I tried setting PidTagSendRichInfo and PidLidUseTnef to "false" without any visible outcome. I defined DAV field instances for these fields using only 2-byte IDs. I did not realize that the "use TNEF" property requires a 4-byte ID, and I do not know how to send it).

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-15

    Just to allow verification of my negative findings, here is the combined patch,

    @@ -1471,7 +1462,7 @@
    if (noneMatch != null) {
    putmethod.setRequestHeader("If-None-Match", noneMatch);
    }
    - putmethod.setRequestHeader("Content-Type", "message/rfc822");
    + putmethod.setRequestHeader("Content-Type", "message/rfc821");
    putmethod.setRequestEntity(new ByteArrayRequestEntity(mimeContent, "message/rfc822"));
    try {
    httpClient.executeMethod(putmethod);
    @@ -2234,6 +2225,8 @@
    if (properties != null && properties.containsKey("messageFormat")) {
    davProperties.add(Field.createDavProperty("messageFormat", properties.get("messageFormat")));
    }
    + davProperties.add(Field.createDavProperty("sendRichInfo", "false"));
    + davProperties.add(Field.createDavProperty("useTnef", "false"));
    if (!davProperties.isEmpty()) {
    patchMethod = new PropPatchMethod(messageUrl, davProperties);
    try {
    @@ -2251,11 +2244,12 @@
    // update message body
    PutMethod putmethod = new PutMethod(messageUrl);
    putmethod.setRequestHeader("Translate", "f");
    - putmethod.setRequestHeader("Content-Type", "message/rfc822");
    + putmethod.setRequestHeader("Content-Type", "message/rfc821");

    try {
    // use same encoding as client socket reader
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    + baos.write( "MAIL FROM:<ilatypov@rim.com>\r\nRCPT TO:<ilatypov@rim.com>\r\n".getBytes() );
    mimeMessage.writeTo(baos);
    baos.close();
    putmethod.setRequestEntity(new ByteArrayRequestEntity(baos.toByteArray()));
    @@ -2367,6 +2361,7 @@
    properties.put("draft", "9");
    String contentType = mimeMessage.getContentType();
    if (contentType != null && contentType.startsWith("text/plain")) {
    + properties.put("mailOverrideFormat", String.valueOf(ENCODING_PREFERENCE | ENCODING_MIME | MAPI_SEND_NO_RICH_INFO));
    properties.put("messageFormat", "1");
    } else {
    properties.put("mailOverrideFormat", String.valueOf(ENCODING_PREFERENCE | ENCODING_MIME | BODY_ENCODING_TEXT_AND_HTML));
    Index: src/java/davmail/exchange/dav/Field.java
    ===================================================================
    --- src/java/davmail/exchange/dav/Field.java (revision 1450)
    +++ src/java/davmail/exchange/dav/Field.java (working copy)
    @@ -132,9 +132,10 @@
    createField(URN_SCHEMAS_MAILHEADER, "bcc");//PS_INTERNET_HEADERS/bcc
    createField(URN_SCHEMAS_HTTPMAIL, "datereceived");//PR_MESSAGE_DELIVERY_TIME, 0x0E06

    - // unused: force message encoding
    createField("messageFormat", 0x5909, PropertyType.Integer);//PR_MSG_EDITOR_FORMAT EDITOR_FORMAT_PLAINTEXT = 1 EDITOR_FORMAT_HTML = 2
    createField("mailOverrideFormat", 0x5902, PropertyType.Integer);//PR_INETMAIL_OVERRIDE_FORMAT ENCODING_PREFERENCE = 2 BODY_ENCODING_TEXT_AND_HTML = 1 ENCODING_MIME = 4
    + createField("useTnef", 0x8582, PropertyType.Boolean); // PidLidUseTnef "do not use TNEF" = false "override PR_MSG_EDITOR_FORMAT and use TNEF" = true
    + createField("sendRichInfo", 0x3a40, PropertyType.Boolean); // PidTagSendRichInfo "do not use TNEF" = false "override PR_MSG_EDITOR_FORMAT and use TNEF" = true

    // IMAP search

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-15

    PROPPATCHing textdescription preserves simple text encoding in OWA and fixed font output in MS Outlook. I do not know how to save attachments with PROPPATCH and whether there is a way to preserve MIME skeletons. Perhaps, my earlier experiment with skeletons failed due to generating an unexpected binary format.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-09-15

    Check the type of the last non-attachment text part. Save a skeleton. Drawbacks: Outlook will still show the message in a proportional font; OWA shows forced text/plain parts decoded from base64. Correct definitions of "html" and "useTnef".

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2010-11-03

    Proppatch the message body property to get a quoted-printable transfer encoding as a side effect. This has a side eff. of Outlook showing the text in fixed font. Last change: disregard plain text attachments when looking for body. STILL BREAKS CHARSET.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2011-02-08

    (a) My uninformed-turned-incorrect handling of the message encoding results in Unicode characters displayed as a series of 2-3 Latin1 symbols.

    (b) My patch has another major deficiency where it saves an EML attachment (plain text storage of the complete email message with all headers) as if it were an unknown binary file (octec/stream). This prevents the MUA itself from displaying attached EML files inline.

    I switched to EWS after finding my EWS URL with EWSEditor from Microsoft Code Gallery,

    http://code.msdn.microsoft.com/ewseditor

    It has a command to add a default EWS connection. Then I looked at the properties of that connection to find the EWS URL. According to a response by a Microsoft employee, finding the EWS URL may be as easy as calling one function.

    http://social.technet.microsoft.com/Forums/en-US/exchangesvrdevelopment/thread/9a916133-00cf-4150-b787-b764ec0e0343/

    Perhaps, this may be useful in filling out the default value for the configuration dialog.

     
  • Mickael Guessant

    DavMail uses autodiscover as a failover, but it relies on Exchange and DNS settings that are usually not setup correctly.

     
  • Ilguiz Latypov

    Ilguiz Latypov - 2011-02-09

    Thanks for clarifying on the way davmail uses autodiscovery.

    With regard to preventing MS Exchange server from presenting my plain text emails in the variable width font, I suspect this occurs only in MS Office Outlook. When I start Outlook, it would show my messages in fixed font during its first minute, after which clicking my messages shows them in proportional font. I do not understand the reason and mechanism behind this delayed re-formatting. I suspect it may relate to the MS Exchange server converting text parts of my messages to HTML.

    I feel relieved that davmail stores and retrieves messages intact. If I send plain text messages and messages with plain text parts to myself, I can see them in fixed font in my Thunderbird nightly (Shredder). Seeing the message source confirms that their storage in MS Exchange allows retrieving them in the original MIME structure.

    Perhaps, MS Office Outlook retrieves messages in a non-MIME structure requesting the server to HTMLize text bodies regardless of their original MIME type. Or Outlook may attempt to HTMLize text bodies itself. I see that plain text in some messages survives in Outlook. Some of such messages declare an ASCII character set. Perhaps, the character set property of the message affects its text plain preservation as a side effect.

     
  • Mickael Guessant

    The root cause is probably that Outlook does not support MIME messages but uses MAPI internally to access Exchange Store.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks