fetchmail-devel

[fetchmail-devel] broken behavior in Exchange Server 2003 IMAP4rev1

[Adam S. <sim...@ci...>]

Hi,

I use fetchmail to grab my mail over IMAP from an Exchange server.
I've noticed that a number of multipart MIME messages become broken
after being downloaded by fetchmail.  The messages look fine when I
point my mail client directly at the IMAP server.  However, once
they've been downloaded by fetchmail, the boundary used in the message
body doesn't match the one in the Content-Type header.

I tracked down the problem, and it occurs because the Exchange server
is rewriting the MIME boundary before sending it to the IMAP client.
It rewrites the boundary differently (using a different randomly
generated string) for each IMAP request.  Because fetchmail downloads
the headers and body separately, this results in a different boundary
in the Content-Type header than in the message body.

This only happens for messages with a Content-Transfer-Encoding of
"binary".  Messages with a transfer encoding of "7bit" and "8bit" seem
to work just fine.


This looks like the same problem described with Lotus Domino 5 servers
in section X6 of the FAQ.  This message seems to indicate that you are
reluctant to modify fetchmail to work around this broken behavior:

  http://lists.berlios.de/pipermail/fetchmail-devel/2006-August/000760.html

I don't suppose I can convince you to reconsider?  (I somehow doubt
I'll have much luck convincing Microsoft to fix exchange.)


In case it helps, my IMAP server reports itself as:
  Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1

-- 
Adam Simpkins

Re: [fetchmail-devel] broken behavior in Exchange Server 2003 IMAP4rev1

[Rob M. <rob...@gm...>]

On 11/2/07, Adam Simpkins <sim...@ci...> wrote:
> Hi,
<---SNIP--->>
> I tracked down the problem, and it occurs because the Exchange server
> is rewriting the MIME boundary before sending it to the IMAP client.
> It rewrites the boundary differently (using a different randomly
> generated string) for each IMAP request.  Because fetchmail downloads
> the headers and body separately, this results in a different boundary
> in the Content-Type header than in the message body.
<---SNIP--->
> In case it helps, my IMAP server reports itself as:
>   Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1

Can you use POP instead, and if so does it have the same bug?

I'd personally suggest filing a bug with Microsoft.  Re-writing emails
like that probably runs the risk of breaking digitally signed emails
that include headers (DKIM springs to mind, though I suspect it
doesn't sign that header by default).  Of course, I'd expect the
answer to be "Upgrade to the latest version" :)

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] broken behavior in Exchange Server 2003 IMAP4rev1

[Matthias A. <mat...@gm...>]

Am Donnerstag, den 01.11.2007, 18:06 -0700 schrieb Adam Simpkins:

> I tracked down the problem, and it occurs because the Exchange server
> is rewriting the MIME boundary before sending it to the IMAP client.

So tell Exchange not to do that.
Why would a client want to cater for servers who tamper with data? It's
the MIME boundary today and the sender address tomorrow. And why would a
client want to use such a server at all?

> It rewrites the boundary differently (using a different randomly
> generated string) for each IMAP request.  Because fetchmail downloads
> the headers and body separately, this results in a different boundary
> in the Content-Type header than in the message body.

...

[bug fixing for Lotus insanity transferred to MS Exchange?]
> I don't suppose I can convince you to reconsider?  (I somehow doubt
> I'll have much luck convincing Microsoft to fix exchange.)

Well, if I had time at my hands, I might offer to fix this for money.
Given that 6.3.9 has been delayed for weeks already, I'd rather not make
any offers in that direction.

I'm aware this isn't too helpful, but I don't see it as my foremost
interest to work around server bugs - especially not if the changes to
code would have to be rather intrusive.

Sorry.

Best regards
Matthias

Re: [fetchmail-devel] broken behavior in Exchange Server 2003 IMAP4rev1

[Adam S. <sim...@ci...>]

On Fri, Nov 02, 2007 at 05:58:31PM +0100, Matthias Andree wrote:
> Am Donnerstag, den 01.11.2007, 18:06 -0700 schrieb Adam Simpkins:
> 
> > I tracked down the problem, and it occurs because the Exchange server
> > is rewriting the MIME boundary before sending it to the IMAP client.
> 
> So tell Exchange not to do that.
> Why would a client want to cater for servers who tamper with data? It's
> the MIME boundary today and the sender address tomorrow. And why would a
> client want to use such a server at all?

Yes, I agree Exchange is broken here.  Unfortunately, I don't have any
control over the mail server.  I have contacted my IT support contacts
about this problem with the server, but I haven't heard anything back
from them yet.  (And I don't have high hopes that they'll be willing
or able to fix this problem anyway.)


> [bug fixing for Lotus insanity transferred to MS Exchange?]
> > I don't suppose I can convince you to reconsider?  (I somehow doubt
> > I'll have much luck convincing Microsoft to fix exchange.)
> 
> Well, if I had time at my hands, I might offer to fix this for money.
> Given that 6.3.9 has been delayed for weeks already, I'd rather not make
> any offers in that direction.
> 
> I'm aware this isn't too helpful, but I don't see it as my foremost
> interest to work around server bugs - especially not if the changes to
> code would have to be rather intrusive.

Sure, I completely understand.  I've switched to POP3 for now, which
shouldn't have the problem.

If I have the time later, I might create a patch for this on my own.
Would you consider accepting the patch if I came up with one?  It
looks like I should probably just create a new method struct identical
to the current "imap" one, but with a single fetch method that gets
everthing, and fetch_body set to NULL.  Is that the preferred
approach, or did you have something else in mind?

-- 
Adam Simpkins
sim...@ci...

Re: [fetchmail-devel] broken behavior in Exchange Server 2003 IMAP4rev1

[Matthias A. <mat...@gm...>]

Sorry, this message was meant to be public. Resending to list, Adam - sorry for the dupe.

Am Sonntag, den 04.11.2007, 12:26 -0800 schrieb Adam Simpkins:

> If I have the time later, I might create a patch for this on my own.
> Would you consider accepting the patch if I came up with one?  It
> looks like I should probably just create a new method struct identical
> to the current "imap" one, but with a single fetch method that gets
> everthing, and fetch_body set to NULL.  Is that the preferred
> approach, or did you have something else in mind?

I will consider the patch, yes.

I think an command line option to switch existing IMAP behavior to
"fetch whole message" is more useful, in order to avoid code duplication
and user irritation at configuration time.

It's already _very_ confusing to have APOP and KPOP as "protocols" when
they are in fact just authentication variants of POP3. Let's not add to
that for IMAP.

New options need to be added to the manpage, the lexer and parser
(the .l and .y files) and to options.c.

-- 
Matthias Andree <mat...@gm...>