Re: [fetchmail-users] Problem with Demon's new Microsoft Exchange Server 2010 service

[Matthias A. <mat...@gm...>]

Am 07.05.2012 12:10, schrieb John Connett:
> Operating system:
>     $ cat /etc/SuSE-release
>     openSUSE 12.1 (x86_64)
>     VERSION = 12.1
>     CODENAME = Asparagus
> 
> Name and origin of the RPM:
>     $ rpm -q fetchmail
>     fetchmail-6.3.21-4.1.3.x86_64
> 
> The name and version of the SMTP listener:
>     $ rpm -q exim
>     exim-4.75-5.1.3.x86_64
> 
> Command-line options you used:
>     openSUSE 12.1 distributed /etc/inid.d/fetchmail with added "-vvv".
> 
> /etc/fetchmailrc (with names changed to protect the innocent):
>     $ cat /etc/fetchmailrc
>     # Configuration created Thu Apr 26 22:58:13 2012
>     set bouncemail
>     set properties ""
>     set daemon 300
>     poll mail.demon.co.uk proto IMAP
>            localdomains yyy.demon.co.uk
>            user 'adm...@yy...' there password 'secret'
> is xxx here
>            sslproto TLS1
>            sslcertck
>     poll mail.demon.co.uk proto IMAP
>            localdomains yyy.demon.co.uk
>            user 'xx...@yy...' there password 'secret' is xxx here
>            sslproto TLS1
>            sslcertck
> 
> Problem:
> I have recently been migrated to Demon's new service which appears to be
> based on Microsoft Exchange Server 2010.  The migration appeared to have
> been successful with messages being fetched.  Then fetching of messages
> ceased.  I added "-vvv" to /etc/init.d/fetchmail and restarted
> fetchmail.  Here are edited highlights from /var/log/fetchmail:
> 
> fetchmail: IMAP< A0004 NO AUTHENTICATE failed.
> fetchmail: IMAP> A0005 LOGIN "xx...@yy..." *
> fetchmail: IMAP< A0005 OK LOGIN completed.
> fetchmail: selecting or re-polling default folder
> fetchmail: IMAP> A0006 SELECT "INBOX"
> fetchmail: IMAP< * 264 EXISTS
> fetchmail: IMAP< * 1 RECENT
> fetchmail: IMAP< * FLAGS (\Seen \Answered \Flagged \Deleted \Draft
> $MDNSent)
> fetchmail: IMAP< * OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted
> \Draft $MDNSent)] Permanent flags
> fetchmail: IMAP< * OK [UNSEEN 1] Is the first unseen message
> fetchmail: IMAP< * OK [UIDVALIDITY 3203] UIDVALIDITY value
> fetchmail: IMAP< * OK [UIDNEXT 2124] The next unique identifier value
> fetchmail: IMAP< A0006 OK [READ-WRITE] SELECT completed.
> fetchmail: 264 messages waiting after first poll
> fetchmail: IMAP> A0007 EXPUNGE
> fetchmail: IMAP< * 264 EXISTS
> fetchmail: IMAP< A0007 OK EXPUNGE completed.
> fetchmail: 264 messages waiting after expunge
> fetchmail: 264 messages for xx...@yy... at mail.demon.co.uk.
> fetchmail: IMAP> A0008 FETCH 1:100 RFC822.SIZE
> fetchmail: IMAP< * 1 FETCH (RFC822.SIZE 8512)
> fetchmail: IMAP< * 2 FETCH (RFC822.SIZE 6404)
> [...]
> fetchmail: IMAP< * 99 FETCH (RFC822.SIZE 2942)
> fetchmail: IMAP< * 100 FETCH (RFC822.SIZE 3914)
> fetchmail: IMAP< A0008 OK FETCH completed.
> fetchmail: IMAP> A0009 FETCH 1 RFC822.HEADER
> fetchmail: IMAP< A0009 OK FETCH completed.
> fetchmail: Incorrect FETCH response: OK FETCH COMPLETED..
> fetchmail: IMAP> A0010 LOGOUT
> fetchmail: IMAP< * BYE Microsoft Exchange Server 2010 IMAP4 server
> signing off.
> fetchmail: IMAP< A0010 OK LOGOUT completed.
> fetchmail: client/server synchronization error while fetching from
> xx...@yy...@mail.demon.co.uk
> fetchmail: 6.3.21 querying mail.demon.co.uk (protocol IMAP) at Mon May 
> 7 10:07:45 2012: poll completed
> fetchmail: Merged UID list from mail.demon.co.uk: <empty>
> fetchmail: Query status=7 (ERROR)
> fetchmail: sleeping at Mon May  7 10:07:45 2012 for 600 seconds
> 
> The problem seems to be the "A0009 OK FETCH completed." response to
> "A0009 FETCH 1 RFC822.HEADER".  There are no patches touching "imap.c"
> in the openSUSE source RPM and I have confirmed that it is identical to
> the 6.3.21 original.
> 
> The "Incorrect FETCH response: OK FETCH COMPLETED.." appears to be
> generated from within imap_fetch_headers() by "an unexpected tagged
> response".  Is this an exciting new innovation in Microsoft Exchange
> Server 2010?

John,

at least it's the first report of its kind that I've become aware of.

I sat down with RFC-3501, the IMAP 4rev1 standard, and my current
understanding is that the server sent an invalid response.

The "unexpected tagged response" means that the server reply lacked
mandatory untagged data.  A valid response to the A0009 FETCH 1
RFC822.HEADER would have been either:

* FETCH (RFC822.HEADER NIL)
A0009 OK FETCH completed.

Or:

* FETCH (RFC822.HEADER "")
A0009 OK FETCH completed.

Or:

* FETCH (RFC822.HEADER {0}
)
A0009 OK FETCH completed.


Having said that, I'll recommend that you report the problem to Demon;
chances are that the message is corrupted on the server and can be
repaired, or that the Exchange 2010 behaviour can be tweaked by their
admins; otherwise and/or on top of that, it would be Demon's task to
talk to Microsoft and have the Redmond guys make Exchange 2010 compliant
to the IMAP protocol, even in the face of incomplete and/or corrupted
messages.

It is likely that the message has no formal Internet header; earlier
Exchange versions used to reassemble messages on-the-fly; and chances
are that configuration options might avoid the server bug.

Note that per RFC-5322, the From: (sender) and Date: (originator's date)
headers are mandatory, so the message couldn't reliably be shipped
forward anyways if it doesn't have these headers, and
garbage-in-garbage-out attempts often fail these days on modern SMTP
servers.


Chances are also that later fetchmail releases may be able to skip over
these messages and leave them on the server unfetched, or delete them
should the user opt to delete corrupted messages.  Given that fetchmail
received the only pending tagged reply, it shouldn't assume it has gone
out of synch though.

Sunil, for resilience towards massive faults on the server side, we'll
probably have to add an error counter so that fetchmail won't attempt to
fetch hundreds of messages' headers and skip over all of them when the
server exhibits systematic faults (and not just some individual
messages, as your report suggests).

Hope that helps for now,
Matthias