#18 Various IMAP errors when syncing message

[Julez]

Hi

Firstly, thanks for creating mbsync. It looks really useful and I have just started using mbsync (version 1.2.0). After lots of testing on small mailbox test message bases and everything seemingly working OK, I have now tried to sync the last few months of real emails from gmail to icloud mail (Pull All from gmail Push Flags from icloud). This unfortunately has resulted in various errors and crashes including the following in order of prevalence:

• IMAP command 'APPEND "INBOX" ' returned an error: NO Message contains NUL characters
• IMAP command 'APPEND "INBOX" ' returned an error: NO Message contains invalid header
• Error from IMAP server: Invalid tag
• IMAP error: unexpected tag <meta (followed by mbsync hanging)
• IMAP error: unexpected command continuation request

Each of these is generally followed by: Warning: slave refuses to store message xxxxx from master

mbsync makes a valiant effort of restarting stating:

Warning: lost track of xxxx pulled message(s) and then tries to continue with the messages it hasn't synced. However, messages that previously failed are not retried.

I have not dug any deeper into these issues yet and not sure if it is some kind of mbsync config issue on my part or result of a buggy IMAP implementation. However, it seems repeatable (i.e. if I clear down the slave mailbox and clear the mbsync status) the same messages seem to cause the same issue.

I would be happy to help in any way but would appreciate your guidance on what you need to look into this further.

[Julez]

Just a quick update. I have ported and adjusted the mbsyncrc file back to version 1.1.3 and it works perfectly over the same message base. So currently seems to be an issue in version 1.2.0 only

[Oswald Buddenhagen]

i would guess that this is a buffering problem, possibly related to compression. that can plausibly be new in 1.2.0.

the first step would be running with -Dn (and possibly -DN, but don't attach that - just check whether you see garbled messages yourself, and describe the corruption if you do).

running under valgrind memcheck would be useful as well.

[Oswald Buddenhagen]

no info forthcoming

[ChriS]

I have the same issue with version 1.2.1-1 in Debian experimental. Unfortunately, I had no chance to run it with -Dn because the error was transient.
The connection that I am using right now (not my usual one) is really slow. Could that explain the message "lost track of xxxx pulled message(s)"?

[ChriS]

Same error with version 1.3.0-2 (in Debian). The errors displayed on screen during a normal run are

Warning: lost track of 5 pulled message(s)
Warning: lost track of 1 pushed message(s)
Socket error from outlook.office365.com ([2603:1026:101::2]:993): Connection reset by peer

The log with -Dn is attached.

[ChriS]

It may well be that a folder became duplicated (the filtering became case sensitive) leading to duplicate messages with the same ID. After removing the duplicated folder, the error is gone. The message “lost track of XXX message(s)” does not point to that direction though.

[Oswald Buddenhagen]

exchange being randomly stupid is tracked in bug #22. that's probably what the original report was about, too.