[bug] Race condition in UIDNEXT handling causes "lost track of message(s)"

[Tomas T. <tr...@se...>]

Hi again.

Playing with mbsync, I found this:

Symptom:
After storing a message in a local store and then syncing with
a remote store, I get
Warning: lost track of 1 pushed message(s)
Almost every time I sync again, this message gets duplicated.

Investigation:
After mbsync APPENDs a message, the server of my provider responds
with UIDNEXT (in the state after storing the message):

>>> 5 APPEND "sent" (\Seen) "20-Dec-2014 19:53:16 +0100" {342}
+ You should send the message.
* 1 EXISTS
* 0 RECENT
* OK [UIDNEXT 460] Predicted next UID
* 1 FETCH (FLAGS (\Seen))
5 OK APPEND completed

This behavior causes mbsync not to see the appended message and to
treat the appended message lost (probably because mbsync updates its
UIDNEXT counter prematurely):

>>> 6 CHECK
6 OK CHECK completed.
>>> 7 UID FETCH 460:1000000000 (UID BODY.PEEK[HEADER.FIELDS (X-TUID)])
7 OK FETCH completed
Warning: lost track of 1 pushed message(s)

See the 460 in "UID FETCH 460:1000000000", which should be 458 in my
case.

According to RFC 3501, the behavior of my server seems correct:
"Server data MAY be sent as a result of a client command, or MAY be
sent unilaterally by the server."
also:
"A client MUST be prepared to accept any server response at all
times. This includes server data that was not requested."


I think mbsync should cache the state of last UIDNEXT received before
APPENDing a message and use this value for
the "UID FETCH xxx:1000000000" command.


I attach the full output of "mbsync -V my-account 2>&1" (with
pipelining deactivated) showing how mbsync sends a message to
the remote store and subsequently gets confused by its
"[UIDNEXT 460]" response.


Note:
I am very new into IMAP (I just started investigating it today), but
if I understand the RFC correctly, you may also consider
using "*" instead of "1000000000" as in "UID FETCH 458:*". But don't
forget about the slightly different behavior of "*" if you choose so:
"Also note that a UID range of 559:* always includes the UID of
the last message in the mailbox, even if 559 is higher than any
assigned UID value."


Thank you again for the great software.

Best regards,
Tomas Tintera


PS. I apologize if I should rather send this report to the tracker
at sf.net. Please correct me if this is the case.