[fetchmail-devel] Re: handle imap EXPUNGE better

[Sunil S. <sh...@bo...>]

[Sorry for the delayed reply]

Quoting from Matthias Andree's mail on Thu, Dec 29, 2005:
> Sunil Shetye <sh...@bo...> writes:
> I have some concerns about this:
> 
> > +		    /* Some servers do not report RECENT after an EXPUNGE.
> > +		     * For such servers, assume that the mail being
> > +		     * expunged is a recent one. For other servers, we
> > +		     * should get an updated RECENT report later and this
> > +		     * assumption will have no effect. */
> > +		    if (recentcount > 0)
> > +			recentcount--;
> > +		    actual_deletions++;
> 
> How do we know that the other client deleting messages is deleting
> recent ones?  A getmail client with "delete_after" or moldremover or
> such might remove old and seen messages and we're decreasing RECENT
> without good reason. It looks as though the assumption were a bit bold.

This issue affects only those imap servers who do not report RECENT on
EXPUNGE. All the good imap servers do report RECENT like this:

IMAP> A0158 EXPUNGE
IMAP< * 1 EXPUNGE
IMAP< * 3 EXISTS
IMAP< * 3 RECENT                 [ some imap servers do not report this line ]
IMAP< A0158 OK EXPUNGE completed

> It would be better to update the cache properly (watching out for
> eventual iterator variables that need to be decremented, too) and
> continue. From our own cached flags we should be able to deduce if the
> expunged message was recent or not. If we don't know and the server
> expunged a different mail.
> 
> Now, before you go delve into the code and hack away, take a step back
> from the blackboard to see the whole picture: the RECENT flags are just
> crap if several clients are accessing the mailbox. What happens to the
> RECENT count if we reselect the mailbox? Can we actually use it? Do we
> need to check \Seen flags instead, for lack of UID support in IMAP?

I plan to implement the recentcount as a difference between EXISTS
count at end and beginning of poll. I can work on that only after this
patch is accepted. At that time, your point will also get implemented.

> The long way to a fix, which is not going to be in 6.3.X, but 6.4.0 at
> the earliest, will be to rewrite the UID code to be
> protocol-independent, allow for storing attributes with messages (such
> as arrival or retrieval timestamp, size) in the .fetchids file, and
> store one file per (server, account, folder) tuple. We have your code
> from 2003 that splits the uid files, and I made some .fetchids
> reformatting change on top of that.  I still find uid.c ugly, and I
> doubt it's up to the job, so we'll have to rewrite that.
> 
> Next, we'd have to teach the IMAP code to use UID/UIDVALIDITY on servers
> that support it for client-side tracking and make that the default, and
> make POP3+UIDL default. We have IMAP patches that use the UID, but
> AFAICT they don't pay attention to the UIDVALIDITY and so require some
> work. Again, this is all 6.4.X stuff.
> 
> Comments, suggestions?

fetchmail uses the RECENT only to decide
a) whether to repoll at the end of mailbox poll
b) whether to come out of IDLE (or NOOP look) while waiting for new
mail.
The UID code is not going to help here as this would require getting
the full list of UIDs at the end of mailbox poll (and every minute,
with "idle"). The RECENT code is much cleaner as imap servers reports
the RECENT count consistently and correctly.

The RECENT count is not being used to decide which mails are new. It
is only being used at the end of the poll or while idling for new
mail.

-- 
Sunil Shetye.