Re: [fetchmail-devel] imap: introduce imapsearchmax parameter

[Sunil S. <sun...@ro...>]

--- On Fri, 29/4/11, Andrea Righi <rig...@gm...> wrote:
> > > > 1. Parse the response to the SELECT command:

...

> A0001 SELECT "INBOX"
> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted
> \Seen \*)]
> * OK [UIDVALIDITY 2]
> * 518299 EXISTS
> * 0 RECENT
> * OK [UIDNEXT 652168]
> A0001 OK [READ-WRITE] INBOX selected. (Success)

Parsing will not give the unseen count information for you. Also, I 
realised later that this method will not work when 'idle' is enabled.

> > > > 2. Send a STATUS command

This may not work correctly on open folders.

> > > > 3. Send an extended SEARCH command

This is not supported by your server.

So, at the IMAP protocol level, we have run out of options.

I will check if the fetchmail parsing can be improved directly in 
fetchmail itself.

The following is a recap of the problem:

==========================================================================
fetchmail gets the unseen message list using the SEARCH command. There is 
no standard mechanism to get the count and range of unseen messages.

fetchmail currently reads the response to the SEARCH command in a buffer 
of size MSGBUFSIZE=8k. In a normal case when there are only a few new 
mails, the response fits nicely in this buffer:

IMAP> A0010 SEARCH UNSEEN
IMAP< * SEARCH 499998 499999 500000
IMAP< A0010 OK SEARCH completed

However, when there are too many unseen mails, the buffer response gets 
truncated (possibly in the middle of a number).

IMAP> A0011 SEARCH UNSEEN
IMAP< * SEARCH 490001 490002 490003 ... 499998 499999 500000
IMAP< A0011 OK SEARCH completed

IMAP_SEARCH_MAX=1k (which in reality should be a function of MSGBUFSIZE 
and count) tries to avoid the truncation by searching for unseen mail in 
batches.

IMAP> A0012 SEARCH 1:1000 UNSEEN
IMAP> A0013 SEARCH 1001:2000 UNSEEN
IMAP> A0014 SEARCH 2001:3000 UNSEEN

The problems with this approach are:

- In the most common case, there are very few unseen mails. This method of 
searching is inefficient in those cases.

- If the mailbox size is huge (say having 500k mails), fetchmail gets 
caught in an unproductive cycle.
==========================================================================

-- 
Sunil Shetye.