Re: [fetchmail-devel] fetchmail fix coordination and future planning (was: [PATCH] imap: introduce imapsearchmax parameter)

[Sunil S. <sun...@ro...>]

Hi Matthias,



----- Original Message -----
> As so the second (with _split and caching), I wonder if it's the right
> approach. It looks a bit as though we're curing the symptoms instead of
> the cause.  The cause is a buffer too small to handle the response, and
> the cure would be to make the buffer dynamic and possibly resize it if
> we don't get a \r?\n in it.
> 
> 
> A dynamic buffer could also save us the strcpy(argbuf, buf) copy which,
> IMO, is quite wasteful.

I had given a thought for implementing dynamic buffers. However, some issues came to my mind:

- What if the buffer required is very long? For example, an extreme response like:

IMAP< * SEARCH 1 2 3 4 5 6 ... 999995 999996 999997 99998 99999 100000

will require a dynamic buffer of size 575k. Is it okay to allocate such a huge buffer?

- Who will free the buffer? Obviously, it will have to be statically allocated in order to free it later.

- How long will it remain? Is it safe to free it immediately on the next call?

In short, without a proper memory manager, it may not be safe to just allocate a dynamic buffer of such a huge size.


Note that the strcpy(argbuf, buf) can still be avoided by directly passing argbuf (if available) to gen_recv().


Also, note that that patch is a bug fix. The current method range search for UNSEEN mails is too slow for huge mailboxes. So, a fix for this issue is required for 6.3.X.


> What I propose is to have three branches, details below the list:
> 
> - one for 6.3.X (new name to be devised), for a final release and
> possibly later critical/security fixes
> 
> - one for a new 6.4.X or 7.X.X branch (master) where such fixes as the
> IMAP search ranges can be made

I think, it is necessary to fix the range search issue in 6.3.X itself as fetchmail is now unusable with IMAP when the mailbox is huge and keep is on.

-- 
Sunil Shetye.