[fetchmail-devel] [PATCH] Re: Incorrect header line and lost mails

[Sunil S. <sh...@bo...>]

Quoting from Matthias Andree's mail on Tue, Mar 14, 2006:
> > One option is to change the default to a lower value from the current
> > value of 10. The lowest practical value is 2, which will use linear
> > search in alternate polls.
> 
> The algorithm tries to find the first unused UIDL with a binary search,
> on the assumption that all new messages were at the end of the
> list. This doesn't work for Fr?d?ric's server and others.

Imagine the worst-case scenario: Frederic's mailbox has 2000 mails on
this server and it has to be accessed on a slow dialup line which is
prone to disconnection. Even if the assumption doesn't work, it will
still download a few mails. Going back to linear search will probably
fetch even less mails due to the overheads.

> > Other solutions are also possible. For example, keeping a map of UID
> > and message number in memory (with corrections for mails deleted after
> > logout).
> 
> Aren't we keeping those in memory anyways? If so, such a check wouldn't
> hurt and serve those users whose servers create messages out of order.
> 
> In the long run, we'd probably better keep them in a database on disk to
> keep the memory footprint down.

The question is not just of keeping in memory or database. The
question is: What should be done if a mismatch is found? If the only
solution is to go back to downloading all UIDs at one go, then I would
prefer continuing with the incorrect assumption.

The fight is not just between linear vs. binary search. The basic idea
of fastuidl is to avoid downloading all UIDs at one go.

> Well. Should I lower the default to 4 for the nonce or wait for a patch? :)

Here it is:

===============================================================================
Index: fetchmail-6.3/fetchmailconf.py
===================================================================
--- fetchmail-6.3/fetchmailconf.py	(revision 4739)
+++ fetchmail-6.3/fetchmailconf.py	(working copy)
@@ -249,7 +249,7 @@
 	self.warnings = 3600	# Size warning interval (see tunable.h)
 	self.fetchlimit = 0	# Max messages fetched per batch
 	self.fetchsizelimit = 100	# Max message sizes fetched per transaction
-	self.fastuidl = 10	# Do fast uidl 9 out of 10 times
+	self.fastuidl = 4	# Do fast uidl 3 out of 4 times
 	self.batchlimit = 0	# Max message forwarded per batch
 	self.expunge = 0	# Interval between expunges (IMAP)
 	self.ssl = 0		# Enable Seccure Socket Layer
Index: fetchmail-6.3/fetchmail.man
===================================================================
--- fetchmail-6.3/fetchmail.man	(revision 4739)
+++ fetchmail-6.3/fetchmail.man	(working copy)
@@ -615,7 +615,7 @@
 once followed by binary searches in 'n-1' polls if 'n' is greater than
 1; binary search is always used if 'n' is 1; linear search is always
 used if 'n' is 0. In non-daemon mode, binary search is used if 'n' is
-1; otherwise linear search is used.
+1; otherwise linear search is used. The default value of 'n' is 4.
 This option works with POP3 only.
 .TP
 .B \-e <count> | \-\-expunge <count>
Index: fetchmail-6.3/fetchmail.c
===================================================================
--- fetchmail-6.3/fetchmail.c	(revision 4739)
+++ fetchmail-6.3/fetchmail.c	(working copy)
@@ -970,7 +970,7 @@
     def_opts.remotename = user;
     def_opts.listener = SMTP_MODE;
     def_opts.fetchsizelimit = 100;
-    def_opts.fastuidl = 10;
+    def_opts.fastuidl = 4;
 
     /* get the location of rcfile */
     rcfiledir[0] = 0;
===============================================================================

-- 
Sunil Shetye.