[fetchmail-devel] Re: BUG (?) fetchmail 6.3.2 hangs with 'idle' on OpenBSD

[Sunil S. <sh...@bo...>]

Quoting from Casper Gripenberg's mail on Wed, Mar 01, 2006:
> Ok I see. On the IMAP server I use the mail delivery is instant. 
> I tried it with direct telnet to the smtp server, and as soon as 
> I send the mail I see it on the IMAP side. There is absolutely 
> no delay.

Interesting.

> Here is the output. I cleaned it up a bit, and it includes my
> mta output too, so you can see when my testmail leaves my machine.

...

> 11:51:38 : IMAP> A0005 NOOP
> 11:51:38 : IMAP< A0005 OK NOOP completed
> 11:51:55 : IMAP< * 1 EXISTS
> 11:51:55 : IMAP> A0006 EXPUNGE
> 11:51:55 : IMAP< * 1 RECENT
> 11:51:55 : IMAP< A0006 OK EXPUNGE completed

So, the EXISTS notification is actually coming unattached and not as a
part of response to any command. Interesting.

> Ok. Right. It was a shoot from the hip fix. I just wanted to avoid the
> delay if possible. But if it's not avoidable in any proper way then
> that is the way it is. I guess people can live with getting their 
> e-mails a few seconds later, it's not THAT critical :) But for me
> the delay in there will still bother me..can't help it, sorry :)
> For anyone else, I guess they will never know. I would have never
> known hadn't I started investigating the problem anyway.

I have finally analysed the patch by Brendan Lynch and found that it
has correctly identified the source of the problem. So, please junk my
previous patch and try this one. Again, log to syslog to get the time
aspect correct.

This patch fixes the imap_idle() code in several ways:

- tag is emptied before waiting for asynchronous notification. This
  comes from the patch by Brendan Lynch.

- stage is currently being set to STAGE_IDLE throughout imap_idle().
  This is incorrect. stage should be STAGE_IDLE only when sending
  "IDLE" command and when waiting for asynchronous notification. In
  particular, it should not be set to STAGE_IDLE when sending "NOOP"
  as "NOOP" is a normal imap command and does not idle. Similar
  argument holds true after "DONE" as been sent.

- imap_idle() has 5 return statements. This makes working out what
  happens in which case difficult. Now, there will be only one return
  statement.

- After getting the "EXISTS" asynchronous notification, the "NOOP"
  command is sent *immediately* to get the updated "RECENT" count.
  
======================================================================
Index: fetchmail-6.3/imap.c
===================================================================
--- fetchmail-6.3/imap.c	(revision 4705)
+++ fetchmail-6.3/imap.c	(working copy)
@@ -621,7 +621,6 @@
 {
     int ok;
 
-    stage = STAGE_IDLE;
     saved_timeout = mytimeout;
 
     if (has_idle) {
@@ -629,6 +628,7 @@
 	 * at least every 28 minutes:
 	 * (the server may have an inactivity timeout) */
 	mytimeout = 1680; /* 28 min */
+	stage = STAGE_IDLE;
 	/* enter IDLE mode */
 	ok = gen_transact(sock, "IDLE");
 
@@ -637,37 +637,43 @@
 	    SockWrite(sock, "DONE\r\n", 6);
 	    if (outlevel >= O_MONITOR)
 		report(stdout, "IMAP> DONE\n");
-	} else
-	    /* not idle timeout */
-	    return ok;
+	    /* reset stage and timeout here: we are not idling any more */
+	    mytimeout = saved_timeout;
+	    stage = STAGE_FETCH;
+	    /* get OK IDLE message */
+	    ok = imap_ok(sock, NULL);
+	}
     } else {  /* no idle support, fake it */
-	/* when faking an idle, we can't assume the server will
-	 * send us the new messages out of the blue (RFC2060);
-	 * this timeout is potentially the delay before we notice
-	 * new mail (can be small since NOOP checking is cheap) */
-	mytimeout = 28;
+	/* the server may not send us new mail notification out of the
+	 * blue (RFC2060). Note: stage and timeout have not been
+	 * changed here as NOOP does not idle */
 	ok = gen_transact(sock, "NOOP");
-	/* if there's an error (not likely) or we just found mail (stage 
-	 * has changed, timeout has also been restored), we're done */
-	if (ok != 0 || stage != STAGE_IDLE)
-	    return(ok);
 
-	/* wait (briefly) for an unsolicited status update */
-	ok = imap_ok(sock, NULL);
-	/* again, this is new mail or an error */
-	if (ok != PS_IDLETIMEOUT)
-	    return(ok);
+	/* no error, but no new mail either */
+	if (ok == PS_SUCCESS && recentcount == 0)
+	{
+	    /* There are some servers who do send new mail
+	     * notification out of the blue. Wait for that with a low
+	     * timeout */
+	    mytimeout = 28;
+	    stage = STAGE_IDLE;
+	    /* We are waiting for notification; no tag needed */
+	    tag[0] = '\0';
+	    /* wait (briefly) for an unsolicited status update */
+	    ok = imap_ok(sock, NULL);
+	    if (ok == PS_IDLETIMEOUT) {
+		/* no notification came; ok */
+		ok = PS_SUCCESS;
+	    }
+	}
     }
 
     /* restore normal timeout value */
+    set_timeout(0);
     mytimeout = saved_timeout;
     stage = STAGE_FETCH;
 
-    /* get OK IDLE message */
-    if (has_idle)
-        return imap_ok(sock, NULL);
-
-    return PS_SUCCESS;
+    return(ok);
 }
 
 static int imap_getrange(int sock, 
======================================================================

-- 
Sunil Shetye.