[fetchmail-devel] idle polling on multiple servers

[Andy S. <and...@gm...>]

Hi, I've been looking into adding support for polling more than one
server using the IMAP IDLE feature. I'm very new to the fetchmail source
code, so any suggestions or comments would be helpful :)


Currently, IDLE support is implemented in the imap_getrange function by
blocking the getrange call until new messages are available. Afterwards,
it forces a repoll and repeats the process waiting for new messages. The
down side of this is that it starves out all the other servers because
the IMAP do_session never returns (except on errors).

From what I understand, many people have expressed concerns about
polling multiple servers concurrently and I agree that using threads or
forks would be a bad idea. However, we should be able to use poll(2) or
select(2) to accomplish the same thing without the threading issues. As
it turns out, fetchmail uses select to implement it's interruptible_idle
function anyway.

I've put up a working prototype for polling multiple servers here if
anyone is interested in taking a look at it. Please treat it with some
skepticism though, it still needs a lot of work ;)

    http://pileus.org/git/?p=~andy/fetchmail;h=idle


The basic idea is that the blocking getrange is removed and the servers
socket is saved between polls and passed to select() so that fetchmail
can wake up when new mail arrives.


I added two new functions to the 'struct method' table that replace the
existing, internal, imap_idle function:

  start_idle:  called at the end of a fetch, this puts the connection in
               IDLE mode so that we can wake up when mail arrives

  finish_idle: called at the start of the next poll, resets the
	       connection to 'normal mode'


There's also a couple additions to 'struct hostdata' (I don't know if
this the correct structure to use, but it seems to work).

  true_idle:   set to TRUE if the server supports IDLE polling,
               otherwise we may need to fake it somehow

  idle_socket: the socket saved between polls, the connection setup and
	       destruction in do_session is skipped while idling

  idle_ready:  set to TRUE after calling select() if data is available
               for reading on the socket


Whew, I hope that wasn't too long, I'll keep poking around at this
during the next few weeks and see if I can clean it up a bit. I thought
it would be good to give the other developers a heads up though.