#131 Improve IDLE handling

[Kevin Ross]

It appears that when going into IDLE mode, DavMail will poll the Exchange server using the FindItem operation. Doing this too often (more often than once every 3 minutes) makes my Exchange server very unhappy, as you can see below:

2021-04-11 12:47:53,780 ERROR [ImapConnection-54202] davmail.exchange.ExchangeSession  - Unable to parse BackOffMilliseconds For input string: "Name: BackOffMilliseconds299957"
2021-04-11 12:47:53,781 WARN  [ImapConnection-54202] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 60 seconds
2021-04-11 12:47:53,906 WARN  [ImapConnection-62246] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 59 seconds
2021-04-11 12:47:53,922 WARN  [ImapConnection-56357] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 59 seconds
2021-04-11 12:48:32,073 WARN  [ImapConnection-62477] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 21 seconds
2021-04-11 12:48:53,819 ERROR [ImapConnection-62477] davmail.exchange.ExchangeSession  - Unable to parse BackOffMilliseconds For input string: "Name: BackOffMilliseconds239919"
2021-04-11 12:48:53,819 WARN  [ImapConnection-62477] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 60 seconds
2021-04-11 12:48:53,830 WARN  [ImapConnection-54202] davmail.imap.ImapConnection  - Client closed connection
2021-04-11 12:48:53,838 ERROR [ImapConnection-62246] davmail.exchange.ExchangeSession  - Unable to parse BackOffMilliseconds For input string: "Name: BackOffMilliseconds239902"
2021-04-11 12:48:53,838 WARN  [ImapConnection-62246] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 60 seconds
2021-04-11 12:48:53,854 ERROR [ImapConnection-56357] davmail.exchange.ExchangeSession  - Unable to parse BackOffMilliseconds For input string: "Name: BackOffMilliseconds239885"
2021-04-11 12:48:53,855 WARN  [ImapConnection-56357] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 60 seconds
2021-04-11 12:49:08,479 WARN  [ImapConnection-51530] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 45 seconds
2021-04-11 12:49:31,778 WARN  [ImapConnection-56471] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 22 seconds
2021-04-11 12:49:53,849 WARN  [ImapConnection-62477] davmail.imap.ImapConnection  - Client closed connection
2021-04-11 12:49:53,880 WARN  [ImapConnection-62246] davmail.imap.ImapConnection  - Client closed connection
2021-04-11 12:49:53,891 ERROR [ImapConnection-56471] davmail.exchange.ExchangeSession  - Unable to parse BackOffMilliseconds For input string: "Name: BackOffMilliseconds179846"
2021-04-11 12:49:53,891 WARN  [ImapConnection-56471] davmail.exchange.ExchangeSession  - Throttling active on server, waiting 60 seconds
2021-04-11 12:49:53,901 ERROR [ImapConnection-56357] davmail  - The server cannot service this request right now. Try again later.
davmail.exchange.ews.EWSThrottlingException: The server cannot service this request right now. Try again later.
        at davmail.exchange.ews.EWSMethod.checkSuccess(EWSMethod.java:766)
        at davmail.exchange.ews.EwsExchangeSession.internalExecuteMethod(EwsExchangeSession.java:2975)
        at davmail.exchange.ews.EwsExchangeSession.executeMethod(EwsExchangeSession.java:2960)
        at davmail.exchange.ews.EwsExchangeSession.searchItems(EwsExchangeSession.java:728)
        at davmail.exchange.ews.EwsExchangeSession.searchItems(EwsExchangeSession.java:661)
        at davmail.exchange.ews.EwsExchangeSession.searchMessages(EwsExchangeSession.java:645)
        at davmail.exchange.ExchangeSession.searchMessages(ExchangeSession.java:340)
        at davmail.exchange.ExchangeSession$Folder.loadMessages(ExchangeSession.java:1210)
        at davmail.exchange.ExchangeSession.refreshFolder(ExchangeSession.java:908)
        at davmail.imap.ImapConnection.run(ImapConnection.java:585)

Better would be to use the Subscribe operation, where you can specify either a list of folders to watch (possibly just the current folder), or set the SubscribeToAllFolders="true" attribute to watch all folders.

The Subscribe operation allows either pull or push notification.

For pull method, you call GetEvents with the token returned by the Subscribe operation. This seems to be a much lighter weight operation on the server, and should make the server much happier, allowing more frequent polling without making the server angry, and it should allow DavMail to perform better, as well.

For push notification, you give a local web service endpoint where Exchange can send the notifications. This would be great, and allow immediate notifications without having to poll the Exchange server. Not everyone would be able/willing to open a local port, but for those of us who are willing, it would be a very nice improvement.

[Mickael Guessant]

You are right implementing an advanced polling would help, however I do not yet know how this works.
IMHO the push mode is not an option: would require a direct access from Exchange/O365 to DavMail which would not work for most DavMail deployments, and this mechanism is linked to recent security vulnerability in Exchange: attacker can trigger a NTLM authentication from Exchange to the rogue endpoint to retrieve Exchange service credentials.

[Dominic Rodriguez]

What about 'pull' functionality - would that be preferable to push?