Re: [Davmail-users] davmail scalability?
Brought to you by:
mguessan
Menu
▾
▴
Re: [Davmail-users] davmail scalability?
|
From: John H. <a27...@sn...> - 2016-12-10 20:14:12
|
I meant to say that the slow message retrieval from davmail may be a separate issue than the issue where lots of sockets are stuck in a CLOSE_WAIT state and davmail seems to be deadlocked. The former (slow message retrieval) always seems to be true (even with just a single client connected) while the latter (deadlock / sockets not closing) occurs intermittently based on some trigger condition (but seems related to having more clients connected). I meant to brush up on java debugging and try to get more details before reporting these issues. But now that someone else has reported similar trouble with multiple connected clients, I am reporting my experience as well. Also, if there's a way to determine conclusively if throttling is going on, please let me know. Patches to print extra debug welcome. I'll attach some davmail.log output when I get a chance, but I don't see anything at all when davmail appears to be deadlocked. I ran this on a single CPU machine for a while, but recently moved it to a 4-cpu host without seeing any appreciable behavior change. John Hein wrote at 12:35 -0700 on Dec 10, 2016: > I'll second this. > > I have a davmail running in server mode (on freebsd). > > When all four of the following clients are connected, davmail goes > into a mode where there are over 80 sockets that never clean up and > davmail is catatonic [1]. > > I don't think it's a throttling issue here. The back end is a local > Exchange server. > > There are some large mailboxes involved (40000 messages spread over a > year). > > It _always_ seems much slower than for instance, going offsite to a > gmail IMAP server. For instance, getting 50 messages will take a few > minutes from davmail (all servers are local) while getting 50 message > from gmail (via somewhat slower internet connection) takes maybe > 10-ish seconds. > > Here is what is connected to the davmail server all via IMAP: > > - two thunderbird clients > - one getmail poller that runs once every two minutes > - one k9-mail (android) client > > [1] > > Here is a healthy netstat output (3 clients connected to davmail's > imap port - 1143): > > tcp4 0 0 192.168.100.100.1143 192.168.100.156.19391 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.38647 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.146.50294 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18091 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18090 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18089 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18087 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18086 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18085 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18084 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18083 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.156.18082 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.146.17780 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.146.48471 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.146.63368 ESTABLISHED > tcp4 0 0 192.168.100.100.1143 192.168.100.209.31715 ESTABLISHED > tcp46 0 0 *.1143 *.* LISTEN > > When it is unhealthy, there are 80+ 1143 sockets, most of which are in > the CLOSE_WAIT state. They never clean up until davmail is restarted. > Then everything is happy again for a while. It can remain happy for > days, but if one of the clients goes away and reconnects (e.g., the k9 > client on the android phone), things go bad again. I'm not sure exactly > what the trigger is, but it can get to the fail state in a couple hours. > > > Mickaël Guessant wrote at 19:41 +0100 on Dec 10, 2016: > > DavMail is definitely multithreaded, even in single user mode: each > > connection gets its own thread. > > > > Each user hase a separate HTTP connection pool, by default with a > > maximum of 100 slots > > => you can change this with davmail.exchange.maxConnections > > > > So basically Emilio is right: you probaby hit Exchange throttling on the > > server side > > > > DavMail now detects Exchange throttling and suspends all connections > > waiting for server provided delay > > => Look for "Throttling active on server, waiting" messages in davmail.log > > > > Regards, > > > > > > Le 25/11/2016 à 13:54, Emilio Recio a écrit : > > > What's on the back end? If it's Office365 (EWS) cloud then it's on the > > > microsoft side throttling per connection & per user to sub-modem speeds. > > > I tend to use fetchmail to grab all the mail on a regular basis through > > > DavMAIL, then run dovecot server and use that as a real server. It > > > doesn't help with calendaring, but some amalgamation of scripts and an > > > office open groupware server would provide similar standardized services. > > > > > > HTH > > > > > > > > > On 11/15/16 19:52, Pet...@da... wrote: > > >> Hi, > > >> I'm attempting to run davmail in server mode for a small > > >> workgroup. About the time we got our tenth user, davmail > > >> became very very slow. It takes minutes to send an email. > > >> > > >> top shows it's using less than 5% of a processor; there is > > >> over a gigabyte of free RAM; the only disk I/O is to davmail's > > >> logs. > > >> > > >> Is the davmail server single-threaded? Or does it use a > > >> thread-per-connection? > > >> > > >> > > >> > > >> > > > The information contained in this transmission contains privileged and confidential information. It is intended only for the use of the person named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. > > > > > > CAUTION: Intended recipients should NOT use email communication for emergent or urgent health care matters. > > > > > > ------------------------------------------------------------------------------ > > > _______________________________________________ > > > Davmail-users mailing list > > > Dav...@li... > > > https://lists.sourceforge.net/lists/listinfo/davmail-users > > > > > > > > > -- > > Mickael Guessant > > mailto:mgu...@fr... > > > > > > ------------------------------------------------------------------------------ > > Developer Access Program for Intel Xeon Phi Processors > > Access to Intel Xeon Phi processor-based developer platforms. > > With one year of Intel Parallel Studio XE. > > Training and support from Colfax. > > Order your platform today.http://sdm.link/xeonphi > > _______________________________________________ > > Davmail-users mailing list > > Dav...@li... > > https://lists.sourceforge.net/lists/listinfo/davmail-users > > ------------------------------------------------------------------------------ > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today.http://sdm.link/xeonphi > _______________________________________________ > Davmail-users mailing list > Dav...@li... > https://lists.sourceforge.net/lists/listinfo/davmail-users |
