[Assp-test] Antwort: SMTP Timeout Problem

[Thomas Eckardt/e. <Tho...@th...>]

At this time there is a 'unsightly design' in ASSP 2, which can cause this 
delay on heavy load systems, if logging is expanded! The MainThread 
handles the initial connection requests and swap them in to the workers, 
but in addition to this 'work' the MainThread is doing the logging 
(STDOUT,maillog,syslog) for all Threads. So disable most of the logging 
options should help to solve this problem (connection log, session log, 
worker log). I hope I can change this!

Memory:

ASSP 2 uses a totaly different way to access the lists and caches 
(whitelist,redlist,spamdb,spamdb.helo,delaydb,ldaplist,penaltybox). ASSP 1 
uses orderedtie, which need less memory. ASSP 2 has to tie all lists and 
caches to all threads and for this reason all lists and caches are loaded 
in to memory! If you make a summary of the filesize of all lists and 
caches * 1.5 * (workernumber +3) you'll get the approximatly needed 
memory. We've made many test to find a better way to handle the memory. 
The best is to use a database for all lists and caches to reduce the 
memory. This will need some more memory for code, but much less memory for 
the data! For example: a windows system with 10 SMTP-workers on perl 5.8.8 
build 822 and more than 100.000 connection requests a day and a replicated 
database needs between 400 and 450 MB RAM for ASSP and 70MB for the mysql 
daemon!

Be careful using the OCR-Plugin. The PDF-to-text option is no problem, but 
using ImageMagic and tesseract on large images, could cause in a 'system 
out of memory' error. All compressed images have to be decompressed by 
ImageMagic to process them with tesseract. But a 2MB compressed jpeg could 
be 30MB decompressed. Doing OCR with tesseract on such a large image will 
freeze your system. Use 'ASSP_OCRocrmaxsize' to prevent that! 


monitoring ASSP:

Try to reduce the monitoring requests (for example nagios check on port 25 
using telnet) to a minimum. Any connection request to ASSP cause a memory 
reservation insite ASSP for this new connection. So monitoring ASSP one 
time every second on port 25, cause 86.400 reservation requests a day!

Thomas 




Maxwell Rathbone <mra...@sa...> 
Gesendet von: ass...@li...
20.08.2008 17:57
Bitte antworten an
ASSP development mailing list <ass...@li...>


An
ASSP development mailing list <ass...@li...>
Kopie

Thema
[Assp-test] SMTP Timeout Problem






Description:
Long delay, or no answer at all when telnet'ing to port 25 (ASSP).
Performing a telnet to port 125 has the mail server answer immediately.

This is a very troublesome problem as it is not only causing our 
monitoring system to go crazy, but it is causing customers to get upset 
as they cannot connect to the mail server to send mail. It is worth 
noting that the mail server itself is working great and answers to port 
125 immediately. It is ASSP that is causing the delay.

At this point in time, the assp.pl process is utilizing 55.7% of the 
systems total memory(2GB) and that appears to be rising the longer it is 
running. This may be the cause of the delay, or possibly a symptom of 
the greater problem.

-------------------------------------------------------------------------------------------
Server Information:
OS & Perl Information:
[root@mail assp]# cat /etc/redhat-release
CentOS release 5.2 (Final)
[root@mail assp]# rpm -qa | grep kernel-
kernel-2.6.18-53.1.21.el5
[root@mail assp]# rpm -qa | grep perl-5
perl-5.8.8-10.el5_2.3

*snippet of log*
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request
Aug 20 11:52:46 mail assp[22178]: [Worker_2] info: Worker_2 is 
interrupted to get new connection
Aug 20 11:52:46 mail assp[22178]: [Worker_2] Info: Worker_2 got 
connection from MainThread
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread freed 
by interrupted Worker_2
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request
Aug 20 11:52:46 mail assp[22178]: [Worker_4] Info: Worker_4 is 
interrupted to get new connection
Aug 20 11:52:46 mail assp[22178]: [Worker_4] Info: Worker_4 got 
connection from MainThread
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread freed 
by interrupted Worker_4
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request
Aug 20 11:52:46 mail assp[22178]: [Worker_4] Info: Worker_4 is 
interrupted to get new connection
Aug 20 11:52:46 mail assp[22178]: 47560-04105 [Worker_5] 119.112.90.59 
<wgk...@ma...> to: co...@gr... recipient delayed: 
co...@gr...
Aug 20 11:52:46 mail assp[22178]: 47560-04105 [Worker_5] [SMTP Error] 
451 4.7.1 Please try again later
Aug 20 11:52:46 mail assp[22178]: [Worker_5] Eb...@md... matches 
@mdrg.com in LocalAddresses_Flat
Aug 20 11:52:46 mail assp[22178]: [Worker_4] Info: Worker_4 got 
connection from MainThread
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread freed 
by interrupted Worker_4
Aug 20 11:52:46 mail assp[22178]: 47564-07162 [Worker_1] 195.18.33.8 
<omc...@br...> to: ba...@gr... recipient 
delayed: ba...@gr...
Aug 20 11:52:46 mail assp[22178]: 47564-07162 [Worker_1] [SMTP Error] 
451 4.7.1 Please try again later
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request
Aug 20 11:52:46 mail assp[22178]: [Worker_2] Info: Worker_2 is 
interrupted to get new connection
Aug 20 11:52:46 mail assp[22178]: [Worker_2] Info: Worker_2 got 
connection from MainThread
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread freed 
by interrupted Worker_2
Aug 20 11:52:46 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request
Aug 20 11:52:46 mail assp[22178]: 47564-04711 [Worker_3] 201.25.88.190 
<nic...@co...> to: o.m...@pm... recipient delayed: 
o.m...@pm...
Aug 20 11:52:46 mail assp[22178]: 47564-04711 [Worker_3] [SMTP Error] 
451 4.7.1 Please try again later
Aug 20 11:52:46 mail assp[22178]: [Worker_2] Ws...@me... 
matches @mega-properties.com in LocalAddresses_Flat
Aug 20 11:52:47 mail assp[22178]: 47566-10548 [Worker_2] 66.227.102.59 
<Loo...@ho...> to: ws...@me... 
recipient delayed: ws...@me...
Aug 20 11:52:47 mail assp[22178]: 47566-10548 [Worker_2] [SMTP Error] 
451 4.7.1 Please try again later
Aug 20 11:52:47 mail assp[22178]: [Worker_5] eb...@md... matches 
@mdrg.com in LocalAddresses_Flat
Aug 20 11:52:47 mail assp[22178]: [Worker_1] Info: Worker_1 is 
interrupted to get new connection
Aug 20 11:52:47 mail assp[22178]: [Worker_1] Info: Worker_1 got 
connection from MainThread
Aug 20 11:52:47 mail assp[22178]: [Main_Thread] Info: Main_Thread freed 
by interrupted Worker_1
Aug 20 11:52:47 mail assp[22178]: [Main_Thread] Info: Main_Thread got 
connection request


Max

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's 
challenge
Build the coolest Linux based applications with Moblin SDK & win great 
prizes
Grand prize is a trip for two to an Open Source event anywhere in the 
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Assp-test mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-test




DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally 
privileged and protected in law and are intended solely for the use of the 

individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no 
known virus in this email!
*******************************************************