Menu
▾
▴
Re: [Assp-test] workers get stuck very frequently (makeSubject)
|
From: <mx...@ma...> - 2010-10-31 12:03:00
|
Hi Thomas, thank you very much for taking your time! Since I added this one sender to nop, thankfully no worker got stuck again.. Apart from that, once again thank you and all other devs so much for this great piece of work! It's only been a week, but still, I'm extremely happy to have only 2 unrecognised spam mails and no false positives out of about 300 spam mails/day since the upgrade. It just plain and simple: kicks ass! :) On 10/31/2010 09:25 AM, Thomas Eckardt wrote: >> 27-10-10 11:07:02 [startup] Info: plugin ASSP_SkeletonTest version 1.06 >> loaded for runlevel 'mail header'. > Remove this Plugin !!!! :D After my first post to the list about this issue, I reviewed my configuration and plugins and noticed that the skeleton plugin being loaded in my new install. Removed it, but didn't help >> 27-10-10 11:07:10 [init] BerkeleyDB DB-version 4.6 / Berkeley DB 4.6.21: >> (September 27, 2007) is installed > 4.6 - was never tested and is theoretical (!) supported by the feature > description of oracle - upgrade your BerkeleyDB engine to 5.x.x > I will install 5.x. But also, after my first post, I removed BerkelyDB altogether since after each killing of a stuck worker there were errors saving cache to berkelydb. Didn't help either. >> using Perl /opt/.perl5/perlbrew/perls/perl-5.12.2/bin/perl5.12.2 version >> 5.012002 (5.12.2) > Perl 5.12 should be OK, but keep in mind - that assp is developed on Perl > 5.10 - it could be possible that I can't help you - if the changed regex > processing of Perl 5.12 will make problems. Ok. In fact I'm only using 5.12 because of assp.. After reading some posts of yours regarding unicode bugs and 'complex AND-NOT' regex could cause stack overflows I got the idea that it might be best to go for 5.12 straight up from the beginning. Do you also recommend sticking to 5.10 for the latest dev version to be more safe? > makeSubject parses the mail header and stores different versions > (encoded/decoded/nomalized/original) of the subject in variables for the > further use in ASSP. > The problematic contents could be anywhere in the mail from 0 to maxBytes > (+ some bytes). It is strange that the mail is processed by assp if you > set it to noprocessing, because 'makeSubject' will be used (called) in > every case. > Strange, but remembers me of an issue I had with stuck workers many months ago, where last message was LocalLDAPDomain each time. I tried very much and removed ldap checks altogether, but at the end I found out it was caused by some really bad and stupid mistake I had done in the uribl config (socket timeout was set to an insanely high value.) BTW: about the razor plugin: I wanted to try that out, when I look at the release notes for version 2.0.1 it's mentioned, but it doesn't load because it says version 2.02 is needed.. is this a mistake, or is there some other razor plugin around just for 2.0.1? Thanks Thomas, I'll send you the mail now. Max > Von: "mx...@ma..."<mx...@ma...> > An: ASSP development mailing list<ass...@li...> > Datum: 27.10.2010 15:36 > Betreff: [Assp-test] workers get stuck very frequently > (makeSubject) > > > > > Hi, > > it would be great if someone could help me out with this. > > I'm currently running 2.0.1(1.2.21). BTW: does this version belong to > assp-user or assp-test? > > Unfortunately since installing that version I get stuck workers at least > once per hour: > > These are the corresponding log lines: > > 27-10-10 14:46:19 [Worker_2] LDAP - found<user>@<mydomain> in LDAPlist > > 27-10-10 14:50:10 [Main_Thread] Info: Loop in Worker_2 was not active > for 231 seconds > 27-10-10 14:50:10 [Main_Thread] Info: Worker_2 : last sigoff in main, > /usr/share/assp/assp.pl, 10080, main::sigoffTry, 1, , , at 10-27-9 > 14:4619 1288183579.45356 - 10080 > 27-10-10 14:50:10 [Main_Thread] Info: Worker_2 : last sigon in main, > /usr/share/assp/assp.pl, 10082, main::sigonTry, 1, , , at 10-27-9 > 14:4619 1288183579.45363 - 10082 > 27-10-10 14:50:10 [Main_Thread] Info: Worker_2 : last action was : > makeSubject > > What exactly does makeSubject do? Can I somehow use this info to find > out what assp was doing when this worker got stuck? > > Thank you very much! > > Max > > > ASSP startup: > > ASSP 2.0.1(1.2.21) is starting in directory /usr/share/assp > on host filter > using Perl /opt/.perl5/perlbrew/perls/perl-5.12.2/bin/perl5.12.2 version > 5.012002 (5.12.2) > > loading configuration [OK] > 1010 values loaded [OK] > defining environment [OK] > loading modules......... [OK] > loading database drivers [OK] > setup regular expressions [OK] > loading plugins [OK] > fixing up config [OK] > rendering HTML for GUI [OK] > check process env [SKIP] > check process permission [OK] > setting up modules...... [OK] > checking directories [OK] > check file permission [OK] > loading caches and lists [OK] > starting maintenance worker thread [OK] > starting rebuild SpamDB worker thread [OK] > starting 5 communication worker threads ...... [OK] > starting main thread and logging [OK] > 27-10-10 11:07:02 [startup] Starting in console mode > 27-10-10 11:07:02 [startup] ASSP-professional version 2.0.1(1.2.21) > (Perl 5.012002) (on linux) initializing > 27-10-10 11:07:02 [startup] Info: try loading plugin ASSP_AFC > 27-10-10 11:07:02 [startup] ASSP_AFC: Plugin successful called! > 27-10-10 11:07:02 [startup] Info: plugin ASSP_AFC version 1.18 loaded > for runlevel 'complete mail'. > 27-10-10 11:07:02 [startup] Info: try loading plugin ASSP_ARC > 27-10-10 11:07:02 [startup] ASSP_ARC: ARC-Plugin successful called! > 27-10-10 11:07:02 [startup] Info: plugin ASSP_ARC version 1.14 loaded > for runlevel 'complete mail'. > 27-10-10 11:07:02 [startup] Info: try loading plugin ASSP_OCR > 27-10-10 11:07:02 [startup] ASSP_OCR: Plugin successful called for > runlevel 'complete mail' using PDF::OCR2! > 27-10-10 11:07:02 [startup] Info: plugin ASSP_OCR version 1.25 loaded > for runlevel 'complete mail'. > 27-10-10 11:07:02 [startup] Info: try loading plugin ASSP_Razor > 27-10-10 11:07:02 [startup] Info: try loading plugin ASSP_SkeletonTest > 27-10-10 11:07:02 [startup] ASSP_SkeletonTest: Plugin successful called! > 27-10-10 11:07:02 [startup] Info: plugin ASSP_SkeletonTest version 1.06 > loaded for runlevel 'mail header'. > 27-10-10 11:07:02 [startup] Info: no valid recipient replacement rule > found > 27-10-10 11:07:02 [startup] Info: Regex invalidFormatHeloRe: 3 weighted > regular expression defined > 27-10-10 11:07:10 [startup] Info: 3 MSGID-secrets activated > 27-10-10 11:07:10 [startup] Info: 3 BATV-secrets activated > 27-10-10 11:07:10 [startup] Info: registered encrypted POP3ConfigFile > file /usr/share/assp/files/pop3cfg.txt > 27-10-10 11:07:10 [startup] Info: an ASSP restart will be done with: > 27-10-10 11:07:10 [startup] ASSP_OCR: ImageMagick's convert found. > 27-10-10 11:07:10 [startup] ASSP version 2.0.1(1.2.21) (Perl 5.012002) > (on linux)running on server: filter (192.168.22.100) > 27-10-10 11:07:10 [init] Threads module 1.81 installed > 27-10-10 11:07:10 [init] Threads::shared module 1.34 installed > 27-10-10 11:07:10 [init] Thread::Queue module 2.11 installed > 27-10-10 11:07:10 [init] IO::Poll module 0.07 installed > 27-10-10 11:07:10 [init] IO::Select module 1.17 installed > 27-10-10 11:07:10 [init] ASSP is using IOEngine - Poll > 27-10-10 11:07:10 [init] Thread::State module version 0.09 installed and > available > 27-10-10 11:07:10 [init] File::Scan::ClamAV module version 1.91 > installed and available > 27-10-10 11:07:10 [init] Net::LDAP module version 0.4001 installed and > available > 27-10-10 11:07:10 [init] Net::DNS module version 0.66 installed and > available > 27-10-10 11:07:10 [init] Email::Valid module version 0.184 installed and > available > 27-10-10 11:07:10 [init] Net::SMTP module version 2.31 installed and > available > 27-10-10 11:07:10 [init] Mail::SPF::Query module version 1.999001 > installed and available > 27-10-10 11:07:10 [init] Mail::SPF module version 2.007 installed and > available > 27-10-10 11:07:10 [init] Mail::SRS module version 0.31 installed - > Sender Rewriting Scheme available > 27-10-10 11:07:10 [init] Compress::Zlib module version 2.03 installed - > HTTP compression available > 27-10-10 11:07:10 [init] Digest::MD5 module version 2.51 installed - > delaying can use MD5 keys for hashes > 27-10-10 11:07:10 [init] Digest::SHA1 module version 2.13 installed - > BATV and FBMTV check available > 27-10-10 11:07:10 [init] File::ReadBackwards module version 1.04 > installed - searching of log files enabled > 27-10-10 11:07:10 [init] Time::HiRes module version 1.9721 installed - > CPU usage statistics available > 27-10-10 11:07:10 [init] Sys::Syslog module version 0.27 installed - > Unix centralized logging enabled > 27-10-10 11:07:10 [init] Net::Syslog module version 0.03 installed - > network Syslog logging enabled > 27-10-10 11:07:10 [init] Tie::RDBM module version 0.73 installed - > database usage available > 27-10-10 11:07:10 [init] DB_File module version 1.82 installed - DB_File > (Berkeley V1) database usage available > 27-10-10 11:07:10 [init] BerkeleyDB module version 0.43 installed - > Berkeley database usage available > 27-10-10 11:07:10 [init] BerkeleyDB DB-version 4.6 / Berkeley DB 4.6.21: > (September 27, 2007) is installed > 27-10-10 11:07:10 [init] Info: griplist is using 'BerkeleyDB' version > 4.6 in file /usr/share/assp/griplist.bdb > 27-10-10 11:07:10 [init] Net::IP::Match::Regexp module version 1.01 > installed - CIDR notation for IP range available > 27-10-10 11:07:10 [init] Net::CIDR::Lite module version 0.21 installed - > hyphenated IP address range available > 27-10-10 11:07:10 [init] Net::SenderBase module version 1.01 installed - > Query the senderbase service available > 27-10-10 11:07:10 [init] LWP::Simple module version 5.835 installed - > procedural LWP interface available > 27-10-10 11:07:10 [init] Email::MIME::Modifier module version 1.906 > installed - MIME charset decoding and conversion interface and > attachment detection available > 27-10-10 11:07:10 [init] MIME::Types module version 1.31 installed - > TNEF conversion may possible > 27-10-10 11:07:10 [init] Email::Send module version 2.198 installed - > sending .eml files available > 27-10-10 11:07:10 [init] Convert::TNEF module version 0.17 installed - > TNEF conversion is available > 27-10-10 11:07:10 [init] Mail::DKIM::Verifier module version 0.38 > installed - DKIM verification is available > 27-10-10 11:07:10 [init] Schedule::Cron module version 1.00 installed - > RebuildSpamdb Scheduler is available > 27-10-10 11:07:10 [init] Sys::MemInfo module version 0.91 installed - > memory calculation is available > 27-10-10 11:07:10 [init] Authen::SASL module version 2.15 installed - > SMTP AUTH is available > 27-10-10 11:07:10 [init] Regex::Optimizer module version 1.07 installed > - Regular Expression Optimization is available > 27-10-10 11:07:10 [init] IO::Socket::SSL module version 1.33 installed - > https and TLS/SSL is possible > 27-10-10 11:07:10 [init] Found valid certificate and privat key file - > https and TLS/SSL is available > 27-10-10 11:07:10 [init] Info: last DB-backup was scheduled before 1 > hour 14 mins > 27-10-10 11:07:10 [init] Info: next DB-backup is scheduled in 45 mins > 27-10-10 11:07:10 [init] Starting maintenance worker thread [10000] - > ThreadCycleTime is set to 3000 microseconds > 27-10-10 11:07:12 [Worker_10000] Worker_10000 started > 27-10-10 11:07:12 [init] Starting rebuild SpamDB worker thread [10001] - > ThreadCycleTime is set to 30 microseconds > 27-10-10 11:07:12 [Worker_10001] Worker_10001 started > 27-10-10 11:07:12 [Worker_10001] Info: found module > /usr/share/assp/lib/rebuildspamdb.pm version 2.34 > 27-10-10 11:07:12 [init] Starting SMTP-worker-threads with > ThreadCycleTime set to 3000 microseconds > 27-10-10 11:07:12 [init] Starting communication worker threads [1 to 5] > 27-10-10 11:07:17 [init] Using table<whitelist> in mysql Database > <assp> instead of file /usr/share/assp/whitelist > 27-10-10 11:07:17 [init] Using table<persblack> in mysql Database > <assp> instead of file /usr/share/assp/persblack > 27-10-10 11:07:17 [init] Using table<redlist> in mysql Database > <assp> instead of file /usr/share/assp/redlist > 27-10-10 11:07:17 [init] Using table<delaydb> in mysql Database > <assp> instead of file /usr/share/assp/delaydb > 27-10-10 11:07:17 [init] Using table<delaywhitedb> in mysql Database > <assp> instead of file /usr/share/assp/delaydb.white > 27-10-10 11:07:17 [init] Using table<PBWhite> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.white.db > 27-10-10 11:07:17 [init] Using table<PBBlack> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.black.db > 27-10-10 11:07:17 [init] Using table<RBLCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.rbl.db > 27-10-10 11:07:17 [init] Using table<URIBLCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.uribl.db > 27-10-10 11:07:17 [init] Using table<PTRCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.ptr.db > 27-10-10 11:07:17 [init] Using table<MXACache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.mxa.db > 27-10-10 11:07:17 [init] Using table<RWLCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.rwl.db > 27-10-10 11:07:17 [init] Using table<SPFCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.spf.db > 27-10-10 11:07:17 [init] Using table<SBCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.sb.db > 27-10-10 11:07:17 [init] Using table<PBTrap> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.trap.db > 27-10-10 11:07:17 [init] Using table<DKIMCache> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.dkim.db > 27-10-10 11:07:17 [init] Using table<BATVTag> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.batv.db > 27-10-10 11:07:17 [init] Using table<BackDNS> in mysql Database > <assp> instead of file /usr/share/assp/pb/pbdb.back.db > 27-10-10 11:07:17 [init] Using table<spamdb> in mysql Database > <assp> instead of file /usr/share/assp/spamdb > 27-10-10 11:07:17 [init] Using table<spamdbhelo> in mysql Database > <assp> instead of file /usr/share/assp/spamdb.helo > 27-10-10 11:07:17 [init] Using table<ldaplist> in mysql Database > <assp> instead of file /usr/share/assp/ldaplist > 27-10-10 11:07:17 [init] Using table<AdminUsers> in mysql Database > <assp> instead of file /usr/share/assp/adminusers > 27-10-10 11:07:17 [Worker_10001] Info: starting RebuildSpamdb Scheduler > with '8 3 * * *' > 27-10-10 11:07:17 [init] Using table<AdminUsersRight> in mysql Database > <assp> instead of file /usr/share/assp/adminusers.right > 27-10-10 11:07:17 [init] Listening for SMTP connections on > 192.168.22.100:25 > 27-10-10 11:07:17 [init] Listening for SMTPS (SSL) connections on > 192.168.22.100:465 > 27-10-10 11:07:17 [init] Listening for admin HTTPS connections on 55555 > 27-10-10 11:07:17 [init] Listening for stat HTTP connections on 55553 > 27-10-10 11:07:17 [init] Listening for SMTP relay connections on > 0.0.0.0:25000 > 27-10-10 11:07:17 [init] Starting PID: 30447 > 27-10-10 11:07:17 [init] Info: command queue released > 27-10-10 11:07:17 [init] Switched effective gid to 1002 (nobody) > 27-10-10 11:07:17 [init] Switched real gid to 1002 (nobody) > 27-10-10 11:07:17 [init] Switched effective uid to 1001 (assp) > 27-10-10 11:07:17 [init] Switched real uid to 1001 (assp) > 27-10-10 11:07:17 [init] Info: central signalhandler - NUM63(63) - > TRAP(5) - NUM42(42) - URG(23) - STOP(19) - NUM39(39) - NUM62(62) - > NUM43(43) - NUM57(57) - NUM56(56) - RTMAX(64) - NUM59(59) - VTALRM(26) - > CONT(18) - NUM45(45) - NUM61(61) - NUM44(44) - NUM36(36) - NUM32(32) - > BUS(7) - NUM40(40) - NUM51(51) - IOT(65) - STKFLT(16) - NUM41(41) - > KILL(9) - QUIT(3) - NUM37(37) - NUM50(50) - ABRT(6) - NUM48(48) - > NUM35(35) - NUM38(38) - TTOU(22) - IO(29) - TSTP(20) - PROF(27) - > NUM53(53) - NUM58(58) - SEGV(11) - RTMIN(34) - POLL(67) - PIPE(13) - > SYS(31) - NUM46(46) - PWR(30) - FPE(8) - NUM54(54) - XCPU(24) - TTIN(21) > - NUM52(52) - NUM55(55) - XFSZ(25) - NUM33(33) - NUM49(49) - UNUSED(68) > - WINCH(28) - ILL(4) - NUM47(47) - NUM60(60) - INT(2) - HUP(1) - > TERM(15) - CHLD(17) - CLD(66) - ALRM(14) - installed > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > Assp-test mailing list > Ass...@li... > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > > > _______________________________________________ > Assp-test mailing list > Ass...@li... > https://lists.sourceforge.net/lists/listinfo/assp-test |
