Re: [Assp-user] HMM-Check has given less than 6 results - using monitoring mode only

[Thomas E. <Tho...@th...>]

>I'm really not thrilled with BerkeleyDB
Why you use a DB engine, that you don't know?

>SQLite3

You may use any ANSI-SQL (RDB) database engine you know (and perl has a 
driver for).

>I'm not keen on setting up a full-blown MySQL instance just
>for this, if I can avoid it.

But keep in mind! ASSP is not lightweight database user, if HMM and 
Bayesian are used. It may possible, that 7 workers (each doing 600 SQL 
queries in one second) are producing 4200 (or more) SQL queries in one 
second. We saw conditions, were 5 ASSP V2 instances brought a well 
designed (16 Cores, 200GB RAM, SSD) super big enterprise MySQL DB-server 
to its physical end!

It is wrong to think about assp as a "simple" spam filter. If you want to 
build an enterprise anti spam solution with assp - you WILL NEED:

enterprise hardware
enterprise software
enterprise IT knowledge

You may change the word "enterprise" to 'standard' , 'simple', 'advanced' 
.....- the relation will be the same. (I'm not sure, that 'simple' 
IT-knowkedge will be enough in any case)

Thomas



Von:    Dossy Shiobara <do...@pa...>
An:     For Users of ASSP <ass...@li...>
Datum:  02.02.2016 20:27
Betreff:        Re: [Assp-user] HMM-Check has given less than 6 results - 
using monitoring mode only



Okay, thank you for the instructions!  I've followed them, and here's
the result of the rebuildspamdb:

Feb-02-16 14:16:41 Spam Weight    :   3,226,691
Feb-02-16 14:16:41 Not-Spam Weight:   3,227,259

Feb-02-16 14:16:41 Corpus norm: 0.9998 - (very good - balanced)
Feb-02-16 14:16:41 Corpus confidence:   1.00000000

I'm really not thrilled with BerkeleyDB (I've ran into very specific
problems using it with a threaded Tcl).  Any chance I could use SQLite3
instead?  I'm not keen on setting up a full-blown MySQL instance just
for this, if I can avoid it.

Any ideas as to why the cleanup of old files takes so long (a fixed 10
minutes) or a way I can disable it and just clean out the files myself
with cron job instead?


On 2/1/16 11:48 PM, Thomas Eckardt wrote:
>> Jan-28-16 09:28:22 Spam Weight    :   3,264,527
>> >Jan-28-16 09:28:22 Not-Spam Weight:   3,265,258
>> >Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced)
>> >Jan-28-16 09:28:22 Corpus confidence:   0.06250000
> Corpus confidence:   0.06250000 - this value is impossible (expected is 
> 1.000) if  ->  Corpus norm: 0.9998 - (very good - balanced).
> I think your Berkeley-DB ENV or DB is damaged for some or all BDB files 
- 
> but at least for HMMdb.
>
> - shutdown assp
> - remove all files (__*.* , *.bdb)  from assp/tmpDB/HMMdb
> - do the same for spamdb
> - remove assp/hmmdb.bdb and assp/spamdb.bdb
> - start assp
> - import any avalable backup for both DB's - or run a rebuildspamdb
> - restart assp to force a recalculation of the used BDB cache
>
> If you use any *nix - KEEP in MIND! Your init.d script for assp (stop 
> case) has to wait until assp has been finished - otherwise your 
BerkeleyDB 
> files WILL BE DESTROYED!
> To be clear - I mean 'WILL BE DESTROYED' - not 'may be' or 'possibly' 
Some 
> damaging of BDB files is fixed by an assp internal BDB repair mechanism 
- 
> some , not all!

-- 
Dossy Shiobara         |      "He realized the fastest way to change
do...@pa...     |   is to laugh at your own folly -- then you
http://panoptic.com/   |   can let go and quickly move on." (p. 70) 
  * WordPress * jQuery * MySQL * Security * Business Continuity *


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Assp-user mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-user






DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally 
privileged and protected in law and are intended solely for the use of the 

individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no 
known virus in this email!
*******************************************************