Re: [Assp-user] HMM-Check has given less than 6 results - using monitoring mode only

[Alexandre de A. P. <ada...@gm...>]

Hi,

If you use a database (like mysql), search in maillog if this records was
tranfered correctly after the rebuilddb terminate.
Here, if this occurs, the message is the same as yours.

2016-01-28 13:54 GMT-02:00 Dossy Shiobara <do...@pa...>:

> Also: I have BayesAfterHMM blank, and the log doesn't show any Bayes
> scoring happening (I have DoHMM and DoBayesian both set to "score").
>
>
> On 1/28/16 10:48 AM, Dossy Shiobara wrote:
> > Recently, it seems my HMM and Bayes checks are no longer working?  In
> > mail log, I see:
> >
> > "HMM-Check has given less than 6 results - using monitoring mode only"
> >
> > I'll include my latest rebuildrun.txt, which looks like it ran
> successfully.
> >
> > Why is this happening?  I'm running ASSP 2.4.7(16004).  Also, it seems
> > like if I get this error, it doesn't even perform Bayesian scoring --
> > basically, spam that was previously being blocked is now being let
> > through...
> >
> >
> > ---rebuildrun.txt---
> >
> > Jan-28-16 09:05:00 RebuildSpamDB-thread rebuildspamdb-version 7.26
> > started in ASSP version 2.4.7(16004)
> >
> > Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB for temporary hashes
> >
> > Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte
> >
> > Jan-28-16 09:05:00 RebuildSpamDB will create a Hidden Markov Model
> >
> > Jan-28-16 09:05:00 RebuildSpamDB will create unicode enabled databases
> >
> > Jan-28-16 09:05:00 RebuildSpamDB will process all words as Sequence of
> > UAX #29 Grapheme Clusters
> >
> > Jan-28-16 09:05:00 RebuildSpamDB will normalize unicode characters
> >
> > Jan-28-16 09:05:00 RebuildSpamDB will use the ASSP_WordStem engine
> >
> > Jan-28-16 09:05:00 ---ASSP Settings---
> > Jan-28-16 09:05:00 Do Not Collect Messages with RedListed address:
> Enabled
> > **Messages with RedListed addresses will be removed from the corpus!**
> >
> > Jan-28-16 09:05:00 Do Not Collect RedRe Messages: Enabled
> > **Messages matching the RedRe will be removed from the corpus!**
> >
> > Jan-28-16 09:05:00 Use Subject as Maillog Names: True
> > Jan-28-16 09:05:00 Maxbytes: 4,000
> > Jan-28-16 09:05:00 RebuildFileTimeLimit: 1 5
> > Jan-28-16 09:05:00 RebuildFileTimeLimit: files will be moved away from
> > the corpus if their processing takes longer than 5 second(s)
> >
> > Jan-28-16 09:05:00 /data/assp/errors/spam
> > Jan-28-16 09:05:00 File Count:  11
> > Jan-28-16 09:05:00 Processing... errors/spam with 11 files
> > Jan-28-16 09:05:00 ignore and remove files older than Sep-11-88 10:05:00
> > in folder errors/spam
> > Jan-28-16 09:05:01 Imported Files for HeloBlackList:    10
> > Jan-28-16 09:05:01 Imported Files for Bayes/HMM:        10
> > Jan-28-16 09:05:01 Finished in 1 second(s)
> >
> > Jan-28-16 09:05:01 /data/assp/errors/notspam
> > Jan-28-16 09:05:01 File Count:  1
> > Jan-28-16 09:05:01 Processing... errors/notspam with 1 files
> > Jan-28-16 09:05:01 ignore and remove files older than Sep-11-88 10:05:01
> > in folder errors/notspam
> > Jan-28-16 09:05:01 Imported Files for HeloBlackList:    0
> > Jan-28-16 09:05:01 Imported Files for Bayes/HMM:        0
> > Jan-28-16 09:05:01 Finished in 1 second(s)
> > Jan-28-16 09:05:01 info: corpusnorm after processing errors/spam and
> > errors/notspam is Spam Weight: 8280 / Not-Spam Weight: 0 => norm: 10.000
> > Jan-28-16 09:05:01 info: require approx. 6,726 files (3,255,584 words)
> > from folder spam to get the wanted corpusnorm (1.000)
> >
> > Jan-28-16 09:05:01 /data/assp/spam
> > Jan-28-16 09:05:01 File Count:  11,195
> > Jan-28-16 09:05:01 Processing... spam with 11,195 files
> > Jan-28-16 09:05:01 ignore and remove files older than Dec-28-15 09:05:01
> > in folder spam
> > Jan-28-16 09:15:31 Removed Old: 5
> > Jan-28-16 09:15:31 Imported Files for HeloBlackList:    11,190
> > Jan-28-16 09:15:31 Imported Files for Bayes/HMM:        6,672
> > Jan-28-16 09:15:31 Finished in 630 second(s)
> > Jan-28-16 09:15:31 info: require approx. all files (3,264,527 words)
> > from folder notspam to get the wanted corpusnorm (1.000)
> >
> > Jan-28-16 09:15:31 /data/assp/notspam
> > Jan-28-16 09:15:31 File Count:  7,009
> > Jan-28-16 09:15:31 Processing... notspam with 7,009 files
> > Jan-28-16 09:15:31 ignore and remove files older than Dec-28-15 09:15:31
> > in folder notspam
> > Jan-28-16 09:25:53 Removed Old: 7
> > Jan-28-16 09:25:53 Imported Files for HeloBlackList:    7,002
> > Jan-28-16 09:25:53 Imported Files for Bayes/HMM:        6,992
> > Jan-28-16 09:25:53 Finished in 622 second(s)
> >
> > Jan-28-16 09:25:53 Generating weighted Bayesian tuplets
> > Jan-28-16 09:26:10 populating Spamdb 503166 records - Bayesian check is
> > now disabled
> > Jan-28-16 09:26:23 done - populating Spamdb records - 503166 - Bayesian
> > check is now enabled
> > Jan-28-16 09:26:23 done - Generating weighted Bayesian tuplets
> >
> > Jan-28-16 09:26:23 Bayesian Pairs: 503,166 now in list
> >
> > Jan-28-16 09:26:23 Generating consolidated Hidden-Markov-Model database
> > from 3,772,337 record model
> > Jan-28-16 09:28:22 HMM sequences: 1,848,357 now in list
> >
> > Jan-28-16 09:28:22 generating Spamdb.helo records from 7,502 collected
> > HELO's
> > Jan-28-16 09:28:22 cleaning old Spamdb.helo records
> > Jan-28-16 09:28:22 done - cleaning old Spamdb.helo records
> >
> > Jan-28-16 09:28:22 HELO Blacklist: 4 new, 0 now in list
> >
> > Jan-28-16 09:28:22 Spam Weight    :   3,264,527
> > Jan-28-16 09:28:22 Not-Spam Weight:   3,265,258
> >
> > Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced)
> > Jan-28-16 09:28:22 Corpus confidence:   0.06250000
> >
> > Jan-28-16 09:28:27 Start populating Hidden Markov Model. HMM-check is
> > disabled for this time!
> > Jan-28-16 09:28:27 start populating Hidden Markov Model with 1,848,357
> > records!
> > Jan-28-16 09:28:59 Finished populating Hidden Markov Model with
> > 1,848,357 records!
> > Jan-28-16 09:28:59 Finished populating Hidden Markov Model. HMM-check is
> > now enabled again!
> >
> > Jan-28-16 09:28:59 Total processing time: 1,439 second(s)
> >
> > Jan-28-16 09:28:59 Total processing data: 118.85 MByte
> >
> >
> > Jan-28-16 09:28:59 Rebuild processed 14.52 files per second.
> >
> > Jan-28-16 09:28:59 After finishing the Rebuild process, the
> > /data/assp/tmpDB folder contains 791.45 MByte.
> >
> > Jan-28-16 09:28:59 After finishing the Rebuild process, the drive that
> > contains the /data/assp/tmpDB folder has 1.22 GByte free space from
> > total 1.90 GByte.
> >
> > Jan-28-16 09:28:59 building new GripList records and bounce report
> > Jan-28-16 09:28:59 processing Logfile /data/assp/logs/maillog.txt
> > Jan-28-16 09:28:59 processing Logfile
> /data/assp/logs/16-01-27.maillog.txt
> > Jan-28-16 09:29:01 processing Logfile
> /data/assp/logs/16-01-26.maillog.txt
> > Jan-28-16 09:29:02 processing Logfile
> /data/assp/logs/16-01-25.maillog.txt
> > Jan-28-16 09:29:03 processing Logfile
> /data/assp/logs/16-01-24.maillog.txt
> > Jan-28-16 09:29:03 processing Logfile
> /data/assp/logs/16-01-23.maillog.txt
> >
> > Jan-28-16 09:29:03 skipping bounce report because 'DoNotCollectBounces'
> > is switched ON
> >
> > Jan-28-16 09:29:03 Uploading Griplist via Direct Connection
> > Jan-28-16 09:29:04 Submitted 6,924 bytes: 0 IPv6 addresses, 768 IPv4
> > addresses
> >
> > Jan-28-16 09:29:04 Trashlist was saved to /data/assp/trashlist.db
> >
>
> --
> Dossy Shiobara         |      "He realized the fastest way to change
> do...@pa...     |   is to laugh at your own folly -- then you
> http://panoptic.com/   |   can let go and quickly move on." (p. 70)
>   * WordPress * jQuery * MySQL * Security * Business Continuity *
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Assp-user mailing list
> Ass...@li...
> https://lists.sourceforge.net/lists/listinfo/assp-user
>