[Assp-user] HMM-Check has given less than 6 results - using monitoring mode only

[Dossy S. <do...@pa...>]

Recently, it seems my HMM and Bayes checks are no longer working?  In
mail log, I see:

"HMM-Check has given less than 6 results - using monitoring mode only"

I'll include my latest rebuildrun.txt, which looks like it ran successfully.

Why is this happening?  I'm running ASSP 2.4.7(16004).  Also, it seems
like if I get this error, it doesn't even perform Bayesian scoring --
basically, spam that was previously being blocked is now being let
through...


---rebuildrun.txt---

Jan-28-16 09:05:00 RebuildSpamDB-thread rebuildspamdb-version 7.26
started in ASSP version 2.4.7(16004)

Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB for temporary hashes

Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte

Jan-28-16 09:05:00 RebuildSpamDB will create a Hidden Markov Model

Jan-28-16 09:05:00 RebuildSpamDB will create unicode enabled databases

Jan-28-16 09:05:00 RebuildSpamDB will process all words as Sequence of
UAX #29 Grapheme Clusters

Jan-28-16 09:05:00 RebuildSpamDB will normalize unicode characters

Jan-28-16 09:05:00 RebuildSpamDB will use the ASSP_WordStem engine

Jan-28-16 09:05:00 ---ASSP Settings---
Jan-28-16 09:05:00 Do Not Collect Messages with RedListed address: Enabled
**Messages with RedListed addresses will be removed from the corpus!**

Jan-28-16 09:05:00 Do Not Collect RedRe Messages: Enabled
**Messages matching the RedRe will be removed from the corpus!**

Jan-28-16 09:05:00 Use Subject as Maillog Names: True
Jan-28-16 09:05:00 Maxbytes: 4,000
Jan-28-16 09:05:00 RebuildFileTimeLimit: 1 5
Jan-28-16 09:05:00 RebuildFileTimeLimit: files will be moved away from
the corpus if their processing takes longer than 5 second(s)

Jan-28-16 09:05:00 /data/assp/errors/spam
Jan-28-16 09:05:00 File Count:  11
Jan-28-16 09:05:00 Processing... errors/spam with 11 files
Jan-28-16 09:05:00 ignore and remove files older than Sep-11-88 10:05:00
in folder errors/spam
Jan-28-16 09:05:01 Imported Files for HeloBlackList:    10
Jan-28-16 09:05:01 Imported Files for Bayes/HMM:        10
Jan-28-16 09:05:01 Finished in 1 second(s)

Jan-28-16 09:05:01 /data/assp/errors/notspam
Jan-28-16 09:05:01 File Count:  1
Jan-28-16 09:05:01 Processing... errors/notspam with 1 files
Jan-28-16 09:05:01 ignore and remove files older than Sep-11-88 10:05:01
in folder errors/notspam
Jan-28-16 09:05:01 Imported Files for HeloBlackList:    0
Jan-28-16 09:05:01 Imported Files for Bayes/HMM:        0
Jan-28-16 09:05:01 Finished in 1 second(s)
Jan-28-16 09:05:01 info: corpusnorm after processing errors/spam and
errors/notspam is Spam Weight: 8280 / Not-Spam Weight: 0 => norm: 10.000
Jan-28-16 09:05:01 info: require approx. 6,726 files (3,255,584 words)
from folder spam to get the wanted corpusnorm (1.000)

Jan-28-16 09:05:01 /data/assp/spam
Jan-28-16 09:05:01 File Count:  11,195
Jan-28-16 09:05:01 Processing... spam with 11,195 files
Jan-28-16 09:05:01 ignore and remove files older than Dec-28-15 09:05:01
in folder spam
Jan-28-16 09:15:31 Removed Old: 5
Jan-28-16 09:15:31 Imported Files for HeloBlackList:    11,190
Jan-28-16 09:15:31 Imported Files for Bayes/HMM:        6,672
Jan-28-16 09:15:31 Finished in 630 second(s)
Jan-28-16 09:15:31 info: require approx. all files (3,264,527 words)
from folder notspam to get the wanted corpusnorm (1.000)

Jan-28-16 09:15:31 /data/assp/notspam
Jan-28-16 09:15:31 File Count:  7,009
Jan-28-16 09:15:31 Processing... notspam with 7,009 files
Jan-28-16 09:15:31 ignore and remove files older than Dec-28-15 09:15:31
in folder notspam
Jan-28-16 09:25:53 Removed Old: 7
Jan-28-16 09:25:53 Imported Files for HeloBlackList:    7,002
Jan-28-16 09:25:53 Imported Files for Bayes/HMM:        6,992
Jan-28-16 09:25:53 Finished in 622 second(s)

Jan-28-16 09:25:53 Generating weighted Bayesian tuplets
Jan-28-16 09:26:10 populating Spamdb 503166 records - Bayesian check is
now disabled
Jan-28-16 09:26:23 done - populating Spamdb records - 503166 - Bayesian
check is now enabled
Jan-28-16 09:26:23 done - Generating weighted Bayesian tuplets

Jan-28-16 09:26:23 Bayesian Pairs: 503,166 now in list

Jan-28-16 09:26:23 Generating consolidated Hidden-Markov-Model database
from 3,772,337 record model
Jan-28-16 09:28:22 HMM sequences: 1,848,357 now in list

Jan-28-16 09:28:22 generating Spamdb.helo records from 7,502 collected
HELO's
Jan-28-16 09:28:22 cleaning old Spamdb.helo records
Jan-28-16 09:28:22 done - cleaning old Spamdb.helo records

Jan-28-16 09:28:22 HELO Blacklist: 4 new, 0 now in list

Jan-28-16 09:28:22 Spam Weight    :   3,264,527
Jan-28-16 09:28:22 Not-Spam Weight:   3,265,258

Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced)
Jan-28-16 09:28:22 Corpus confidence:   0.06250000

Jan-28-16 09:28:27 Start populating Hidden Markov Model. HMM-check is
disabled for this time!
Jan-28-16 09:28:27 start populating Hidden Markov Model with 1,848,357
records!
Jan-28-16 09:28:59 Finished populating Hidden Markov Model with
1,848,357 records!
Jan-28-16 09:28:59 Finished populating Hidden Markov Model. HMM-check is
now enabled again!

Jan-28-16 09:28:59 Total processing time: 1,439 second(s)

Jan-28-16 09:28:59 Total processing data: 118.85 MByte


Jan-28-16 09:28:59 Rebuild processed 14.52 files per second.

Jan-28-16 09:28:59 After finishing the Rebuild process, the
/data/assp/tmpDB folder contains 791.45 MByte.

Jan-28-16 09:28:59 After finishing the Rebuild process, the drive that
contains the /data/assp/tmpDB folder has 1.22 GByte free space from
total 1.90 GByte.

Jan-28-16 09:28:59 building new GripList records and bounce report
Jan-28-16 09:28:59 processing Logfile /data/assp/logs/maillog.txt
Jan-28-16 09:28:59 processing Logfile /data/assp/logs/16-01-27.maillog.txt
Jan-28-16 09:29:01 processing Logfile /data/assp/logs/16-01-26.maillog.txt
Jan-28-16 09:29:02 processing Logfile /data/assp/logs/16-01-25.maillog.txt
Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-24.maillog.txt
Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-23.maillog.txt

Jan-28-16 09:29:03 skipping bounce report because 'DoNotCollectBounces'
is switched ON

Jan-28-16 09:29:03 Uploading Griplist via Direct Connection
Jan-28-16 09:29:04 Submitted 6,924 bytes: 0 IPv6 addresses, 768 IPv4
addresses

Jan-28-16 09:29:04 Trashlist was saved to /data/assp/trashlist.db

-- 
Dossy Shiobara         |      "He realized the fastest way to change
do...@pa...     |   is to laugh at your own folly -- then you
http://panoptic.com/   |   can let go and quickly move on." (p. 70) 
  * WordPress * jQuery * MySQL * Security * Business Continuity *