crm114-general — General CRM114 issues, questions, and such

[Crm114-general] CRLF vs. LF: X-CRM114 header lines on the bottom of message?

[Simon <si...@4l...>]

###
Sorry, if this is a repost for some of you. Sourceforge's mailing list
manager cannot handle DMARC protected domains and my earlier mail
bounced with most recipients (Gmail, Yahoo, Microsoft, etc.). If you see
this message for the first time, you were one of them. Reposting from a
different RFC5322.From domain.
###

Hi list,

After a decade, I get to run a mail server again. Back then, I had used
DSPAM. Now, I want to give CRM114 a try. Excuse my newbie question.

I find all the X-CRM114 lines appended to the body of incoming emails.
That cannot be correct, or? Interestingly, it starts working if I remove
the CR from the RFC5322 email:

    $ file /tmp/SomeMail.rfc5322.eml
    /tmp/SomeMail.rfc5322.eml: SMTP mail, ASCII text, with CRLF line
    terminators

    $ wc -l /tmp/SomeMail.rfc5322.eml
    98 /tmp/SomeMail.rfc5322.eml

    $ cat /tmp/SomeMail.rfc5322.eml | ./mailreaver.crm | tail -n 6
    JTJFZGUmbXNnaWQ9JTNDMjAxNjExMTgyMDU3NDklMkVITSUyRTAwMDAwMDAwMDAwMDhqayU0
    MGF2YW5lZ2FzczA0JTJFd3dsMTcyOSUyRWhhbm1haWwlMkVuZXQlM0UiPgo=
    X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) )
    MR-27CA1CFB
    X-CRM114-CacheID: sfid-20161119_172554_829178_39744B64
    X-CRM114-Status: UNSURE (   0.00  )
    X-CRM114-Notice: Please train this message.

If I convert it to "unix style" line breaks it appends the X-CRM114
lines to the header. As it should be - I think:

    $ cat /tmp/SomeMail.rfc5322.eml|tr -d '\015'|./mailreaver.crm|grep
    "^$" -B4 -A3
    X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) )
    MR-27CA1CFB
    X-CRM114-CacheID: sfid-20161119_174156_811984_46A16809
    X-CRM114-Status: UNSURE (   0.00  )
    X-CRM114-Notice: Please train this message.

    PGh0bWw+PGhlYWQ+PHN0eWxlPiBwIHttYXJnaW4tdG9wOjBweDttYXJnaW4tYm90dG9tOjBw
    eDt9IDwvc3R5bGU+PC9oZWFkPgo8Ym9keT48ZGl2IHN0eWxlPSJmb250LXNpemU6MTJweDsg
    Zm9udC1mYW1pbHk66rW066a8LOq1tOumvOyytCxHdWxpbSxCYWVrbXVrIERvdHVtLFVuZG90

The mailreaver.crm is the unaltered one that shipped with Ubuntu 14.04.
Is this behavior normal or am I missing something?

Simon

[Crm114-general] CRLF vs. LF: X-CRM114 header lines on the bottom of message?

[Sim <si...@si...>]

Hi list,

After a decade, I get to run a mail server again. Back then, I had used
DSPAM. Now, I want to give CRM114 a try. Excuse my newbie question.

I find all the X-CRM114 lines appended to the body of incoming emails.
That cannot be correct, or? Interestingly, it starts working if I remove
the CR from the RFC5322 email:

    $ file /tmp/SomeMail.rfc5322.eml
    /tmp/SomeMail.rfc5322.eml: SMTP mail, ASCII text, with CRLF line
    terminators

    $ wc -l /tmp/SomeMail.rfc5322.eml
    98 /tmp/SomeMail.rfc5322.eml

    $ cat /tmp/SomeMail.rfc5322.eml | ./mailreaver.crm | tail -n 6
    JTJFZGUmbXNnaWQ9JTNDMjAxNjExMTgyMDU3NDklMkVITSUyRTAwMDAwMDAwMDAwMDhqayU0
    MGF2YW5lZ2FzczA0JTJFd3dsMTcyOSUyRWhhbm1haWwlMkVuZXQlM0UiPgo=
    X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) )
    MR-27CA1CFB
    X-CRM114-CacheID: sfid-20161119_172554_829178_39744B64
    X-CRM114-Status: UNSURE (   0.00  )
    X-CRM114-Notice: Please train this message.

If I convert it to "unix style" line breaks it appends the X-CRM114
lines to the header. As it should be - I think:

    $ cat /tmp/SomeMail.rfc5322.eml|tr -d '\015'|./mailreaver.crm|grep
    "^$" -B4 -A3
    X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) )
    MR-27CA1CFB
    X-CRM114-CacheID: sfid-20161119_174156_811984_46A16809
    X-CRM114-Status: UNSURE (   0.00  )
    X-CRM114-Notice: Please train this message.

    PGh0bWw+PGhlYWQ+PHN0eWxlPiBwIHttYXJnaW4tdG9wOjBweDttYXJnaW4tYm90dG9tOjBw
    eDt9IDwvc3R5bGU+PC9oZWFkPgo8Ym9keT48ZGl2IHN0eWxlPSJmb250LXNpemU6MTJweDsg
    Zm9udC1mYW1pbHk66rW066a8LOq1tOumvOyytCxHdWxpbSxCYWVrbXVrIERvdHVtLFVuZG90

The mailreaver.crm is the unaltered one that shipped with Ubuntu 14.04.
Is this behavior normal or am I missing something?

Simon

Re: [Crm114-general] Unsure please train

[Bob B. <bo...@fa...>]

On Thu, 11 Feb 2016, Eugene Crosser wrote:

> Pass the message as stdin to
>
> sudo -u crm114 /usr/bin/crm -u /var/lib/crm114 mailreaver.crm {--spam|--good}

I think I am behind the times, badly, in that I have not 
implemented mailreaver.crm, but are still using mailfilter.crm. 
Hence I have no CRM header lines in my emails, and my efforts to 
train also fail. But I will try your idea, above, anyway!

Make sense?

-- 
IMPORTANT: This email is intended for the use of the individual
addressee(s) named above and may contain information that is
confidential, privileged or unsuitable for overly sensitive
persons with low self-esteem, no sense of humour or irrational
metaphysical beliefs.

Re: [Crm114-general] Unsure please train

[wsy <ws...@me...>]

Yeah, I have the same problem.  I can't even remember my training
password (which thank goodness is in the config file... somewhere.

To mail to yourself, just put "bo...@fa..." as the "to" address.

Then make sure you include the headers, because somewhere in there is 
the X-CRM114-CacheID; it'll look something like this:

X-CRM114-CacheID: sfid-20160211_145921_947996_1F8ACC6B

(format is yearmonthday_hourminsec_microsec_checksum)

That's what the script I use (and distribute) toggles on.

   - Bill


Bob Bernstein <bo...@fa...> writes:

> I have been running crm-114 for so long I forget how to "train" 
> a message. The above "Unsure please train" message has begun to 
> appear on one of the email accounts for which I have installed 
> crm-114. To be exact, it appears when, in mutt, I press 'Esc-b' 
> to "learn" the mail in question as spam.
>
> Here's the command as it exists in my .muttrc:
>
> macro index \eb "<pipe-entry>crmlearn 
> --learnspam\n<save-entry>=crm-114" "crm114 learn as spam, save 
> in crm-114"
>
> I wonder if much of my problem may be that I do not know how to 
> mail a message to myself. I've never done it. Is that operation 
> part of the "training" I am being urged to implement?
>
> Sorry to sound so dense.
>
> Best,

Re: [Crm114-general] Unsure please train

[Eugene C. <cr...@av...>]

On 02/11/2016 09:20 PM, Bob Bernstein wrote:
> I have been running crm-114 for so long I forget how to "train" 
> a message.

Pass the message as stdin to

sudo -u crm114 /usr/bin/crm -u /var/lib/crm114 mailreaver.crm {--spam|--good}

or if you have filled the cache at the classify stage, the "message" can be just
one line: "X-CRM114-CacheID: ...".

Eugene

Re: [Crm114-general] Unsure please train

[Bob B. <bo...@fa...>]

On Thu, 11 Feb 2016, wsy wrote:

> Then make sure you include the headers, because somewhere in 
> there is the X-CRM114-CacheID; it'll look something like this:
>
> X-CRM114-CacheID: sfid-20160211_145921_947996_1F8ACC6B

I don't have that in my emails.

They get scored, and, for the most part, appropriately so, via 
this line in my maildrop .mailfilter

CRMSCORE=`grep -a -v "^X-CRM114" | crm -u $HOME/.crm-114/ 
/usr/local/share/crm114/mailfilter.crm --stats_only

And then I add a header to display that score:

xfilter "formail -I \"X-CRM114-Score: $CRMSCORE\""

...and that's it. Like I say those scores have been, in keeping 
with crm-114's evident abilities, abundantly reliable. It's a 
rare occation when I feel I want to click 'Esc-b' in mutt to 
relearn a message.


Thanks,

-- 
IMPORTANT: This email is intended for the use of the individual
addressee(s) named above and may contain information that is
confidential, privileged or unsuitable for overly sensitive
persons with low self-esteem, no sense of humour or irrational
metaphysical beliefs.

[Crm114-general] Unsure please train

[Bob B. <bo...@fa...>]

I have been running crm-114 for so long I forget how to "train" 
a message. The above "Unsure please train" message has begun to 
appear on one of the email accounts for which I have installed 
crm-114. To be exact, it appears when, in mutt, I press 'Esc-b' 
to "learn" the mail in question as spam.

Here's the command as it exists in my .muttrc:

macro index \eb "<pipe-entry>crmlearn 
--learnspam\n<save-entry>=crm-114" "crm114 learn as spam, save 
in crm-114"

I wonder if much of my problem may be that I do not know how to 
mail a message to myself. I've never done it. Is that operation 
part of the "training" I am being urged to implement?

Sorry to sound so dense.

Best,

-- 
IMPORTANT: This email is intended for the use of the individual
addressee(s) named above and may contain information that is
confidential, privileged or unsuitable for overly sensitive
persons with low self-esteem, no sense of humour or irrational
metaphysical beliefs.

[Crm114-general] Issues working CRM114 in windows

[Panging, P. <pan...@ba...>]

Hi Folks!

I have installed CRM114 ( BlameBarack). And have tested it

using:

> crm114 -v

Works

fine!

But a simple Hello world is hard in

coming for me in the command line after that.

The CRM documentation says: to use RETURN followed  by Ctrl-Z in Windows machine and Ctrl-D in Unix.

crm '{output

/Hello, world!\n/ }'

Hit return, then

hit EOF (ctrlD on a Linux system, ctrlZ on Windows3). You  should see:

# crm '{output /Hello,

world!\n/ }'

return, then ctrl-D

Hello, world!

#

But all I get is an error message saying  'cannot read the file 'output'' after I hit  RETURN.

If I hit Ctrl-Z all I get is ^Z

printed on the next command line.



Any Suggestions?



The more advanced command such as : crm114 '-{ learn osb unique microgroom ( D:/Data/good.css ) }'

Does not work either nothing gets written to good.css after this command is executed followed by
pipe = os.popen(command, 'w')
pipe.write(text)

Thanks
Pankaj

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.

Re: [Crm114-general] Mailreaver/mailtrainer TOE

[Jeff R. <li...@jr...>]

OK, after perusing the "CRM Revealed" book and delving into 
mailtrainer.crm (and adding a bunch of output statements to follow the 
flow), I do think mailtrainer.crm implements DSTTTR training.

If the message is over the threshold, it is not learned.  If it is in 
the unsure zone (or misclassified), a learn command is executed.  If it 
*still* doesn't score over the threshold, a learn "refute" command is 
run on the opposite bin.  (ie, if it's spam, it is refuted against 
nonspam)  If the classifier is hyperspace, the learn refute action is 
replaced by a simple repeat of the learn.

It would be nice if the header added to the file actually reflected what 
was going on.  Having a header that says "LEARNED AND CACHED GOOD" when 
in fact that did not happen is misleading.  The training report is more 
accurate, although again I think it should say clearly when a learn 
command wasn't executed because the message was already over the threshold.

Perhaps this was obvious to most people, but it wasn't to me!  (and due 
to the lack of responses, not obvious to at least a few others!)

Jeff

> Jeff Rice <mailto:li...@jr...>
> August 22, 2014 at 1:56 PM
>
> To clarify, I'm not talking about retraining.  This would be a "fresh" 
> message as far as CRM goes - it's never seen it before.  But from what 
> I've read and recall from prior discussions, CRM is most effective 
> when trained using TOE.  So if it already correctly classifies a new 
> message, I want to make sure I don't try to retrain using it.
>
> It was simple enough to add this check to my automated script to 
> prevent this.  But things get convoluted and hard to maintain when so 
> many checks take place in so many different places!
> ------------------------------------------------------------------------------
> Slashdot TV.
> Video for Nerds. Stuff that matters.
> http://tv.slashdot.org/
> _______________________________________________
> Crm114-general mailing list
> Crm...@li...
> https://lists.sourceforge.net/lists/listinfo/crm114-general
> Jeff Rice <mailto:li...@jr...>
> August 22, 2014 at 12:15 PM
> Hi,
> I realize this list appears to be comatose, but hopefully it's not
> completely unresponsive!
>
> My question is related to mailreaver.crm & mailtrainer.crm, and TOE.
> It's hard to tell from the docs and I'm not fluent enough in CRMese to
> decipher from the code if TOE training is implemented. Put simply, if I
> tell mailreaver to train using a message it classifies as very spammy,
> what happens? (mailreaver.crm --spam < spammy.msg).
>
> When I tested this on message getting a score of -127, the score of the
> message didn't change after training and my spam.css feature count was
> unchanged. That suggests to me that no training took place. However,
> the "documents learned" counter incremented by one, and a header was
> added saying "LEARNED AND CACHED AS SPAM". Those imply, at least, that
> it was learned as spam but (perhaps?) that no new tokens were found.
> (but would have been learned, had any existed) Does this make sense?
>
> Primarily, I'm thinking about this in the context of a spamtrap on my
> server, where all received emails at that address are fed into
> Spamassassin and CRM to be learned as spam without user intervention. I
> wonder if I need to put a check in before CRM to make sure the message
> isn't ALREADY classified as spam before telling CRM to learn it.
>
> Anyway, I hope people are still using CRM114. I had great results with
> it back in the 2007-8 or so but stopped using it for reasons I can't
> exactly recall. Recently I added it back in using the Spamassassin
> plugin and it's working great, with a few minor hiccups. (lovely
> program but a bit temperamental sometimes!)
>
> Jeff
>
>
> ------------------------------------------------------------------------------
> Slashdot TV.
> Video for Nerds. Stuff that matters.
> http://tv.slashdot.org/
> _______________________________________________
> Crm114-general mailing list
> Crm...@li...
> https://lists.sourceforge.net/lists/listinfo/crm114-general

Re: [Crm114-general] Mailreaver/mailtrainer TOE

[Jeff R. <li...@jr...>]

> Paul Fox <mailto:pg...@fo...>
> August 22, 2014 at 1:33 PM
> eugene wrote:
> > Hello,
> >
> > I am no expert either, so...
> >
> > On 08/22/2014 08:15 PM, Jeff Rice wrote:
> >
> > > wonder if I need to put a check in before CRM to make sure the 
> message
> > > isn't ALREADY classified as spam before telling CRM to learn it.
> >
> > ... I just vaguely recall that over-training may be bad; I would add a
> > check just to be on the safe side.
>
> i think retraining the same message isn't overtraining -- i've long
> thought that it was a null operation.
>
To clarify, I'm not talking about retraining.  This would be a "fresh" 
message as far as CRM goes - it's never seen it before.  But from what 
I've read and recall from prior discussions, CRM is most effective when 
trained using TOE.  So if it already correctly classifies a new message, 
I want to make sure I don't try to retrain using it.

It was simple enough to add this check to my automated script to prevent 
this.  But things get convoluted and hard to maintain when so many 
checks take place in so many different places!

Re: [Crm114-general] Mailreaver/mailtrainer TOE

[Paul F. <pg...@fo...>]

eugene wrote:
 > Hello,
 > 
 > I am no expert either, so...
 > 
 > On 08/22/2014 08:15 PM, Jeff Rice wrote:
 > 
 > > wonder if I need to put a check in before CRM to make sure the message 
 > > isn't ALREADY classified as spam before telling CRM to learn it.
 > 
 > ... I just vaguely recall that over-training may be bad; I would add a
 > check just to be on the safe side.

i think retraining the same message isn't overtraining -- i've long
thought that it was a null operation.

if only someone on this list knew for sure.  ;-)

paul
 > 
 > > Anyway, I hope people are still using CRM114.
 > 
 > I am, and I am quite happy with it.
 > I classify all mail via global procmailrc, and sort it to folders via
 > user-specific procmailrc's. Training is via dovecot-antispam (which
 > involves a fair number of wrapper scripts and and entry in sudoers).
 > 
 > Eugene
 > 
 > ------------------------------------------------------------------------------
 > Slashdot TV.  
 > Video for Nerds.  Stuff that matters.
 > http://tv.slashdot.org/
 > _______________________________________________
 > Crm114-general mailing list
 > Crm...@li...
 > https://lists.sourceforge.net/lists/listinfo/crm114-general

=----------------------
 paul fox, pg...@fo... (arlington, ma, where it's 65.1 degrees)

Re: [Crm114-general] Mailreaver/mailtrainer TOE

[Eugene C. <cr...@av...>]

Hello,

I am no expert either, so...

On 08/22/2014 08:15 PM, Jeff Rice wrote:

> wonder if I need to put a check in before CRM to make sure the message 
> isn't ALREADY classified as spam before telling CRM to learn it.

... I just vaguely recall that over-training may be bad; I would add a
check just to be on the safe side.

> Anyway, I hope people are still using CRM114.

I am, and I am quite happy with it.
I classify all mail via global procmailrc, and sort it to folders via
user-specific procmailrc's. Training is via dovecot-antispam (which
involves a fair number of wrapper scripts and and entry in sudoers).

Eugene

[Crm114-general] Mailreaver/mailtrainer TOE

[Jeff R. <li...@jr...>]

Hi,
I realize this list appears to be comatose, but hopefully it's not 
completely unresponsive!

My question is related to mailreaver.crm & mailtrainer.crm, and TOE.  
It's hard to tell from the docs and I'm not fluent enough in CRMese to 
decipher from the code if TOE training is implemented.  Put simply, if I 
tell mailreaver to train using a message it classifies as very spammy, 
what happens?  (mailreaver.crm --spam < spammy.msg).

When I tested this on message getting a score of -127, the score of the 
message didn't change after training and my spam.css feature count was 
unchanged.  That suggests to me that no training took place.  However, 
the "documents learned" counter incremented by one, and a header was 
added saying "LEARNED AND CACHED AS SPAM".  Those imply, at least, that 
it was learned as spam but (perhaps?) that no new tokens were found.  
(but would have been learned, had any existed)  Does this make sense?

Primarily, I'm thinking about this in the context of a spamtrap on my 
server, where all received emails at that address are fed into 
Spamassassin and CRM to be learned as spam without user intervention.  I 
wonder if I need to put a check in before CRM to make sure the message 
isn't ALREADY classified as spam before telling CRM to learn it.

Anyway, I hope people are still using CRM114.  I had great results with 
it back in the 2007-8 or so but stopped using it for reasons I can't 
exactly recall.  Recently I added it back in using the Spamassassin 
plugin and it's working great, with a few minor hiccups.  (lovely 
program but a bit temperamental sometimes!)

Jeff

Re: [Crm114-general] "provide a proper snprintf() implementation, please"

[Chris B. <cba...@as...>]

Thank you both. My 25,000 lines of CRM are running fine with just some
minor tweaks.

The biggest change moving from Ger's BlameBarrak branch back to the
mainline was not scoping, (Code written for BlameBarrak scoping seems to
"just work" on the mainline.) but the indexing of positional arguments. In
BlameBarrak, :_pos0: is the CRM interpreter, while the mainline has :_pos0:
as the first argument after the script name. It was nothing that a little
":1,$s/pos2/pos0/" couldn't fix.

Chris
--
To Boldly Go - A Free Turn-Based Strategy Game
Now in public beta!
http://tbg.asciiking.com/about.html

Re: [Crm114-general] "provide a proper snprintf() implementation, please"

[Nico Kadel-G. <nk...@gm...>]

On Tue, Mar 4, 2014 at 4:33 PM, Chris Babcock <cba...@as...> wrote:
> Thanks, Dominik. I was able to install crm without a hitch using yum.
>
> How do I check the version? I was using one of Ger Hebbut's builds with
> variable scoping before. If I have understood the implications of scoping
> properly then the code I've written over the past 5 years should be
> portable, but it's only been tested with BlameBarack.
>
> Thanks,
> Chris

It's in the version number of the EPEL published RPM. You can also
download the SRPM and review its source contents, along with the
pathches applied by the SRPM author.

Re: [Crm114-general] "provide a proper snprintf() implementation, please"

[Chris B. <cba...@as...>]

Thanks, Dominik. I was able to install crm without a hitch using yum.

How do I check the version? I was using one of Ger Hebbut's builds with
variable scoping before. If I have understood the implications of scoping
properly then the code I've written over the past 5 years should be
portable, but it's only been tested with BlameBarack.

Thanks,
Chris
--
To Boldly Go - A Free Turn-Based Strategy Game
Now in public beta!
http://tbg.asciiking.com/about.html

Re: [Crm114-general] "provide a proper snprintf() implementation, please"

[Dominik 'R. M. <do...@gr...>]

On Tuesday, 04 March 2014 at 04:22, Chris Babcock wrote:
> Hi, it's been a long time...
> 
> I'm trying to install crm114-20081111-BlameBarack on a new server
> running Centos 6 and I'm getting the error about snprintf(). Google is
> being worse than useless here. What do I need to do to either satisfy
> this dependency or work around it?

You can enable the EPEL repository[1] for CentOS and then just use:
yum install crm114

Please let me know if it doesn't work.

Regards,
Dominik

[1] https://fedoraproject.org/wiki/EPEL
-- 
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org | MPlayer http://mplayerhq.hu
"Faith manages."
        -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

[Crm114-general] "provide a proper snprintf() implementation, please"

[Chris B. <cba...@as...>]

Hi, it's been a long time...

I'm trying to install crm114-20081111-BlameBarack on a new server
running Centos 6 and I'm getting the error about snprintf(). Google is
being worse than useless here. What do I need to do to either satisfy
this dependency or work around it?

Thanks,
Chris
--
To Boldly Go - A Free Turn-Based Strategy Game
Now in public beta!
http://tbg.asciiking.com/about.html

[Crm114-general] Markovian microgrooming hangs

[Kai R. <Kai...@ar...>]

Every once in a blue moon, I have problems with training hanging indefinitely during a microgroom. The process just sits there eating CPU, but neither strace nor ltrace shows anything. 

I am running version 20100106-BlamceMichelson on Linux and using the Markovian classifier with unique and microgroom flags.

After some debugging there seems to be some very rare occasions when the crm_zapcss finds itself in a situation where it is virtually unable to find any data to zero out, i.e. zcountdown is larger than zero and it just keeps incrementing vcut higher and higher without accomplishing anything. 

I don't know if it just is bad luck with regarding to the data or if the distance calculations overflow the long value being compared against. I do not fully understand the intricacies of the data structures and interpretations of their contents, but the following patch seems to solve the problems I am having:



--- crm114-20100106-BlameMichelson.src/crm_css_maintenance.c.orig       2013-08-26 14:03:58.000000000 +0300
+++ crm114-20100106-BlameMichelson.src/crm_css_maintenance.c    2013-08-26 14:03:39.000000000 +0300
@@ -260,7 +260,7 @@
 #define DWEIGHT 1.0
 #define DWEIGHT2 0.0

-  long vcut;
+  double vcut, dist, nextcut;
   long zcountdown;
   unsigned long packlen;
   unsigned long k;
@@ -271,9 +271,10 @@
   //  fprintf (stderr, " S: %ld, E: %ld, L: %ld ", start, end, packlen );
   zcountdown = packlen / 32.0 ;  //   get rid of about 3% of the data
   actually_zeroed = 0;
-  while (zcountdown > 0)
+  while (zcountdown > 0 && vcut > 0)
     {
-      //  fprintf (stderr, " %ld ", vcut);
+      //  fprintf (stderr, " %.1f ", vcut);
+      nextcut = 0;
       for (k = start; k <= end;  k++)
        {
          if (h[k].key != 0 )       // key == 0 means "special- don't zero!"
@@ -282,21 +283,23 @@
              if (h[k].value > 0)      //  can't zero it if it's already zeroed
              {
                //  fprintf (stderr, "b");
-               if ((VWEIGHT * h[k].value) +
-                   (VWEIGHT2 * h[k].value * h[k].value ) +
-                   (DWEIGHT * (k - h[k].hash % hs)) +
-                   (DWEIGHT2 * (k - h[k].hash % hs) * (k - h[k].hash % hs))
-                   <= vcut)
+               dist = (VWEIGHT * h[k].value) +
+                (VWEIGHT2 * h[k].value * h[k].value ) +
+                      (DWEIGHT * (k - h[k].hash % hs)) +
+                      (DWEIGHT2 * (k - h[k].hash % hs) * (k - h[k].hash % hs));
+               if (dist <= vcut)
                  {
                    //  fprintf (stderr, "*");
                    h[k].value = 0;
                    zcountdown--;
                    actually_zeroed++;
-                 };
+                 }
+               else if( dist < nextcut || nextcut==0 )
+                 nextcut = dist;
              };
            };
        };
-      vcut++;
+      vcut = nextcut;
     };
   return (actually_zeroed);
 }



--
Kai...@ar...     GSM  +358-40-767 8282
Oy Arrak Software Ab   http://www.arrak.fi

Re: [Crm114-general] no training effect from small similar messages

[Paul F. <pg...@fo...>]

bump.  i guess i'll whitelist the MBTA messages unless someone
has any ideas.

paul wrote:
 > hi --
 > 
 > the boston mbta just switched to a different email alert system.
 > 
 > the messages from the new system consistently get classed as "unsure",
 > and training the messages has no affect whatever on the css file
 > statistics (other than a bump in the "documents learned" counter).
 > 
 > is there a way, other than whitelisting the source, to get these
 > messages to auto-train?
 > 
 > i'd include lots of data and examples, but i'll wait until i
 > see an answer other than "no" to the above question.  :-)
 > 
 > paul

----------------------
 paul fox, pg...@fo... (arlington, ma, where it's 81.1 degrees)

[Crm114-general] no training effect from small similar messages

[Paul F. <pg...@fo...>]

hi --

the boston mbta just switched to a different email alert system.

the messages from the new system consistently get classed as "unsure",
and training the messages has no affect whatever on the css file
statistics (other than a bump in the "documents learned" counter).

is there a way, other than whitelisting the source, to get these
messages to auto-train?

i'd include lots of data and examples, but i'll wait until i
see an answer other than "no" to the above question.  :-)

paul
----------------------
 paul fox, pg...@fo... (arlington, ma, where it's 80.6 degrees)

Re: [Crm114-general] BlameMichelson Makefile bug

[<ws...@me...>]

From: Thomas Spahni <ts...@la...>

Hi all,

I noticed that the Makefile contained in
crm114-20100106-BlameMichelson.src.tar.gz
lists in OFILES = ... crmregex_tre.c ... (line 226).

This should read crmregex_tre.o
                              ^^

Aha!  Yes, you are correct.

I will insert it.  Thanks!

   - Bill Yerazunis

[Crm114-general] BlameMichelson Makefile bug

[Thomas S. <ts...@la...>]

Hi all,

I noticed that the Makefile contained in
crm114-20100106-BlameMichelson.src.tar.gz
lists in OFILES = ... crmregex_tre.c ... (line 226).

This should read crmregex_tre.o
                              ^^
Best regards,
Thomas Spahni

Re: [Crm114-general] OT regex question. Sorrry!

[Martin T. <ma...@ma...>]

Reading Eric's regex  - ^NFG\s\FOAD$ - and wondering what \F does? Even if
the backslash before the F is removed, it will only match if the entire
inout is 8 characters including a siongle space.  Maybe it should be
^NFG\s+FOAD$  ? Apologies if I am reading this out of context.

Regards,
Martin


On Mon, Jun 17, 2013 at 5:14 PM, Eric d'Halibut <eri...@gm...>wrote:

> On 6/17/13, ws...@me... <ws...@me...> wrote:
>
> > However, I'm surprised that NFG\sFOAD didn't work.
>
> In that form, somewhat simpler than what I tried, it does work!
>
> I had been using, iirc,
>
> ^NFG\s\FOAD$
>
> Needless to add, I will continue to futz with this! And, just for the
> record and full disclosure and all that, I am running mailfilter.
> (That's right...one of the dweebs who wanted "black-" and "white-"
> lists. Oh welll.)
>
> Thanks!
> --
> No no no, my fish's name is Eric, Eric the fish. He's an halibut. I am
> not a looney! Why should I be tarred with the epithet looney merely
> because I have a pet halibut?
>
> http://trollboy.thruhere.net:8080/MontyPython/fish.htm
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Crm114-general mailing list
> Crm...@li...
> https://lists.sourceforge.net/lists/listinfo/crm114-general
>

Re: [Crm114-general] OT regex question. Sorrry!

[Eric d'H. <eri...@gm...>]

On 6/17/13, ws...@me... <ws...@me...> wrote:

> However, I'm surprised that NFG\sFOAD didn't work.

In that form, somewhat simpler than what I tried, it does work!

I had been using, iirc,

^NFG\s\FOAD$

Needless to add, I will continue to futz with this! And, just for the
record and full disclosure and all that, I am running mailfilter.
(That's right...one of the dweebs who wanted "black-" and "white-"
lists. Oh welll.)

Thanks!
-- 
No no no, my fish's name is Eric, Eric the fish. He's an halibut. I am
not a looney! Why should I be tarred with the epithet looney merely
because I have a pet halibut?

http://trollboy.thruhere.net:8080/MontyPython/fish.htm