spamscan-cvs Mailing List for Spamscan
Brought to you by:
destari
Menu
▾
▴
spamscan-cvs — Mailing list for commit logs from the cvs tree.
You can subscribe to this list here.
| 2003 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(5) |
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
|
|---|
|
From: <de...@us...> - 2003-08-11 21:35:10
|
Update of /cvsroot/spamscan/spamscan/bin
In directory sc8-pr-cvs1:/tmp/cvs-serv32569
Modified Files:
spamscan
Log Message:
Made notspam messages more powerful, added option to turn off db usage via config file, and now shows db results in headers (temp only)
Index: spamscan
===================================================================
RCS file: /cvsroot/spamscan/spamscan/bin/spamscan,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** spamscan 23 Jul 2003 02:23:05 -0000 1.5
--- spamscan 11 Aug 2003 21:35:07 -0000 1.6
***************
*** 37,40 ****
--- 37,41 ----
if ($User_Preferences{'striphtml'}) { $striphtml = $User_Preferences{'striphtml'}; }
if ($User_Preferences{'passcode'}) { $passcode = $User_Preferences{'passcode'}; }
+ if ($User_Preferences{'use_db'}) { $use_db = $User_Preferences{'use_db'}; }
***************
*** 42,46 ****
##########################################################
! $version = "1.71b";
$url = "http:\/\/spamscan.sourceforge.net";
--- 43,47 ----
##########################################################
! $version = "1.75b";
$url = "http:\/\/spamscan.sourceforge.net";
***************
*** 148,152 ****
&to_filter;
&body_filter;
! &db_filter;
if ($spampassword eq $passcode) {
--- 149,155 ----
&to_filter;
&body_filter;
! if ($use_db eq "y") {
! &db_filter;
! }
if ($spampassword eq $passcode) {
***************
*** 428,431 ****
--- 431,436 ----
# $phrasedb{$wordcksum}--;
$phrasedb{$wordling}--;
+ $phrasedb{$wordling}--;
+ $phrasedb{$wordling}--;
$wordling = "";
}
***************
*** 525,531 ****
}
$wordling =~ s/\s+$//;
! $wordcksum = unpack ("%32C*", $wordling) % 32767;
# $phrasedb{$wordcksum}--;
$dbcount = $dbcount + $phrasedb{$wordling};
$wordling = "";
}
--- 530,538 ----
}
$wordling =~ s/\s+$//;
! # $wordcksum = unpack ("%32C*", $wordling) % 32767;
# $phrasedb{$wordcksum}--;
$dbcount = $dbcount + $phrasedb{$wordling};
+ $rsnline = "$wordling \- $phrasedb{$wordling}";
+ push (@spamreasons, $rsnline);
$wordling = "";
}
|
|
From: <de...@us...> - 2003-07-23 02:23:09
|
Update of /cvsroot/spamscan/spamscan/bin
In directory sc8-pr-cvs1:/tmp/cvs-serv32083
Modified Files:
spamscan
Log Message:
Added additional code to check in db for words on incoming spam.
This needs to be configurable in the users' spamscan.conf file.
Index: spamscan
===================================================================
RCS file: /cvsroot/spamscan/spamscan/bin/spamscan,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** spamscan 22 Jul 2003 20:52:25 -0000 1.4
--- spamscan 23 Jul 2003 02:23:05 -0000 1.5
***************
*** 148,151 ****
--- 148,152 ----
&to_filter;
&body_filter;
+ &db_filter;
if ($spampassword eq $passcode) {
***************
*** 295,299 ****
sub spam_algorithm {
# super lame right now
! $spam_weight = $from_value + $to_value + $body_value;
if ($spam_weight >= $spam_threshold) {
$itsspam = 1;
--- 296,300 ----
sub spam_algorithm {
# super lame right now
! $spam_weight = $from_value + $to_value + $body_value + $dbcount;
if ($spam_weight >= $spam_threshold) {
$itsspam = 1;
***************
*** 317,321 ****
$itsspam = 0;
}
! $spamreason_msg = "X-Spam-Scanned: Weight Totals: TOTAL: $spam_weight - From: $from_value\, To: $to_value\, Body: $body_value\n";
foreach $reason (@spamreasons) {
$spamreason_msg = $spamreason_msg . "X-Spam-Scanned: $reason\n";
--- 318,322 ----
$itsspam = 0;
}
! $spamreason_msg = "X-Spam-Scanned: Weight Totals: TOTAL: $spam_weight - From: $from_value\, To: $to_value\, Body: $body_value\, DB: $dbcount\n";
foreach $reason (@spamreasons) {
$spamreason_msg = $spamreason_msg . "X-Spam-Scanned: $reason\n";
***************
*** 500,501 ****
--- 501,536 ----
@completemailmsg = @newcompletemailmsg;
}
+
+ sub db_filter {
+ $dbfile = $filterdir . "spamscan";
+ dbmopen(%phrasedb, "$dbfile", 0644) or die "dbmopen: $! ($dbfile)";
+ # first, check the from address. If it exists in the db, then it's a non-spam address.
+ if ($phrasedb{$fromemail}) {
+ $dbcount = $dbcount - 20;
+ }
+ # now look through phrases, and total up count.
+ @words = split/\s+/,$learntext;
+
+ $maxdepth = 4;
+ $maxsize = ($#words + 1);
+
+ # this loop does all the 1,2, or 3 word combos etc
+ for ($i=0; $i < $maxdepth; $i++) {
+ # this is the "starting element" loop
+ for ($j=0; $j < $maxsize; $j++) {
+ $end = $j + $i;
+ # this would be the printing loop
+ for ($k=$j; $k <= $end; $k++) {
+ $wordling = "$wordling " . "$words[$k]";
+ }
+ $wordling =~ s/\s+$//;
+ $wordcksum = unpack ("%32C*", $wordling) % 32767;
+ # $phrasedb{$wordcksum}--;
+ $dbcount = $dbcount + $phrasedb{$wordling};
+ $wordling = "";
+ }
+ }
+
+ }
+
+
|
|
From: <de...@us...> - 2003-07-22 20:52:28
|
Update of /cvsroot/spamscan/spamscan/bin In directory sc8-pr-cvs1:/tmp/cvs-serv3364 Modified Files: spamscan Log Message: Major changes happening. Added smarts to allow user to analyze mail messages, and build an index. Also allows user to view db all over an email interface. Not fully implemented, but should be bug free. Index: spamscan =================================================================== RCS file: /cvsroot/spamscan/spamscan/bin/spamscan,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** spamscan 22 Jul 2003 01:57:57 -0000 1.3 --- spamscan 22 Jul 2003 20:52:25 -0000 1.4 *************** *** 42,46 **** ########################################################## ! $version = "1.70b"; $url = "http:\/\/spamscan.sourceforge.net"; --- 42,46 ---- ########################################################## ! $version = "1.71b"; $url = "http:\/\/spamscan.sourceforge.net"; *************** *** 55,62 **** --- 55,64 ---- if (!($inbody) && !($endofmsg)) { if ($_ =~ /^From\: (.+)/gci) { + $fromemail = $1; $fromfield = $message_line; } elsif ($_ =~ /^To\: (.+)/gci) { $tofield = $message_line; } elsif ($_ =~ /^Subject\: (.+)/gci) { + $subjemail = $1; $subjectfield = $message_line; $xheader = $message_line; *************** *** 76,83 **** next; } else { ! if ($_ =~ /^\s*spamscan\s+(.+)\s+(.+)/cgi) { # ! $spampassword = $1; ! $spamcommand = $2; } $body = $body . $_; --- 78,93 ---- next; } else { ! $spamcmdtmp = $_; ! chomp $spamcmdtmp; ! if ($spamcmdtmp =~ /^\s*spamscan\s+(.+)\s+(.+)/gi) { # ! $spamcmdtmp =~ s/^\s+//g; ! $spamcmd = $spamcmdtmp; ! @spamcmdline = split/\s+/,$spamcmd; ! $spampassword = $spamcmdline[1]; ! $spamcommand = $spamcmdline[2]; ! next; ! } else { ! $bodyonly = $bodyonly . $_; } $body = $body . $_; *************** *** 92,96 **** $body =~ s/([0-9])\,([0-9])/\1\2/gi; $body =~ s/\>\<\/\\\]\[\{\}\=\+/ /gi; ! $body =~ s/\(\)\"\'\`\,\.\?\!\;/\n/gi; # Now, lets look for indicators in the From: field: --- 102,115 ---- $body =~ s/([0-9])\,([0-9])/\1\2/gi; $body =~ s/\>\<\/\\\]\[\{\}\=\+/ /gi; ! $body =~ s/\(\)\"\'\`\,\.\?\!\;/ /gi; ! ! $bodyonly = "$subjemail" . $bodyonly; ! $bodyonly =~ s/\n/ /gi; ! $bodyonly =~ s/<[^>]*>//gs; ! $bodyonly =~ s/\>\<\/\\\]\[\{\}\=\+\*\:/ /gi; ! $bodyonly =~ s/\(\)\"\'\`\,\.\?\!\;/ /gi; ! $bodyonly =~ s/\s+/ /gi; ! ! $learntext = $bodyonly; # Now, lets look for indicators in the From: field: *************** *** 124,128 **** } ! # run thru the filters &from_filter; --- 143,147 ---- } ! ################# # run thru the filters &from_filter; *************** *** 130,135 **** &body_filter; ! # then, put it all together, and determine if it's spam or not. ! &spam_algorithm; @junk = split/\s+/,$mailmessage[$fromfield]; --- 149,161 ---- &body_filter; ! if ($spampassword eq $passcode) { ! $from_value = $to_value = $body_value = 0; ! @spamreasons = (); ! &spam_algorithm; ! &do_command; ! } else { ! # then, put it all together, and determine if it's spam or not. ! &spam_algorithm; ! } @junk = split/\s+/,$mailmessage[$fromfield]; *************** *** 156,160 **** $xheaderdone = 1; print RMAIL "X-Spam-Scanned: $progname $version - $url\n"; - print RMAIL "X-Spam-Scanned: $spamcommand $spampassword\n"; print RMAIL "$spamreason_msg"; print RMAIL $line; --- 182,185 ---- *************** *** 308,312 **** $xheaderdone = 1; print LOCALMAILER "X-Spam-Scanned: $progname $version - $url\n"; ! # if ($itsspam) { print LOCALMAILER "$spamreason_msg"; } print LOCALMAILER "$spamreason_msg"; print LOCALMAILER $line; --- 333,337 ---- $xheaderdone = 1; print LOCALMAILER "X-Spam-Scanned: $progname $version - $url\n"; ! if ($spampassword eq $passcode) { print LOCALMAILER "X-Spam-Scanned: Executed command: $spamcmd\n"; } print LOCALMAILER "$spamreason_msg"; print LOCALMAILER $line; *************** *** 319,322 **** --- 344,501 ---- } + sub do_command { + if ($spamcommand =~ /^spam$/i) { + &analyze_spam; + &cmd_ok; + } elsif ($spamcommand =~ /^notspam$/i) { + &analyze_notspam; + &cmd_ok; + } elsif ($spamcommand =~ /^showdb$/i) { + &showdb; + &cmd_ok; + } else { + # command not recognized + &cmd_unknown; + } + } + + sub analyze_spam { + # do analyze + $dbfile = $filterdir . "spamscan"; + dbmopen(%phrasedb, "$dbfile", 0664) or die "dbmopen: $! ($dbfile)"; + + @words = split/\s+/,$learntext; + + $maxdepth = 4; + $maxsize = ($#words + 1); + # this loop does all the 1,2, or 3 word combos etc + for ($i=0; $i < $maxdepth; $i++) { + # this is the "starting element" loop + for ($j=0; $j < $maxsize; $j++) { + $end = $j + $i; + # this would be the printing loop + for ($k=$j; $k <= $end; $k++) { + $wordling = "$wordling " . "$words[$k]"; + } + $wordling =~ s/\s+$//; + $wordcksum = unpack ("%32C*", $wordling) % 32767; + # $phrasedb{$wordcksum}++; + $phrasedb{$wordling}++; + $wordling = ""; + } + } + + # write to file + untie %phrasedb; + $xheaderdone = 0; + foreach $line (@completemailmsg) { + if (($xheaderdone == 0) && ($line =~ /\A\r?\n\Z/)) { + $xheaderdone = 1; + push (@newcompletemailmsg, $line); + push (@newcompletemailmsg, $line); + next; + } + if ($xheaderdone == 0) { + push (@newcompletemailmsg, $line); + } + } + } + + sub analyze_notspam { + # do analyze + $dbfile = $filterdir . "spamscan"; + dbmopen(%phrasedb, "$dbfile", 0664) or die "dbmopen: $! ($dbfile)"; + + @words = split/\s+/,$learntext; + + $maxdepth = 4; + $maxsize = ($#words + 1); + + # this loop does all the 1,2, or 3 word combos etc + for ($i=0; $i < $maxdepth; $i++) { + # this is the "starting element" loop + for ($j=0; $j < $maxsize; $j++) { + $end = $j + $i; + # this would be the printing loop + for ($k=$j; $k <= $end; $k++) { + $wordling = "$wordling " . "$words[$k]"; + } + $wordling =~ s/\s+$//; + $wordcksum = unpack ("%32C*", $wordling) % 32767; + # $phrasedb{$wordcksum}--; + $phrasedb{$wordling}--; + $wordling = ""; + } + } + $phrasedb{$fromemail}--; + + + # write to file + untie %phrasedb; + $xheaderdone = 0; + foreach $line (@completemailmsg) { + if (($xheaderdone == 0) && ($line =~ /\A\r?\n\Z/)) { + $xheaderdone = 1; + push (@newcompletemailmsg, $line); + push (@newcompletemailmsg, $line); + next; + } + if ($xheaderdone == 0) { + push (@newcompletemailmsg, $line); + } + } + } + + sub showdb { + # open db file, read it, and write to email file + $xheaderdone = 0; + foreach $line (@completemailmsg) { + if (($xheaderdone == 0) && ($line =~ /\A\r?\n\Z/)) { + $xheaderdone = 1; + push (@newcompletemailmsg, "X-Spam-Scanned: SHOWDB - $spampassword $spamcommand\n"); + push (@newcompletemailmsg, $line); + push (@newcompletemailmsg, $line); + next; + } + if ($xheaderdone == 0) { + push (@newcompletemailmsg, $line); + } + } + + $dbfile = $filterdir . "spamscan"; + dbmopen(%phrasedb, "$dbfile", 0644) or die "dbmopen: $! ($dbfile)"; + foreach $wordling (sort keys %phrasedb) { + $newline = "$wordling -> $phrasedb{$wordling}\n"; + push (@newcompletemailmsg, $newline); + } + } + + sub cmd_ok { + # write out "command successful" message to email file + $newline = "Spamscan command completed succesfully.\n\n$spamcmd\n\n"; + push (@newcompletemailmsg, $newline); + @completemailmsg = @newcompletemailmsg; + } + + sub cmd_unknown { + $xheaderdone = 0; + foreach $line (@completemailmsg) { + if (($xheaderdone == 0) && ($line =~ /\A\r?\n\Z/)) { + $xheaderdone = 1; + push (@newcompletemailmsg, $line); + push (@newcompletemailmsg, $line); + next; + } + if ($xheaderdone == 0) { + push (@newcompletemailmsg, $line); + } + } + + + # write out "command unknown" message to email file + $newline = "Unknown Spamscan command:\n\n$spamcmd\n\n"; + push (@newcompletemailmsg, $newline); + @completemailmsg = @newcompletemailmsg; + } |
|
From: <de...@us...> - 2003-07-22 01:58:00
|
Update of /cvsroot/spamscan/spamscan/bin
In directory sc8-pr-cvs1:/tmp/cvs-serv22492/spamscan/bin
Modified Files:
spamscan wordmunger.pl
Log Message:
added prelim code for authentication and commands
Index: spamscan
===================================================================
RCS file: /cvsroot/spamscan/spamscan/bin/spamscan,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -d -r1.2 -r1.3
*** spamscan 18 Jul 2003 20:12:44 -0000 1.2
--- spamscan 22 Jul 2003 01:57:57 -0000 1.3
***************
*** 36,39 ****
--- 36,40 ----
if ($User_Preferences{'remotemx'}) { $remotemx = $User_Preferences{'remotemx'}; }
if ($User_Preferences{'striphtml'}) { $striphtml = $User_Preferences{'striphtml'}; }
+ if ($User_Preferences{'passcode'}) { $passcode = $User_Preferences{'passcode'}; }
***************
*** 41,45 ****
##########################################################
! $version = "1.60";
$url = "http:\/\/spamscan.sourceforge.net";
--- 42,46 ----
##########################################################
! $version = "1.70b";
$url = "http:\/\/spamscan.sourceforge.net";
***************
*** 75,78 ****
--- 76,84 ----
next;
} else {
+ if ($_ =~ /^\s*spamscan\s+(.+)\s+(.+)/cgi) {
+ #
+ $spampassword = $1;
+ $spamcommand = $2;
+ }
$body = $body . $_;
}
***************
*** 150,153 ****
--- 156,160 ----
$xheaderdone = 1;
print RMAIL "X-Spam-Scanned: $progname $version - $url\n";
+ print RMAIL "X-Spam-Scanned: $spamcommand $spampassword\n";
print RMAIL "$spamreason_msg";
print RMAIL $line;
Index: wordmunger.pl
===================================================================
RCS file: /cvsroot/spamscan/spamscan/bin/wordmunger.pl,v
retrieving revision 1.1.1.1
retrieving revision 1.2
diff -C2 -d -r1.1.1.1 -r1.2
*** wordmunger.pl 17 Jul 2003 19:52:02 -0000 1.1.1.1
--- wordmunger.pl 22 Jul 2003 01:57:57 -0000 1.2
***************
*** 11,15 ****
@words = split/\s+/,$text;
! $maxdepth = 3;
####################################
--- 11,15 ----
@words = split/\s+/,$text;
! $maxdepth = 4;
####################################
|
|
From: <rd...@us...> - 2003-07-18 21:10:57
|
Update of /cvsroot/spamscan/www In directory sc8-pr-cvs1:/tmp/cvs-serv27061 Modified Files: docs.shtml Log Message: fixed the <BR> problems that was in the docs file to make it look better Index: docs.shtml =================================================================== RCS file: /cvsroot/spamscan/www/docs.shtml,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** docs.shtml 18 Jul 2003 19:39:20 -0000 1.3 --- docs.shtml 18 Jul 2003 21:10:54 -0000 1.4 *************** *** 1,9 **** <!--#include file="head.shtml" --> ! <center><font size=+2>SpamScan Documentation</font></center><BR> ! <BR> What this small tool does, is filters email based on content using key words and phrases set by the user it is filtering mail for, and determines (or attempts to anyway) whether the email is spam or not. If it is, it optionally labels the mail (in the Subject: line, ! it would simply add SPAM: in the front of the subject), or just removes it immediately.<br> <BR> Usage:<BR> --- 1,9 ---- <!--#include file="head.shtml" --> ! <center><font size=+2>SpamScan Documentation</font></center> ! <p> What this small tool does, is filters email based on content using key words and phrases set by the user it is filtering mail for, and determines (or attempts to anyway) whether the email is spam or not. If it is, it optionally labels the mail (in the Subject: line, ! it would simply add SPAM: in the front of the subject), or just removes it immediately.<BR> <BR> Usage:<BR> *************** *** 13,21 **** Then run the newaliases command to get it going.<BR> <BR> ! That's basically it. if you are going to run this system wide, you need to set it as the local mailer for your system,<BR> and let it handle all the mail. More on this later.<BR> <BR> ! The only editing you may need to do, is the first few lines of the spamscan program have some configurable parts, <BR> ! so you'll need to point it to your local mailer, filter directories, etc.<BR> ! <BR> <!--#include file="tail.shtml" --> --- 13,22 ---- Then run the newaliases command to get it going.<BR> <BR> ! That's basically it. if you are going to run this system wide, you need to set it as the local mailer for your system, and let it handle all the mail. More on this later.<BR> <BR> ! The only editing you may need to do, is the first few lines of the spamscan program have some configurable parts, ! so you'll need to point it to your local mailer, filter directories, etc. <BR> ! </p> ! <!--#include file="tail.shtml" --> |
|
From: Ross D. <ro...@ce...> - 2003-07-18 20:43:02
|
work dammit |
