Menu ▾ ▴

Getting_Started

Anonymous Pascal Dreissen

Requirements

To run the ASSP you must choose where it will run. ASSP is not CPU intensive, but it can use a significant chunk of RAM to run effectively: the Bayesian and DNSBL databases cache lookups and 10k of each incoming message must be cached prior to passing on to your mail transport’s SMTP server. Our install folder including the spam and non-spam databases takes 250mb on disk. If you can change the port of your mail transport’s SMTP server to something besides 25 you can run ASSP on the same server as your mail transport. If you want to run ASSP on a separate server that’s ok too, so long as you have a fast network connection from your ASSP server to your SMTP server. Periodically you’ll need to rebuild the Bayesian database based on your spam and non-spam databases. This is very CPU and memory intensive. It takes about 5 minutes to run on our server, but could be slower if you have less ram, slower disks, or a slower CPU.

ASSP is implemented in Perl, so you’ll need a working Perl on your ASSP server. See http://www.cpan.org/ports/index.html for Perl for your computer. ASSP uses only IO::Select and IO::Socket which come preinstalled on all Perls. If you plan to run it as a service in Windows you’ll want Win32::Daemon from http://www.roth.net/perl/Daemon/. Install Win32-Daemon this way: 

  perl ppm.pl install http://www.roth.net/perl/packages/win32-daemon.ppd

To filter spam the proxy uses a modification of the Bayesian statistical approach outlined in articles referenced here: http://www.spamarchive.org/more_community_resources.htm, but in short it looks at words and phrases used in unsolicited email, and words and phrases used in your site’s normal email traffic. When a new message arrives it compares the words and phrases in it with those from the statistical comparison, combines the individual probabilities and decides if this message is more like unsolicited email or like normal email.

For that to work it needs collections of spam and non-spam messages. There’s folders [directories] (specified in the configuration) which contain these collections. It works best if there are between 5000 and 10000 unique messages in each collection.

This program also makes use of a whitelist – a list of email addresses which would generally never send you spam. It maintains this list automatically.

Because the ASSP server becomes the public interface to your email system, it also must enforce email relaying – an open email relay is a source of great joy to spammers, young and old. ASSP also recognizes authentication confirmation from your mail transport’s SMTP server.

Get a working Perl

If you don’t have a working Perl, go to http://www.cpan.org/ports/index.html (or http://activestate.com/Products/Download/Register.plex?id=ActivePerl for Windows) and follow the instructions for installing Perl on your system. If you don’t know if you have a working Perl, go to your command line and type “perl –v” – if you see the Perl version, then you have Perl. You need Version 5.8.8 or higher for ASSP. If you get an error, you probably need to install it. If you plan to run it as a service in Windows you’ll want Win32::Daemon from http://www.roth.net/perl/Daemon/. Install Win32-Daemon this way: 

  ppm install http://www.roth.net/perl/packages/win32-daemon.ppd

Install ASSP as a service on NT4/W2K: 1) Go to the start menu and select 'Perl Package Manager' 2) Type 'install http://www.roth.net/perl/packages/win32-daemon.ppd' 3) Check that you have "as a service" checked in your webconfig 4) Run the 'perl addservice.pl -i c:\assp\assp.pl c:\assp' 5) Use your service manager to start the service, or do 'perl addservice.pl -s' ASSP Documentation : Getting Started : Download and unpack the ASSP scripts. Go to http://sourceforge.net/projects/assp/ and find the latest version to download. Then unzip it into the folder of your choice. Security note: Because the email database is stored without encryption you’ll want to consult your local security expert to find the best way to protect your user’s confidential emails. It probably involves setting the access rights to the notspam directory to only allow access to administrators and the assp process.

Run the ASSP program and point your browser to http://127.0.0.1:55555/

Run the program this way (starting in your assp directory): perl assp.pl Point your browser to http://127.0.0.1:55555/ -- put anything you like in the username and use the password nospam4me to connect. If everything is working correctly you will see a beautifully formatted configuration screen with all of ASSP’s fabulous features spelled out for you in painful detail. Simply click the options you want, type a few customizations if you like and you’re on your way. At a minimum you’ll want to change values for "Web Admin Password," "Accept All Mail," "Local Domains," "Spam Error," and "Spam Addresses." The password for your server is posted in bold on the internet for EVERYONE to see, so if you don’t change it, you deserve what you get. Also remember to press Enter or click the button at the bottom to register your changes – simply clearing a checkbox doesn’t send the change to ASSP.

You want your mail to flow like this: 

Internet -> ASSP -> Mail server -> Clients

and 

Clients -> ASSP -> Mail server -> Internet

For example, internet mail needs to connect to ASSP on port 25 (ASSP's listen port), and ASSP can proxy to your mail server on port 125 (or any port you choose) -- ASSP's SMTP Destination. You need to change your mail server to match. If you have an antivirus proxy, do it like this: 

Internet -> ASSP -> Anti-virus -> Mail server -> Clients

and 

Clients -> ASSP -> Anti-virus -> Mail server -> Internet

If you run Microsoft Exchange, Lotus Notes, or a similar system that collects email from users local through something besides SMTP, then you’ll need to use a relay host or “smart host” to relay your mail. ASSP needs this to be able to automatically maintain your whitelist and non-spam email collection. See HOWTO: General instructions for any non-SMTP mail server (MS Exchange, Lotus Notes, etc) for specific instructions. But in short it will look something like this: 

Clients -> Exchange -> ASSP -> Mail server -> Internet

In the config you’ll see "Spam Addresses" – these are addresses of recipients at your site that only receive spam. You can put spambait on your website, or in Usenet posts, but most email administrators find that they continue to receive email for users who have been gone for months (years sometimes). These addresses are useful for filling your spam database, and for mail addressed to multiple recipients, for recognizing spam.

In *nix you’ll want to add ASSP to your local rc scripts so ASSP starts on system boot. Check the "As a Daemon" configuration box.

In Windows 2000 you’ll want to install it as a service: 1)Check that you have "As A Service" checked in your config; 2)Run the 

perl addservice.pl –i c:\assp\assp.pl c:\assp

to create the service; 3) Use your service manager to start the service, or do perl addservice.pl –s

Rebuild your Bayesian database

The first few days you'll want to just prime the system without trying to filter spam. This phase is very important because it starts to build up the whitelist and spam and notspam collections. To do this, make sure "Test Mode" is checked. Be sure the "Prepend Spam Subject" is also blank, no user should see anything strange. So the first several days you run in testmode WITHOUT a tag. You probably want to check "Use Subject as Maillog Names." Spend a few minutes each day moving the new messages that are miss-categorized from the spam directory to the notspam directory (or visa versa). If you are unsure if a message is misscategorized, just delete it -- it's not worth spending much time on. You may also set up the "Copy Spam and Send to this Address" in section "Copy Spam/Ham" and configure the email-Interface to send false hits tp the appropriate folder. Once you have at least 1000 messages that are properly categorized, do this:

perl rebuildspamdb.pl

This will build your own site's customized anti-spam dictionary. The program will make a file called spamdb. This is the file that ASSP uses to qualify spam or non-spam.

Possibly change your existing SMTP server’s port

Consult your mail transport’s documentation on how to do this. The default port is 25, and that is where your ASSP will need to run. If ASSP is running on a separate server then there is no need to change your mail transport’s SMTP port, however, you’ll have to arrange your IP addresses and DNS MX records to point to the ASSP server rather than your message transport’s SMTP server – see http://www.topology.org/linux/mx.html or Google “setting up mx” if you need help doing this. ASSP Documentation : Getting Started : Test that it is working. Try telnetting to ASSP server’s port 25 (or whatever is on the config screen): 

 telnet 127.0.0.1 25

You should get the banner response from your message transport’s SMTP server, but you should also see log entries appearing in the ASSP server’s log file. Note that you can also test your configuration by setting ASSP to listen on a different port (like 125) and connect to your SMTP server on port 25 and then set your email client to connect to port 125 to send email.

If all is working properly you should see email connections logged in the maillog.txt file (also visible from the admin page). You should see also see whitelist additions in that log. You should also see mail files being saved in the spam and notspam directories.

Schedule regular updates for the Bayesian database

In Windows, go to Control Panel and Scheduled Tasks, then Add New Scheduled Task, and follow the instructions from the wizard. I’ve scheduled mine to run at 3:00am every day: 

 perl rebuildspamdb.pl

In *nix you’ll need to create a cron job. Try man cron if you don’t know how.

Instructions for use for your end users

I send the following information to our email clients so they know how to take best advantage of ASSP’s features. Note you'll have to change the assp-spam address below to match your domain. We are using one of the most advanced SPAM filters on the internet. It continually adapts itself to detect spam. For the most part it does a good job, but occasionally a spam slips through.

When you receive a spam, you have a choice to either just delete it and move on, or report it to the spam filter to help improve its performance. To report the spam select the message in your inbox, right-click and select "Forward as attachment." (If your email software doesn't support "Forward as attachment" simply forward the mail.) Address it to assp-spam@mydomain.com and click send.

You've done your duty and added the evil spam to the spam filter's collection.

Just to remind you of the basic operation of our spam filter: 1) Anyone you email will never have a message blocked. 2) Never email a spammer -- you validate their address authorizing them to spam us. 3) Don't forward spam -- it makes the spam filter think we like it. 4) Report spam by forwarding it to the assp-spam@mydomain.com address. 5) The spam filter keeps track of mail we send and spam we receive -- if an incoming message is not from someone we've emailed and it's more like the mail we send than the spam we receive then it gets through. Otherwise it's blocked and the sender gets the message, "Mail appears to be unsolicited -- report errors to postmaster@ourhost.com" 6) If you become aware that mail you want is being blocked send the sender an email so their mail isn't blocked any more.

Periodic Maintenance

ASSP generates log files, initally maillog.txt. Over time these can build up. You may want to archive or delete old log files. Check to be sure that rebuildspamdb is running as scheduled.

After a few months you may want to clear out some of the files in the errors/spam and errors/notspam folder, especially if these folders are becoming quite large. At some point I'll write a utility to do this automatically, but I haven't written it yet. For now it's probably safe just to delete the oldest files.

Relaying not allowed?

When I try to send mail I get the error "relaying not allowed". What do I do? Relaying is allowed for IP's that match those in the "Accept All Mail." The "Local Domains" are the domain names that your mail system considers local. AOL.COM though would not be a local domain because your mail handler doesn't manage its mail. That a mail claims to be from one of your local domains does not allow it to be relayed -- this is easily spoofed and not useful as a security measure. Spoofing an IP address is more complicated in this type of environment, and generally relaying is limited by IP address.

Note that you can add entire groups of addresses: 10. represents 10.1.2.3 or 10.3.2.1 or 10.anything. Similarly, 169.254. will match any IP that starts with those quads.

If your clients dialup or are dynamically assigned from an untrusted pool, then the only relaible way to allow relaying is through AUTHENTECATED smtp, and your mail handler must support this type of authentication, and you must enable it in your clients. ASSP recognizes authenticated connections and allows them to relay.

Not all ISPs will allow their customers to connect to your SMTP port. Many block connections to port 25 (except to their own mail server) to prevent spam.

When people try to email me they get the message, "Relaying not allowed." What do I do?

For ASSP to recognize incoming mail you must set the "Local Domains" configuration item. For example if your mailserver is configured to receive mail for myhost.com and myfriendshost.com then you should put myhost.com|myfriendshost.com in the Local Domains configuration item. The format of the local domains file is rather strict; one domain per line, no comments. If you do put two words on a line, they'll get stuck together as one word.

How can I learn [Regular_Expressions]?

What is a regular expression? A regular expression is simply a string that describes a pattern. Patterns are in common use these days; examples are the patterns typed into a search engine to find web pages and the patterns used to list files in a directory, e.g., ls .txt or dir .*. For ASSP, the patterns described by regular expressions are used to search strings, extract desired parts of strings, and to do search and replace operations.

In basic terms, asd2 will match jjasd2 or asd2jj or jjasd2jj. A . matches any single character, so if you want to match a period put a backslash in front of it: \. Almost all punctuation has special meaning, so you need to put a backslash in front of it, unless you want the special meaning: 

. -- match any single character besides \n
.* -- match 0 or more characters (not \n)
.+ -- match 1 or more characters (not \n)
\d -- match any number
\w -- match any alpha-numeric or _
[0-9a-zA-Z_] -- same as \w
this|that -- match this or that

Try http://www.perldoc.com/perl5.8.0/pod/perlretut.html... http://www.english.uga.edu/humcomp/pe.../regex2a.html http://directory.google.com/Top/Compu.../Perl/

[How_do_i_use_the_e-mail_interface]?

ASSP's email interface is an easy way to add/remove addresses to the whitelist/redlist, report spam, or false-positives. To use it you must have it enabeled in the configuration, and have names set for the addresses. The interface accepts only mail addressed to addresses at any of your localdomains, and only from "Accept All Mail" hosts, or authenticated SMTP connections. 

 assp-white     -- for whitelist additions
 assp-notwhite  -- for whitelist removals
 assp-spam      -- to report spam that got through
 assp-notspam   -- to report miscategorized spam

There are basically two different mailflows for outgoing mails possible:

  • User -> ASSP -> Mailserver -> Internet
  • User -> Mailserver -> ASSP -> Additional SMTP Server -> Internet
A: User -> ASSP -> Mailserver -> Internet

Assuming that your local-domain is mydomain.com, to add addresses to the whitelist, create a message to assp-white@mydomain.com. You can either put the addresses in the body of the message, or as recipients of the message. For example, if you wanted to add all the addresses in your address book to the whitelist, create a message to assp-white@mydomain.com and then add your entire address book to the BCC part of the message and click send. Note that no mail will be delivered to any address except assp-white@mydomain.com (and that won't actually be passed to your mail transport). Within a short time (depends on your mail server) you'll receive a response from ASSP showing the results of your mail. To report a spam that got through, simply forward the mail to assp- spam@mydomain.com. It's best to forward it as an attachment, but you can just forward it normally if you must. In a short time you will receive a confirmation. The process is the same to report a miscategorized spam, but send it to assp-notspam@mydomain.com. Of course all these addresses can be changed to suit your sites preferences.

Any connection blessed by one of these will (a) be able to relay mail, and (b) be able to access the email interface. 

1) mail from a "allow all mail" host
2) mail with a SMTP AUTH affermative response
3) mail through the relay port
4) mail approved for relaying by popb4smtp
5) addresses in EmailSenderOK
B: User -> Mailserver -> ASSP -> Additional SMTP Server -> Internet

ASSP's email interface allows you to send messages to the ASSP server to add addresses to the whitelist, report spam, or report a false-positive. If your clients use Exchange, Notes, and/or a RelayHost / RelayPort setup, this feature becomes more complicated. ASSP is looking for it's command addresses at one of its localdomains. However your mail server will not accept mail for ASSP's addresses (and if it does, it will never be passed to ASSP). The work-around is to find a domain you never expect to actually send mail to, tell ASSP it is local, and use that domain for ASSP commands. So you could add spamreport.gov to ASSP's localdomains, and direct whitelist additions to assp-white@spamreport.gov.

Please note that the assp-white address cannot see bcc addresses this way, and cannot prevent delivery to people. Consequently if you use this configuration, be sure you only put addresses in the body of your mail message. Don't include them in the delivery part of the message. 

1) invent a "dummy" domain, like "assp-nospam.org" (seems to be an available domain name) 
2) tell ASSP that this domain is one of your "local" domains
3) tell your users to send their spam to "assp-spam@assp-nospam.org";

instead of "assp-spam@yourdomain.com"; or tell your Exchange-server to forward the later to the former. If yu prefer the "forwarding" approach, you can pick a dummy domain name that is more unlikely to be ever registered (like foo123blob.tac), as nobody has to remember the name.

IF you have ASSP on the outbound channel - as you should for the auto-whitelist function to work - and your Exchange is using the ASSP-relay-Port to deliever external mails, then the server thinks it is "external" and hands it over to delivery. ASSP thinks it's "local" and processes the magic-names as expected.

What command-line options does ASSP support?

ASSP can be run as follows: 

perl assp.pl /path/to/base 55556

Where /path/to/base is the path that ASSP will use as its install base And 55556 is the port on which to run the admin interface, overriding the default (but not a configured option). To override the configured option you must edit (or delete) your assp.cfg file.

Note that if you leave off a path/to/base then ASSP will look in the following locations (in this order) to try to find an assp.cfg and make that base: 

'.','assp','/usr/local/assp','/home/assp','/usr/assp','/assp','/applications/assp', '/etc/assp'

How do I use ASSP's Whitelist-only mode?

Some people hate spam so much they want to only receive mail from pre-approved addresses. This makes it difficult to get mail from people you don't know, but means you won't get any spam. You also don't have to set up a collection of spam and non-spam emails or setup the DNSBL stuff. If you want no bayesian filtering, only whitelist, then: 1) Check the "Reject All But Whitelisted Mail." 2) Check the "Don't log mail." 3) Check "Disable Good Hosts Antispam." 4) Delete all files in: 

a) spam
b) notspam
c) errors/spam
d) errors/notspam
f) other (if you used it)

5) Delete spamdb and dnsbl. 6) You probably want to put some things in "Expression to identify non-spam." 7) You may want to add "whitelisted domains." 8) You may want to add spamlover or nonprocessed addresses. 9) You can still use testmode and spamsubject. 10) Send an email to the assp-white address with everybody in your addressbook. Same for other users on your system. 11) Note that as an option one can put a password in the "spam error" and the "expression to identify non-spam" so that when non-whitelisted mail bounces they get the error message: 500 Unsolicited mail not accepted, include 'IllNeverSpamYou' in your mail to ensure delivery then if illneverspamyou is in your "expression to identify non-spam" they can get mail through to you without you haveing to whitelist them. Of course you should use a site-speicific phrase instead of illneverspamyou. ASSP Documentation : Getting Started : How do I help ASSP recognize mail from traveling users? Question: We have users that use other ISPs sending mail aparently from our domain. ASSP can recognize these incoming messages as spam. What do I do? Unfortunately, the from-local-address-to-local-address strategy is VERY COMMONLY used by spammers for exactly this reason.

If your mail system allows users to authenticate over SMTP, ASSP recognizes this authentication and you can use this to allow them to relay.

If not, the best solution for now is to use the "Expression to recognize Non-spam" (or WhiteRE) to make "passwords" for your road-warrior users. An easy way would be to look at their email signature(s) (if they use them -- and instruct them to use them if not). For example if they have a phone number in their signature, add that to the WhiteRE -- \(123\) 456-7890|\(123\) 123-1234 for example. You could also look closely at their email header to try to identify something unique to their outgoing mails that could go in the WhiteRE. Or just tell them to put NoSpam4Me in their signature and put NoSpam4Me in the WhiteRE.

From: "David Stanton" Sent: November 17 I'm still hoping to negotiate a local user allowed on whitelist option. If I can convince/badger you into it, is it a difficult change? ;-)

It's not really difficult, but it is highly ill-advised. Spammers routinely forge the from address to match someone in your local domain. Adding local users to the whitelist will allow a significant portion of spam to those users.

To do that, put the following in your "expression to identify non-processing mail" setting: 

from: .*(user1@localdomain\.com|user2@localdomain\.com)

Apply that change and any mail claiming to be from the listed users will be passed through with no processing.

Switch Off Test Mode

After a few days of monitoring the spam and notspam folders you will find that mail is no longer getting miscategorized. For the rare exceptions, look at the message headers or text to find some unique identifier, and add this to your "Expression to identify Whitelisted Mail." You are now ready to really begin filtering mail. 1) Either turn off "Test Mode" or put something like "[SPAM]" in the "Prepend to Subject" option. 2) Clear the "Use Subjects as File Names" option. 3) do: 

perl move2num.pl -r

4) Send the "Instructions to End Users" to your end users. 5) Ask for a raise when your boss says, "Wow, you really fixed the spam problem." 6) Submit a generous donation to the ASSP project because of all the time and money it's saved you and your company.

Good luck, and never give up!

Related

Wiki: ASSP_Documentation
Wiki: How_do_i_use_the_e-mail_interface
Wiki: Regular_Expressions

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.