#117 Add SpamAssassin filter to mail pipeline

Mailman 2.1
open
nobody
1
2003-03-17
2002-03-25
No

This filter adds support for discarding or holding spam
sent to the mailing list. It contacts a spamd daemon
(from SpamAssassin -- http://spamassassin.taint.org\) to
score the message.

If the score is above a certain threshold (default 10),
the message is discarded and an entry is written to the
vette log.

If the score is above another lower threshold (default
5), the message is held for moderation.

The SpamAssassin.py file should be installed in
Mailman/Handlers/. The LIST_PIPELINE variable in
Mailman/Handlers/HandlerAPI.py should be modified to
include a 'SpamAssassin' item (I put it just after the
existing 'SpamDetect' item).

To change the defaults, the following can be added to
the mm_cfg.py file:
SPAMASSASSIN_HOST = 'host:port' # how to contact SA
SPAMASSASSIN_DISCARD_SCORE = 10
SPAMASSASSIN_HOLD_SCORE = 5

If you don't want to discard messages, then
DISCARD_SCORE can be set to something very high (1000
should do it).

It looks the MM2.1 filter APIs have changed a bit, so
this filter will need some modifications to work with
that version. When I get round to upgrading, I might
look into updating it.

Discussion

1 2 > >> (Page 1 of 2)
  • James Henstridge

    SpamAssassin.py

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    There is a fairly easy optimisation for this filter that I
    missed when writing it. It calls str() on the message
    object twice. It would be quicker to call str() on the
    message once.

     
  • James Henstridge

    new version of SpamAssassin.py (27/04/2002)

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    Just attached my updated version of the patch. This version
    requires SpamAssassin 2.20 (for the extra commands that the
    spamd daemon understands). It now displays a list of which
    rules were triggered for held messages, and can give
    messages from list members a bonus (defaults to 2), so that
    they are less likely to get held as spam.

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    This version is essentially the same as the previous
    version, but adds compatibility with python > 1.5.2, which
    doesn't like you passing two arguments to socket.connect().

     
  • James Henstridge

    new version of SpamAssassin.py (14/05/2002)

     
  • Sean Reifschneider

    Logged In: YES
    user_id=81797

    FYI: I've been running the 2002-05-14 version of this patch
    with spamassassin 2.20 for the last day on our main mailman
    box and it seems to be working great.

     
  • James Henstridge

    new version of SpamAssassin.py (03/07/2002)

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    Yet another version. There were some bugs in handling of
    certain error conditions when talking to spamd. These would
    result in exceptions and the messages staying in the
    delivery queue :(

    With the new version, the message will be passed through
    unchecked under these conditions, and a message will be
    added to the error log.

     
  • Sean Reifschneider

    Logged In: YES
    user_id=81797

    FYI, I ran the previous version since installation and it
    seemed to work fine. I didn't run into any problems, with
    probably 500 messages handled. I've updated to the new
    version and it seems ok so far, but I've only sent about 10
    messages through.

    Sean

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    The Mailman installation on mail.gnome.org also uses this
    filter. I don't think there are any stability problems with
    the filter.

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    Yet another new version that fixes a small typo. With
    previous messages, you couldn't approve messages that had
    been identified as spam once (they would get identified
    again when the queue got processed, instead of passing the
    message through).

     
  • James Henstridge

    new version (25/07/2002)

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    remembering to check the "upload file" checkbox this time ...

     
  • dann frazier

    dann frazier - 2002-08-17

    Logged In: YES
    user_id=146718

    hey James,
    found a typo. also wanted to point out:
    http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=139942&repeatmerged=yes

    --- SpamAssassin.py.orig Sat Aug 17 12:05:41 2002
    +++ SpamAssassin.py Sat Aug 17 12:06:13 2002
    @@ -35,7 +35,7 @@
    SPAMD_HOST = mm_cfg.SPAMASSASSIN_HOST
    i = string.find(SPAMD_HOST, ':')
    if i >= 0:
    - SPAMD_HOST, SPAMD_PORT = SPAMD_HOST[:i], host[i+1:]
    + SPAMD_HOST, SPAMD_PORT = SPAMD_HOST[:i],
    SPAMD_HOST[i+1:]
    try: SPAMD_PORT = int(SPAMD_PORT)
    except: SPAMD_PORT = None
    except:

     
  • Sean Reifschneider

    Logged In: YES
    user_id=81797

    How about changing that chunk of code to:

    SPAMD_HOST = 'localhost'
    SPAMD_PORT = None
    if hasattr(mm_cfg, 'SPAMASSASSIN_HOST):
    SPAMD_HOST = mm_cfg.SPAMASSASSIN_HOST
    try:
    SPAMD_HOST, SPAMD_PORT = string.split(SPAMD_HOST,
    ':', 1)
    SPAMD_PORT = int(SPAMD_PORT)
    except ValueError:
    SPAMD_PORT = None
    if not SPAMD_PORT: SPAMD_PORT = 783

    This gets rid of the "bare except"s, and I think it's a
    little clearer than the previous code. The ValueError will
    be tripped if the string doesn't have a : in it, or if the
    int coercion fails. Though perhaps in that instance you'd
    want to log an error or something...

    Sean

     
  • Sean Reifschneider

    Logged In: YES
    user_id=81797

    How about changing that chunk of code to:

    SPAMD_HOST = 'localhost'
    SPAMD_PORT = None
    if hasattr(mm_cfg, 'SPAMASSASSIN_HOST):
    SPAMD_HOST = mm_cfg.SPAMASSASSIN_HOST
    try:
    SPAMD_HOST, SPAMD_PORT = string.split(SPAMD_HOST,
    ':', 1)
    SPAMD_PORT = int(SPAMD_PORT)
    except ValueError:
    SPAMD_PORT = None
    if not SPAMD_PORT: SPAMD_PORT = 783

    This gets rid of the "bare except"s, and I think it's a
    little clearer than the previous code. The ValueError will
    be tripped if the string doesn't have a : in it, or if the
    int coercion fails. Though perhaps in that instance you'd
    want to log an error or something...

    Sean

     
  • Sean Reifschneider

    Logged In: YES
    user_id=81797

    That last one had a missing quote. Try this patch:

    *** SpamAssassin.py.orig Fri Aug 23 00:28:59 2002
    --- SpamAssassin.py Fri Aug 23 00:31:00 2002
    ***************
    *** 30,45 ****
    from Mailman.Logging.Syslog import syslog
    from Hold import hold_for_approval

    ! SPAMD_PORT = 0
    ! try:
    ! SPAMD_HOST = mm_cfg.SPAMASSASSIN_HOST
    ! i = string.find(SPAMD_HOST, ':')
    ! if i >= 0:
    ! SPAMD_HOST, SPAMD_PORT = SPAMD_HOST[:i], host[i+1:]
    ! try: SPAMD_PORT = int(SPAMD_PORT)
    ! except: SPAMD_PORT = None
    ! except:
    ! SPAMD_HOST = 'localhost'
    if not SPAMD_PORT: SPAMD_PORT = 783

    try: DISCARD_SCORE = mm_cfg.SPAMASSASSIN_DISCARD_SCORE
    --- 30,44 ----
    from Mailman.Logging.Syslog import syslog
    from Hold import hold_for_approval

    ! SPAMD_HOST = 'localhost'
    ! SPAMD_PORT = None
    ! if hasattr(mm_cfg, 'SPAMASSASSIN_HOST'):
    ! SPAMD_HOST = mm_cfg.SPAMASSASSIN_HOST
    ! try:
    ! SPAMD_HOST, SPAMD_PORT =
    string.split(SPAMD_HOST, ':', 1)
    ! SPAMD_PORT = int(SPAMD_PORT)
    ! except ValueError:
    ! SPAMD_PORT = None
    if not SPAMD_PORT: SPAMD_PORT = 783

    try: DISCARD_SCORE = mm_cfg.SPAMASSASSIN_DISCARD_SCORE

    Sean

     
  • Barry Warsaw

    Barry Warsaw - 2002-11-19
    • priority: 5 --> 1
     
  • James Henstridge

    spamd.py (17/03/2003)

     
  • James Henstridge

    Logged In: YES
    user_id=146903

    Attached is an updated version of the filter for adding
    SpamAssassin support to mailman. This version is targetted
    at Mailman 2.1.x.

    The code for talking to spamd has been split out into a
    separate file, so that it can be updated independently of
    the Mailman specific code. It has also been updated to work
    with SpamAssassin 2.50 (and should be a lot more robust to
    future additions to the spamd protocol).

    The filter has also been changed to use the list name as the
    username passed to spamd, which means that separate
    auto-whitelists and bayes databases can be maintained for
    each list.

    Installation is trivial. Simply copy spamd.py and
    SpamAssassin.py to the Mailman/Handlers directory and add
    the following line to Mailman/mm_cfg.py:
    GLOBAL_PIPELINE.insert(1, 'SpamAssassin')

     
  • James Henstridge

    SpamAssassin.py (17/03/2003) - for Mailman 2.1

     
  • James Henstridge

    • milestone: 172189 --> Mailman 2.1
     
  • Pug Bainter

    Pug Bainter - 2003-05-05

    Logged In: YES
    user_id=484284

    After installing patch 668685 for the HTDig integration into
    Mailman 2.1.1, I started getting the following:

    May 02 16:50:34 2003 (23484) Uncaught runner exception:
    global name 'False' is not defined
    May 02 16:50:34 2003 (23484) Traceback (most recent call last):
    File "/var/mailman2/Mailman/Queue/Runner.py", line 105, in
    _oneloop
    self._onefile(msg, msgdata)
    File "/var/mailman2/Mailman/Queue/Runner.py", line 155, in
    _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
    File "/var/mailman2/Mailman/Queue/IncomingRunner.py", line
    130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
    File "/var/mailman2/Mailman/Queue/IncomingRunner.py", line
    153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
    File "/var/mailman2/Mailman/Handlers/SpamAssassin.py",
    line 75, in process
    score, symbols = check_message(mlist, str(msg))
    File "/var/mailman2/Mailman/Handlers/SpamAssassin.py",
    line 57, in check_message
    connection = spamd.SpamdConnection(SPAMD_HOST)
    File "/var/mailman2/Mailman/Handlers/spamd.py", line 79,
    in __init__
    self.request_headers =
    mimetools.Message(StringIO.StringIO(), seekable=False)
    NameError: global name 'False' is not defined

    I corrected this by defining "False = 0" in spamd.py. I
    don't know what the "real" solution should be though.

     
  • James Henstridge

    spamd.py (06/05/2003)

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks