Thread: [Dspam-devel] More patches for dspam.cgi: delivery of false positives from quarantine
Brought to you by:
paulcockings,
sbajic
From: Tod A. S. <san...@ri...> - 2013-01-08 01:24:10
Attachments:
patch.deliver_fp
|
Here is another patch for dspam.cgi that deals with the delivery of false positives from the quarantine. It is a patch on top of the recent quarantine file locking patch I submitted. It allows for specifying in configure.pl an external command for delivering false positives. If this command is not defined, the patch has no effect, other than some optional additional logging. For us, this is a script that first attempts to retrain/deliver with dspam as normal. But then the script tries to verify if the message was actually delivered, and if not, it will try to deliver the command directly (for instance, with procmail). This external script is quite site specific. Background: If the DSPAM signature of a message cannot be found in the database (most often because it has expired and has been cleaned from the database), the dspam command will exit with a successful status even though it does not deliver the message. Long ago, we had several unhappy professors after they returned from sabbatical and attempted to deliver old messages in their quarantines. Tod Sandman Sr. Systems Administrator Middleware Development & Integration Rice University |
From: Troy A. <dsp...@wc...> - 2013-01-08 01:59:33
|
indeed! Another scenario: Customer opts of dspam. Purge signature & token data for the account... But leave the quarantine intact (we have a cron job to purged old messages). But then you can't recover messages from the quarantine with the web gui, as you will encounter the behavior below. Not unless you can restore signature and token data... Or manipulate the mbox quarantine file directly. If the customer hasn't tried to retrain the message already. It's a mess. Dspam should never lose email due to an unexpected DB result. Thanks a ton Tod! I'm excited to try this out. -Troy On 1/7/2013 7:24 PM, Tod A. Sandman wrote: > If the DSPAM signature of a message cannot be found in the database (most often because it has expired and has been cleaned from the database), the dspam command will exit with a successful status even though it does not deliver the message. |
From: Troy A. <dsp...@wc...> - 2013-01-08 02:33:30
|
Could I trouble you for a sanitized version of your /usr/site/dspam_train/bin/dspam_fp script? My system is Linux/sendmail/procmail. Hate to re-invent the wheel... I'm having trouble envisioning how I would verify success or failure of delivery by DSPAM. Does it involve checking the users retrain.log? (thinking out loud) I should check for a mailbox full condition too (Quarantine mbox is separate from per user quotas in my case.) -Troy On 1/7/2013 7:24 PM, Tod A. Sandman wrote: > Here is another patch for dspam.cgi that deals with the delivery of false positives from the quarantine. It is a patch on top of the recent quarantine file locking patch I submitted. It allows for specifying in configure.pl an external command for delivering false positives. If this command is not defined, the patch has no effect, other than some optional additional logging. > > For us, this is a script that first attempts to retrain/deliver with dspam as normal. But then the script tries to verify if the message was actually delivered, and if not, it will try to deliver the command directly (for instance, with procmail). This external script is quite site specific. > > > Background: > > If the DSPAM signature of a message cannot be found in the database (most often because it has expired and has been cleaned from the database), the dspam command will exit with a successful status even though it does not deliver the message. > > > Long ago, we had several unhappy professors after they returned from sabbatical and attempted to deliver old messages in their quarantines. > > > > Tod Sandman > Sr. Systems Administrator > Middleware Development & Integration > Rice University > > > ------------------------------------------------------------------------------ > Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS > and more. Get SQL Server skills now (including 2012) with LearnDevNow - > 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. > SALE $99.99 this month only - learn more at: > http://p.sf.net/sfu/learnmore_122512 > > > _______________________________________________ > Dspam-devel mailing list > Dsp...@li... > https://lists.sourceforge.net/lists/listinfo/dspam-devel |
From: Jani P. <ji...@so...> - 2013-01-23 19:03:21
|
I would like to toss in this mess again my idea what I already mention few years ago here, but don't remember end result. Would it be time to add support for db quarantine? Dspam already use database, so it should not be so hard to store quarantine messages into database. It would make Dspam much more scalable, example together with dbmail you could do all kind of magic with db replication etc... // JiiPee On 8.1.2013 4:33, Troy Ayers wrote: > Could I trouble you for a sanitized version of your > /usr/site/dspam_train/bin/dspam_fp script? My system is > Linux/sendmail/procmail. Hate to re-invent the wheel... I'm having > trouble envisioning how I would verify success or failure of delivery by > DSPAM. Does it involve checking the users retrain.log? > > (thinking out loud) I should check for a mailbox full condition too > (Quarantine mbox is separate from per user quotas in my case.) > > -Troy > > > > On 1/7/2013 7:24 PM, Tod A. Sandman wrote: >> Here is another patch for dspam.cgi that deals with the delivery of false positives from the quarantine. It is a patch on top of the recent quarantine file locking patch I submitted. It allows for specifying in configure.pl an external command for delivering false positives. If this command is not defined, the patch has no effect, other than some optional additional logging. >> >> For us, this is a script that first attempts to retrain/deliver with dspam as normal. But then the script tries to verify if the message was actually delivered, and if not, it will try to deliver the command directly (for instance, with procmail). This external script is quite site specific. >> >> >> Background: >> >> If the DSPAM signature of a message cannot be found in the database (most often because it has expired and has been cleaned from the database), the dspam command will exit with a successful status even though it does not deliver the message. >> >> >> Long ago, we had several unhappy professors after they returned from sabbatical and attempted to deliver old messages in their quarantines. >> >> >> >> Tod Sandman >> Sr. Systems Administrator >> Middleware Development & Integration >> Rice University >> >> >> ------------------------------------------------------------------------------ >> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS >> and more. Get SQL Server skills now (including 2012) with LearnDevNow - >> 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. >> SALE $99.99 this month only - learn more at: >> http://p.sf.net/sfu/learnmore_122512 >> >> >> _______________________________________________ >> Dspam-devel mailing list >> Dsp...@li... >> https://lists.sourceforge.net/lists/listinfo/dspam-devel > > > > ------------------------------------------------------------------------------ > Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS > and more. Get SQL Server skills now (including 2012) with LearnDevNow - > 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. > SALE $99.99 this month only - learn more at: > http://p.sf.net/sfu/learnmore_122512 > > > > _______________________________________________ > Dspam-devel mailing list > Dsp...@li... > https://lists.sourceforge.net/lists/listinfo/dspam-devel > |
From: Paul C. <ds...@cy...> - 2013-01-23 19:10:42
|
I've had discussions about using IMAP for storage, but no developer resources seem keen to make it happen. Jani Partanen <ji...@so...> wrote: >I would like to toss in this mess again my idea what I already mention >few years ago here, but don't remember end result. > >Would it be time to add support for db quarantine? Dspam already use >database, so it should not be so hard to store quarantine messages into > >database. >It would make Dspam much more scalable, example together with dbmail >you >could do all kind of magic with db replication etc... > >// JiiPee > > >On 8.1.2013 4:33, Troy Ayers wrote: >> Could I trouble you for a sanitized version of your >> /usr/site/dspam_train/bin/dspam_fp script? My system is >> Linux/sendmail/procmail. Hate to re-invent the wheel... I'm having >> trouble envisioning how I would verify success or failure of delivery >by >> DSPAM. Does it involve checking the users retrain.log? >> >> (thinking out loud) I should check for a mailbox full condition too >> (Quarantine mbox is separate from per user quotas in my case.) >> >> -Troy >> >> >> >> On 1/7/2013 7:24 PM, Tod A. Sandman wrote: >>> Here is another patch for dspam.cgi that deals with the delivery of >false positives from the quarantine. It is a patch on top of the >recent quarantine file locking patch I submitted. It allows for >specifying in configure.pl an external command for delivering false >positives. If this command is not defined, the patch has no effect, >other than some optional additional logging. >>> >>> For us, this is a script that first attempts to retrain/deliver with >dspam as normal. But then the script tries to verify if the message >was actually delivered, and if not, it will try to deliver the command >directly (for instance, with procmail). This external script is quite >site specific. >>> >>> >>> Background: >>> >>> If the DSPAM signature of a message cannot be found in the database >(most often because it has expired and has been cleaned from the >database), the dspam command will exit with a successful status even >though it does not deliver the message. >>> >>> >>> Long ago, we had several unhappy professors after they returned from >sabbatical and attempted to deliver old messages in their quarantines. >>> >>> >>> >>> Tod Sandman >>> Sr. Systems Administrator >>> Middleware Development & Integration >>> Rice University >>> >>> >>> >------------------------------------------------------------------------------ >>> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, >SSRS >>> and more. Get SQL Server skills now (including 2012) with >LearnDevNow - >>> 200+ hours of step-by-step video tutorials by Microsoft MVPs and >experts. >>> SALE $99.99 this month only - learn more at: >>> http://p.sf.net/sfu/learnmore_122512 >>> >>> >>> _______________________________________________ >>> Dspam-devel mailing list >>> Dsp...@li... >>> https://lists.sourceforge.net/lists/listinfo/dspam-devel >> >> >> >> >------------------------------------------------------------------------------ >> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, >SSRS >> and more. Get SQL Server skills now (including 2012) with LearnDevNow >- >> 200+ hours of step-by-step video tutorials by Microsoft MVPs and >experts. >> SALE $99.99 this month only - learn more at: >> http://p.sf.net/sfu/learnmore_122512 >> >> >> >> _______________________________________________ >> Dspam-devel mailing list >> Dsp...@li... >> https://lists.sourceforge.net/lists/listinfo/dspam-devel >> > > >------------------------------------------------------------------------------ >Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >MVPs and experts. ON SALE this month only -- learn more at: >http://p.sf.net/sfu/learnnow-d2d >_______________________________________________ >Dspam-devel mailing list >Dsp...@li... >https://lists.sourceforge.net/lists/listinfo/dspam-devel |
From: Tod A. S. <san...@ri...> - 2013-01-23 21:36:41
Attachments:
dspam_fp
dspam-3.10.2-log_stderr.patch
|
On 8.1.2013 4:33, Troy Ayers wrote: > Could I trouble you for a sanitized version of your /usr/site/dspam_train/bin/dspam_fp script? My system is ... I am so sorry. I missed your original email a couple weeks ago. I will attach the script we use. The bad news is that the script relies on a slightly patched DSPAM: it causes DSPAM to print error messages to STDERR in addition to it's normal logging. The patch is only a few lines, and we've used it for many years. However, I don't know if it's OK for the general community. I'll attach that patch too. Before invoking DSPAM for retraining, you could verify that the signature exists; if it does not, don't invoke DSPAM - invoke direct delivery instead. We implimented this for testing long ago, but we decided on the STDERR route. Tod Sandman Sr. Systems Administrator Middleware Development & Integration Rice University Voice: 713.348.5816 |
From: Benny P. <me...@ju...> - 2013-01-24 04:10:10
|
Jani Partanen skrev den 23-01-2013 19:28: > I would like to toss in this mess again my idea what I already > mention > few years ago here, but don't remember end result. +1 > Would it be time to add support for db quarantine? so its time to consider when mysql/postgresql/sqlite3 can handle it ? > Dspam already use > database, so it should not be so hard to store quarantine messages > into > database. correct, but filesystem is already a database, no ? > It would make Dspam much more scalable, example together with dbmail > you > could do all kind of magic with db replication etc... super to restore emails if postgresql is out with a new version :|&> i still like to dream |