Re: [maildb-devel] Server-side filtering

[Jeff S. <jsq...@os...>]

On Wed, 9 Jul 2003, Darrell Kresge wrote:

> Even if only temporary, I see using procmail as choosing a sledgehammer
> to hang a picture (not that I'm necessarily opposed to such things ;-) )
>
> Since you've already a requirement to parse/extract header information,
> why not just implement regex -> folder filters directly?

Not sure what you mean here...?

> 1) I realize that configuration of the rule file will be an issue, but
> no worse so than dealing w/ .procmailrc.

Yes and no.  I mentioned procmail because it's well known/loved/trusted,
and it would be good to be able to support it (in some way).  This would
give us the leverage to have flexible filtering even in 1.0 (when we don't
have native/internal filtering).

That being said, I just thought of a problem with my proposed approach --
see below.

> 2) Provided the interface to the "pattern select/route" mechanism is
> well defined, the regex stuff could easily be replaced downstream with
> something more powerful.

Agreed -- some kind of generalized mechanism would be good.

> 3) Using an X-Header to determine routing inside the maildb proper is
> going to end up being a hack on top of a hack -- you'll end up needing
> to eliminate it later when you decide to do the Right Thing

Possibly.  But it could be good to be able to support *both* procmail
*and* native filtering (if, perhaps, on the back end, they actually end up
doing the same thing -- then it wouldn't be nasty.  i.e., separate the
decision-making process from the acting-on-the-decision process).

But I did just think of a problem with the X-Maildb-Category approach:
what it someone sends you a message with:

	X-Maildb-Category: inbox

That is -- anyone can force a message to go into any of your categories
simply by adding header lines to messages that they send to you.  And
that's clearly not a Good Thing.  :-)

So back to what I said above -- perhaps we could do a "do no harm"
approach in a .procmailrc, where instead of adding a header line, you
actually run some maildb executable that adds the message to that
category (this may get a little complicated, but bear with me for this
thought experiment...).  So instead of:

-----
:0 fc
* ^TO_...@li...
| formail -A "X-Maildb-Category: maildb/devel"

:0 fc
* ^FROM_.*@squyres.com
| formail -A "X-Maildb-Category: received/squyres/family"
-----

Instead, you'd have:

-----
:0 fc
* ^TO_...@li...
| /usr/local/bin/maildb.insert --category mailb/devel

:0 fc
* ^FROM_.*@squyres.com
| /usr/local/bin/maildb.insert --category received/squyres/family
-----

...and so on.

The real trick/complication would be for a message that matches multiple
rules; that maildb.insert (or whatever) will have to recognize that it's
the same message and simply add another category to the message that's
already in the db.  Since we can't rely on the Message-Id, this is the
part that I don't really know how to do...  :-(

It seems that these procmail rules would need to put in some kind of
forward reference saying "there's an incoming message coming, make sure
that it gets added to category ABC" (but don't forget that
procmail/mail.local/etc. can be run asynchronously, so 2 different
messages with the same Message-ID can come in and be processed
"simultaneously.  So it still reduces to the same problem as above).

And perhaps procmail isn't the thing we want to support.  But any
rules-based agent will follow the same general principles.  So this is
probably still worth discussing...

-- 
{+} Jeff Squyres
{+} jsq...@os...
{+} Research Associate, Open Systems Lab, Indiana University
{+} http://www.osl.iu.edu/