File Release Notes and Changelog
Notes:
Some code cleanup and configuration file enhancements, and
a few new features: improvements to logging and
mbox/maildir delivery code, handle timeout properly with
SSL connections, some extra default tags, NNTPS fetching,
ifdef/endif blocks, inline shell commands in configuration
file, and built-in string cache using TDB.
The 1.3a version fixes a bug on OpenBSD with SSL and large
timeouts (>327 seconds). This does not affect FreeBSD or
Linux. 1.3b is because I am an idiot and uploaded the wrong
tarball.
1.3c fixes a few bugs which predictably showed up shortly
after release rather than during the last month of testing,
most notably some problems with SSL.
Changes:
01 August 2007
* fdm 1.3c released.
* Wrong size for match buffer in pcre.c. SF.net bug 1765524.
* Try to buffer as much as possible when reading, this prevents problems since
SSL_read seems to buffer data itself which poll can't know about.
* Fix a bug with SSL where a "don't stop" error (want read/want write) would
make fdm think there was data received when there wasn't, resulting in
spurious new lines and fetching being aborted. This somehow failed to show
up in the two months since 1.1 was released. Reported by Steven Mestdagh.
31 July 2007
* Accept only ".\r\n" as terminator for POP3 and NNTP, not a '.' followed by
anything except '.'.
* Don't include a leading space in the POP3 UID.
30 July 2007
* fdm 1.3a and 1.3b released.
* Change to using setitimer for SSL_connect timeout. OpenBSD, and possibly
others, limit SO_RCVTIMEO to 327 seconds. Also try smaller timeouts if setting
the first one fails, to a minimum of 30 seconds. Reported by Steven Mestdagh.
* fdm 1.3 released.
25 July 2007
* Tidied and refactored much of file.c and deliver-{mbox,maildir}.c.
* Brought back fatal and fatalx as #defines which prepend __func__ and pass
through to log_fatal/log_fatalx.
24 July 2007
* Embedding an undefined macro in a string with ${} is no longer an error, it
is just treated as empty. This makes ifdef/endif more useful.
17 July 2007
* Only store shared filename (NAME_MAX) in struct shm and construct path into
temp dir on-the-fly. Saves (PATH_MAX - NAME_MAX) per struct.
* Trailing newlines are now stripped from message_id tag (as they used to
be). This makes any existing caches by message-id useless.
* Use SO_RCVTIMEO to apply configured timeout when connecting to SSL servers
(SSL_connect must block as it is the easiest way to ensure the certificate is
received before checking its validity).
16 July 2007
* Outright reject configuration files that use caches unless built with -DDB,
rather than waiting until the cache is used.
* Rewrite the malloc debugging stuff: lose the call counting, use a splay
tree instead of a fixed array and use __builtin_return_address and direct
calls from xmalloc.c rather than horrible #define games, even if it means
the ultimate output is less useful.
15 July 2007
* Permit each account to start multiple delivery child processes simultaneously
(up to the maximum queue length). This doesn't help much in normal cases but
with particularly lengthy but non-CPU-intensive delivery actions (try exec
"sleep 1" ;-P), it can help a lot.
* Introduce some randomness into the lock sleep delay when waiting for mboxes.
14 July 2007
* Use mremap(2) on Linux.
13 July 2007
* Fix stupid bug (grr) in strb.c which caused a segfault if realloc moved the
buffer when it was being expanded; also clean up macros a bit when here.
11 July 2007
* Add %[mail_rfc822date] which contains the mail date and time (or current date
and time if invalid) in RFC822 format and %[rfc822date] which is the same for
the current date/time. I probably need to stop adding tags now $() works.
* Additional example from Giorgio Lando (examples/g-lando.conf).
* Add two-digit year (year2/mail_year2) and hostname tags.
* Simplify fetching: line counting and body detection is now done globally for
all fetch types in the mail queuing code, as is checking for empty and
oversize mails.
* There is now no concept of a body-less mail (m->body == -1). Mails with
no separating blank line are assumed to be entirely body (m->body == 0),
otherwise the body is the first byte after the separator.
* Fix a long-standing bug in openlock (mail.c): if open(2) failed, the lock
file was not removed.
* NNTPS fetching, suggested by Maximilian Gass.
* After rethinking, remove some useless fclose error checks.
08 July 2007
* Check permissions on include files as well as main configuration file.
* First new parser bug: include files should be subject to tag replacement,
noticed by Frank Terbeck.
05 July 2007
* Move to rewritten log.c which which makes the syslog/stderr hacks I had made
to the old version less fugly.
02 July 2007
* If a mail has no body, insert new headers at the start instead of the end.
This matches formail's behaviour. Requested by Giorgio Lando. Note there is
no change to the regexp "in body" behaviour, although perhaps there should be.
* Sort out the localtime(3)/strftime(3) mess by actually reading the man page
and time.h. This means July is now the seventh rather than sixth month.
Reported by Giorgio Lando.
29 June 2007
* Split parse helper functions out into parse-fn.c and do lots of other cleanup
of parse.y and lex.c.
* New lexer makes ifdef/ifndef/endif possible, although not completely elegant.
Syntax is: ifdef $macro .... endif. Note that stuff inside ifdefs must still
be valid syntax!
* Scrap use of lex in favour of custom C lexer. This allows (yet another)
workaround for include files, to make stuff like this work:
$host = $(hostname)
include "file.${host}"
* Combine multiple IMAP deletions together into one command where possible,
although I doubt it makes any actual difference.
28 June 2007
* Shell commands may now be called when parsing the ruleset using $() and %()
operators:
%year = %(date +%y)
$arch = $(uname -m)
These are executed at parse time. Sections between the brackets within "s
are subject to escape and macro replacement as normal; parts outside
quotes or inside 's are not. For example:
$SHELL = "test"
$cmd = $(echo "${SHELL} $SHELL \${SHELL}")
Takes advantage of the fact that fdm requires {}s and the shell does not to
yield:
added macro "$cmd": "test /bin/ksh /bin/ksh"
As with macros, commands may be used at any point a string or number is
appropriate.
* The mail time tags (mail_year, mail_quarter, etc) now use underscores in their
names rather than dashes, for consistency.
* Deliver append-string is no more, it is trivially replaced using rewrite.
* Built-in cache using TDB (http://sourceforge.net/projects/tdb/). Currently
disabled by default, enabled by building with make -DDB.
Used as followed;
# Declare the cache.
$db = "~/.fdm.db"
cache $db expire 2 weeks
match not string "%[message_id]" to "" {
# Filter messages already in the cache.
match in-cache $db key "%[message_id]" action
mbox "duplicate-mail"
# Add message-ids to the cache (any other string can be added
# but message-ids are most useful).
match all action
to-cache $db key "%[message_id]" continue
}
Note the guard statement - the key can't be empty and news (NNTP) messages
don't have a message-id.
27 June 2007
* New tags taken from the date header corresponding to the current time tags:
mail_hour, mail_year, etc. If the header is invalid, the current time is used.
Request/idea from Maximilian Gass.
* Make fdm-sanitize also obfuscate the lengths of login/pass with IMAP.
* Bug fix: "and not" rather than "not and" in ruleset -vv output.
* "match tag ..." is now completely gone.
* "value" keyword is now preferred for add-header action:
action "ah" add-header "My-Header" value "ABC"
but is not mandatory.
