[AMaViS-user] ANNOUNCE: amavisd-new-2.4.5 RELEASED

[Mark M. <Mar...@ij...>]

amavisd-new-2.4.5 has been released, it is available at:

  http://www.ijs.si/software/amavisd/amavisd-new-2.4.5.tar.gz

The 2.4.5 is primarily a maintenance release, fixing known
bugs and deficiencies in 2.4.4. It is expected to be the last
release in the 2.4.x chain, and 2.5.0 is to follow shortly.

The version 2.4.5 of amavisd-new is fully compatible with 2.4.4.

If using p0f-analyzer, please switch to a version of p0f-analyzer.pl
as supplied with 2.4.5 (or later) when there is an opportunity.
It fixes an endless-loop in p0f-analyzer.pl which happens if
a p0f daemon crashes (piped to stdin of p0f-analyzer) or is
manually terminated without also terminating p0f-analyzer.
The fixed p0f-analyzer.pl may be (and should be) used even
with earlier versions of amavisd.

Additionally, it binds only to a loopback interface by default
(as some marginal security improvement) and brings the following
hardening for setups where p0f-analyzer.pl is running on a
separate host from (amavisd or other) client:

- p0f-analyzer.pl will no longer reply to queries coming from low-numbered
  UDP ports below 1024 or from nfsd port 2049, and will ignore queries
  with nonce longer than 1024 character or containing characters outside
  of \040-\177 range to limit its usefulness as a potential reflector
  for an attacker from internal networks.


BUG FIXES

- let p0f-analyzer.pl exit when a pipe on stdin is closed (e.g. when p0f
  is killed or crashes), instead of entering a tight loop; reported by
  Justin Piszcz and Henrik Krohns;

- hard-blacklisting no longer skips quarantining when
  $spam_quarantine_cutoff_level is undefined (or is an empty string);

- restart timer after Sophie times out; previously the next attempt
  would run with no time limit; reported by Nick Leverton and
  Nicklas Bondesson;

- fixed AM.PDP code to always provide smtp-quoted form in angle brackets
  in delrcpt and addrcpt attributes of a response, i.e. in the same form
  as was received in sender and recipient attributes;

- fix error reporting in open_on_specific_fd when POSIX::dup2 fails;
  thanks to Chris (decoder);

- fix signal handling in read_snmp_variables() and register_proc(),
  a signal could previously get lost (not re-signaled) if it occurred
  within these subroutines;

- fixed get_body_digest which incorrectly determined 7- or 8-bitness
  of mail header and body, setting body_type incorrectly (with only
  cosmetic ill-effects);

- AM.PDP protocol: ensure proper address form is used in server response
  attributes 'delrcpt' and 'addrcpt': the same form should be used as
  in 'sender' and 'recipient' attributes. The attribute value syntax is
  specified in RFC 2821 as 'Reverse-path' (i.e. smtp-quoted form, enclosed
  in <>); previously enclosing angle brackets were missing in a server reply;

- documentation - amavisd.conf-default incorrectly stated that a default
  value for $prepend_header_fields_hdridx is 1;  actually the default is 0
  as correctly indicated in release notes; reported by Jo Rhett;

OTHER

- qmail interfacing notice:
  MTA timeout for waiting on results from amavisd should be longer than
  $child_timeout (8 minutes by default) with some margin, setting MTA timeout
  to 15 or 20 minutes is usual. With qmail however the QMQP code in qmail
  has hard-coded timeouts set, 10 seconds for connect and 60 seconds for
  read/write. If amavisd processing takes longer than 60 seconds, the MTA
  drops connection and retries later, yet amavisd continues processing
  and eventually delivers a mail (with each MTA retry), causing repeated
  deliveries of the same message. The following patch by Eric Huss on
  the www.qmail.org page: http://www.ehuss.org/qmail/qmqpc-timeout.tar.gz
  should be applied to qmail when interfacing it to a post-queue content
  filter. Problem researched by Nicklas Bondesson;

- better timeout handling in interface code to daemonized virus scanners
  like clamd,  Sophie, Trophie: allow short time (10 s) for connect and
  for sending a request, then allow normal (long) time to collect results;
  keep evidence of the initial deadline on retries;

- prefer '7bit' as Content-Transfer-Encoding when attaching original message
  or its headers (message/rfc822 or text/rfc822-headers) to DSN or to a
  defanged mail, and only specify '8bit' when necessary;

- remove protecting the $ and @ characters in second argument
  of a regexp selector macro, it is unnecessary and confusing;

- sanitize Message-ID and Resent-Message-ID header field bodies in
  macros %m, %r and header_field by providing angle brackets if missing
  to facilitate log parsing (angle brackets are RFC 2822 required syntax
  and are semantically not part of a message id);

- updated $map_full_type_to_short_type_re to avoid mapping file(1) result
  'MS-DOS executable (built-in)' to types 'exe-ms' and 'exe'; the file(1)
  utility generously declares any text file starting with LZ to be a
  'MS-DOS executable (built-in)';  thanks to Noel Jones, Jakob Curdes
  and Clifton Royston for troubleshooting;

- add X-Spam-* header fields to quarantined mail if spam score is at or
  above tag_level. Previously message needed to be recognized as spammy
  or spam (tag2 or kill level) in order to receive spam header fields
  in quarantined copy. This also makes it more consistent with adding
  such header fields to passed mail;  suggested by Michael Gaskins;

- add X-Amavis-OS-Fingerprint header field to quarantined mail;

- header field X-Spam-Score in a passed or quarantined mail now reflects
  score boost even when SA score is unknown (e.g. when SA was not called),
  and reflects white and blacklisting by pushing score to 0 or 64, to
  make it consistent with a bar size in X-Spam-Level header field;

- resignal "timed out" after (almost) every eval {} which has no subsequent
  call to prolong_timer() to ensure we do not continue running with
  disabled timer. Exceptions are DESTROY and END handlers, and code which
  handles timer in some other way (e.g. by keeping evidence of a deadline);

- for the purpose of looking up client IP address in @mynetworks_maps,
  treat unknown/unavailable IP address as 0.0.0.0;  this allows treating
  directly submitted mail on the MTA host (not submitted through SMTP)
  as coming from IP address 0.0.0.0 (i.e. "This" Network - according to
  RFC 1700);

  Note that this is indistinguishable from other reasons when IP address
  is not made available to amavisd, e.g. when smtp_send_xforward_command
  option in Postfix smtp service is not enabled, which is why the default
  setting of @mynetworks does not include a 0.0.0.0/8 network to prevent
  falsely loading a MYNETS policy bank.

  One should add 0.0.0.0/8 to a @mynetworks list only when XFORWARD is known
  to work and if some software on the MTA host is submitting its mail to MTA
  directly, e.g. through a sendmail command, and MYNETS policy bank loading
  is needed for proper processing of such mail;

- report a more informative message when a file(1) utility fails to produce
  useful results: joins exit status with a parsing report into one message;
  thanks to Andres, whose file(1) utility was crashing with SEGV;

- consistency: rearrange implicitly adding $X_HEADER_TAG to a hash
  %allowed_added_header_fields so that it is possible to turn off
  insertion of $X_HEADER_TAG header field by turning off associated key in
  %allowed_added_header_fields even when $X_HEADER_TAG is explicitly defined;

- let %allowed_added_header_fields also control insertion of header fields
  into quarantined message;

- amavisd-nanny now displays a title line indicating the semantics of columns;

- Courier patch: ensure the information is stored to newly introduced
  recip_addr_smtp and sender_smtp object attributes, which are needed
  to preserve pristine address forms for DSN and ORCPT use and for logging;
  a patch by Martin Orr;

- qmqpqq (qmail): ensure the information is stored to newly introduced
  recip_addr_smtp and sender_smtp object attributes;

- qmail patch now activates line-by-line sending to qmail to avoid qmail bug
  ('bare LF' reported when CR and LF are separated by a TCP packet boundary);

- tighten a regexp on matching a p0f fingerprint for Windows XP to avoid
  matching 'Windows XP SP1+, 2000 SP3';  suggested by Michael Scheidell;

- updated AV entry for CentralCommand Vexira (vascan):
  removed hard-coded option '--vdb';  by Brian Wong;

- internal: move code dealing with a SA call to a dedicated
  subroutine call_spamassassin;

- internal: provide new routines to collect scalar and structured results
  from a subprocess (collect_results, collect_results_structured) and
  take advantage of them in decoding, in AV and in dspam interface routines,
  unifying code and providing results size sanity limit and consistent
  killing of runaway external programs;

- experimental: taking advantage of the above, make it possible to run SA in
  a spawned process, requested by setting a new config variable $sa_spawned
  to true (it is off by default); benefits are that a mainstream child process
  can not be brought down by potential processing problems in SA or its
  external modules, and timeouts are handled cleanly by a calling process;
  downside is an increase of process count (worst case: doubled), with
  corresponding increase in memory footprint, plus about 20 .. 30 ms
  of additional processing time for each call to SA;

- added a tuning tip on buffer sizes to README.sql for MySQL with InnoDB,
  by Wayne Smith;

- updated URL of Sophie AV scanner;



Mark