assp-test — ASSP V2 development version Testing

Re: [Assp-test] Updating to Perl 5.40 / problems with outdated modules

[Thomas E. <tho...@th...>]

>Concerning Archive::Libarchive::XS I wonder if it is still needed,

Yes, it is recommended, if the ASSP_AFC plugin is used and 7z is not 
installed

the try to install DBD::mysql and Archive::Libarchive::XS using the cpan 
shell - should show the install errors

a valid compiled version of DBD::mysql should be available in the apt repo 
of debian (e.g. libdbd-mysql-perl)

LEOCHARRE::Debug is not requested by the module installer (and is not 
available at cpan) - but the provided package contains (also) 
LEOCHARRE::DEBUG
this package is only required on windows systems

for nix system LEOCHARRE::DEBUG should be available at cpan - if it can't 
be installed, show the cpan install errors


Thomas






Von:    "Dirk Kulmsee" <d.k...@ne...>
An:     "ass...@li..." 
<ass...@li...>
Datum:  13.08.2025 03:56
Betreff:        [Assp-test] Updating to Perl 5.40 / problems with outdated 
modules







Hi all,
 
I am now running ASSP version 2.8.2  build 25223. I just updated my box to 
Debian 13 (trixie) which brings in Perl 5.40. 
I ran mod_inst_ocr.pl (2.12) and tried updating perl modules via cpan. 
I ran into trouble with the installation of DBD::mysql, LEOCHARRE::Debug, 
PDF::OCR2 and Archive::Libarchive::XS. I finally found a package to 
install LEOCHARRE::Debug (not LEOCHARRE::DEBUG) and then PDF::OCR2 could 
also be installed. I still did not succeed installing DBD::mysql and 
Archive::Libarchive::XS. Concerning Archive::Libarchive::XS I wonder if it 
is still needed, because metacpan tells me „NOTE: This module has been 
deprecated in favor of Archive::Libarchive.“ which installs without 
problems.
 
Maybe some wise wizard could do a refresh on mod_inst.pl / assp.mod.zip so 
that it is again our one-stop-shop for all modules needed? 
 
Thanks for your hard work!

Dirk_______________________________________________
Assp-test mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Assp-test] Updating to Perl 5.40 / problems with outdated modules

[Dirk K. <d.k...@ne...>]

Hi all,

I am now running ASSP version 2.8.2  build 25223. I just updated my box to Debian 13 (trixie) which brings in Perl 5.40.
I ran mod_inst_ocr.pl (2.12) and tried updating perl modules via cpan.
I ran into trouble with the installation of DBD::mysql, LEOCHARRE::Debug, PDF::OCR2 and Archive::Libarchive::XS. I finally found a package to install LEOCHARRE::Debug (not LEOCHARRE::DEBUG) and then PDF::OCR2 could also be installed. I still did not succeed installing DBD::mysql and Archive::Libarchive::XS. Concerning Archive::Libarchive::XS I wonder if it is still needed, because metacpan tells me "NOTE: This module has been deprecated in favor of Archive::Libarchive." which installs without problems.

Maybe some wise wizard could do a refresh on mod_inst.pl / assp.mod.zip so that it is again our one-stop-shop for all modules needed?

Thanks for your hard work!

Dirk

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25223

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25223


- The feature to remove not visible text from HTML content of a mail for 
bayesian and HMM analyzes (e.g. display: none; visibility: hidden;)
  introduced in 2.8.2 build 25075, may sometimes leaded into very much too 
long runtime while parsing a mail for not visible text.
  This was caused by too havy backtracking in a regular expression.
  Possible results were SMTP-timeouts, GUI-timeouts and/or rebuildspamdb 
errors.
 
- Depending on stored corpus file names, reading the content of a folder 
may caused a worker crash with SEGV in module Win32::Unicode.
  The SEGV can't be prevented, but the workers will no longer crash.

- Even if 'HideIPandHelo' was configured, the real connected IP-address 
and HELO was provided in the added Authentication-Results and
  ARC-Authentication-Results header for outgoing mails.

- Delay a connection using the SPF-record of a domain, may caused havy CPU 
load, if the SPF-record contained a large amount of IP-addresses.

- The speed to parse and normalize the HTML/XML content of a mail is 
improved.

- Removing a SPF-record from another SPF-record in Group definitions was 
not working correctly of any of the SPF-records contained
  IPv4 and IPv6 addresses.
  The same mistake happened, if such a removement was used in an 
IP-address list definition.

- It was possible, that merging IP-addresses/ranges/CIDR in the 'work with 
IP-addresses' dialog has shown overlapping IP's in the result list.

- If debug mode was enabled short time before a scheduled job was started, 
it was possible, that debugging was working for the scheduled task
  but not for scheduler code itself.

changed:

- perl verion 5.42 is supported by this code

Thomas

[Assp-test] minimum perl version requirement for assp

[Thomas E. <tho...@th...>]

Hi all,

more and more perl module authors have changed the minimum required perl 
version to perl 5.14, because of 10 years perl policy.
Some of them doing this, even the running code has no such requirement. 
How ever an upgrade or the installation of such a module will fail on perl 
12.3.

I plan to follow this policy in 2026, but I don't want to bring anyone 
into trouble.

Is still anyone using a perl version before 5.14.0 for assp?
If so, please send a hint to me using the forum or the mailing list.


Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25213

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25213

- build 25212 caused an regular expression exception 'Variable length 
lookbehind not implemented in regex' in perl version before 5.30.0

 
Thomas

[Assp-test] issue in assp 2.8.2 build 25212

[Thomas E. <tho...@th...>]

Hi all,

I just got a report, that build 25212 is causing an exception:

'Variable length lookbehind not implemented in regex' .....

Investigation has shown that this happens only with perl version before 
perl 5.30.0 - so, if you are using perl version 5.30.0 or higher - you 
don't have to worry about!

assp build 25212 uses the following part in regexes at several places

(?<=\A|\n)any content

which means: is the 'a' in 'any' preceded by the start of the string or a 
newline character (do not capture nor count the match)

For me this looked like a fixed length of 1 character (which would be 
supported by all perl versions 5.10 and higher) - but I was wrong: the \A 
is zero width, the \n is one char - so the length is "variable" and
this is not supported before perl 5.30

I found a solution that will work with all assp supported perl versions - 
testing  as much as possible perl versions will take some hours.
The regex is splitted into two fixed length lookbehind pattern:

(?:(?<=\A)|(?<=\n))any content

If the changed regex part works with all perl versions, I'll publish the 
fix in build 25213 tomorrow

For those of you, who are using perl versions before 5.30 - please be 
patent and stay at version 2.8.2 build 25192 until tomorrow.

I'm sorry if anyone was running in to trouble because of my stupidness.

Thank you

Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25212

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25212

- build 25211 caused an regular expression exception in workers
 

Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25211

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25211

- if more than one header tag was configured in 
'removeDispositionNotification', some times only the first header part was 
removed

- if a rebuildspamdb was scheduled and started - and while it was running 
a scheduled restart was outdated - the restart was skipped
  in this case, the restart is now done right after the rebuild is 
finished

- if a mail was blocked because of a missing signed sender, the NOTSPAMTAG 
was sent in the SMTP-reply
  but the NOTSPAMTAG is ignored for missing signed senders in every case - 
the NOTSPAMTAG is no longer provided in this blocking case

- if a trusted forwarding host provided an ARC-signature with the 
DKIM-result 'pass' and an i=identity tag and
  added its own DKIM-signature - but removed the original received DKIM 
signature from the mail header - 
  the provided DKIM-identity was not used to detect 'DKIMWLAddresses' and 
'DKIMNPAddresses'

- running 'perlcritic -4 assp.pl' has shown some severety 4 and one 5 
coding mistakes - the involved code parts are corrected


changed:
- the detected clamd version is now logged in maillog.txt at startup

- the regex configured in 'removeDispositionNotification' is now checked 
for mistakes at configuration time

- under certain conditions a rebuildspamdb was silently not started - if 
this is the case, the reason for skipping the rebuild is now logged


Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25192

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25192

- In case TCP was used instead of UDP to query a DNS server and the 
DNS-answer was invalid, the assp process
  was dieing (and possibly restarted).

Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25098

[Thomas E. <Tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25098

- domains were added to the DKIMCache in case no DKIM-signature was found, 
but a DMARC-record was published by the doman holder
  this caused wrong scoring for missing DKIM signatures for future mails 
from those domains

- the IP-address CIDR calculation was not correct, if a SPF-record query 
was used in the GUI-dialog, were the SPF record contained IPv4 and IPv6 
addresses
  depending on the type of CIDR calculation, some IPv4 addresses were 
unexpectedly converted to IPv6 addresses


- changed:

- the internal ClamAV code got some minor changes to better support error 
messages from the latest File::Scan:ClamAV module version 1.96


Thomas

Re: [Assp-test] DKIMCache and DMARC

[Thomas E. <Tho...@th...>]

This bug will be fixed in the next release.

Thank you for reporting!

Thomas



Von:    "Zrin Žiborski" <zri...@zi...>
An:     "ASSP development mailing list" <ass...@li...>
Datum:  08.04.2025 00:58
Betreff:        [Assp-test] DKIMCache and DMARC







Hello Thomas,
I still find domains in the DKIMCache which do not use DKIM, but do have 
DMARC policy. Subsequently, messages from that domains which do not arrive 
signed get undeserved score.
What has changed within last few months is that many domains got a DMARC 
entry, but do not use DKIM, I guess because a big player decided that any 
domain that sends e-mails shall have one.

The sub DKIMCacheAdd is called in sub DKIMpreCheckOK_Run, sub DMARCok and 
sub DKIMOK_Run.
- Can you please check and make sure that sub DKIMCheckAdd is called only 
for domains for which there is a signature in the e-mail which exactly 
matches the sender (from) domain?
- Can you add debug output so adding a domain to DKIMCache gets logged and 
we can identify which e-mails get a domain added to DKIMCache where it 
should not get added?
In sub DMARCok:
    if (! $this->{DMARC_arc} && $DKIMCacheStrict) {
        DKIMCacheAdd($this->{dmarc}->{domain}) if $this->{dmarc}->{domain} 
&& $this->{dmarc}->{domain} ne $this->{dmarc}->{dom};
        DKIMCacheAdd($this->{dmarc}->{dom}) if $this->{dmarc}->{dom};
        for my $dom (@{$this->{dmarc}->{DKIMdomains}}) {
            DKIMCacheAdd($dom) if $dom && $dom ne $this->{dmarc}->{domain} 
&& $dom ne $this->{dmarc}->{dom};
        }
    }

Do I understand correctly that the domain will get added to DKIMCache if 
the message does not contain ARC and $DKIMCacheStrict is true - no matter 
if the domain actually uses DKIM?
As many domains have DMARC entry in the DNS but do not use DKIM - is this 
the source of the problem?

In sub DKIMpreCheckOK_Run:
DKIMCacheAdd($domain) if $dkimdomain && ($qdtxt ne 'unknown' || 
$DKIMCacheStrict);

- Can you please check that $dkimdomain is true only if the domain really 
uses DKIM keys?

Please let me know if I can help any further.
Thank you very much in advance,
best regards,
Zrin
_______________________________________________
Assp-test mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Assp-test] DKIMCache and DMARC

[Zrin Ž. <zri...@zi...>]

Hello Thomas,

I still find domains in the DKIMCache which do not use DKIM, but do have 
DMARC policy. Subsequently, messages from that domains which do not 
arrive signed get undeserved score.

What has changed within last few months is that many domains got a DMARC 
entry, but do not use DKIM, I guess because a big player decided that 
any domain that sends e-mails shall have one.


The sub DKIMCacheAdd is called in sub DKIMpreCheckOK_Run, sub DMARCok 
and sub DKIMOK_Run.

- Can you please check and make sure that sub DKIMCheckAdd is called 
only for domains for which there is a signature in the e-mail which 
exactly matches the sender (from) domain?

- Can you add debug output so adding a domain to DKIMCache gets logged 
and we can identify which e-mails get a domain added to DKIMCache where 
it should not get added?

In sub DMARCok:

     if (! $this->{DMARC_arc} && $DKIMCacheStrict) {
         DKIMCacheAdd($this->{dmarc}->{domain}) if $this->{dmarc}->{domain} && $this->{dmarc}->{domain} ne $this->{dmarc}->{dom};
         DKIMCacheAdd($this->{dmarc}->{dom}) if $this->{dmarc}->{dom};
         for my $dom (@{$this->{dmarc}->{DKIMdomains}}) {
             DKIMCacheAdd($dom) if $dom && $dom ne $this->{dmarc}->{domain} && $dom ne $this->{dmarc}->{dom};
         }
     }

Do I understand correctly that the domain will get added to DKIMCache if 
the message does not contain ARC and $DKIMCacheStrict is true - no 
matter if the domain actually uses DKIM?
As many domains have DMARC entry in the DNS but do not use DKIM - is 
this the source of the problem?


In sub DKIMpreCheckOK_Run:

DKIMCacheAdd($domain) if $dkimdomain && ($qdtxt ne 'unknown' || $DKIMCacheStrict);

- Can you please check that $dkimdomain is true only if the domain 
really uses DKIM keys?


Please let me know if I can help any further.

Thank you very much in advance,
best regards,
Zrin

[Assp-test] Recent problems with RebuildSpamdb

[Zrin Ž. <zri...@zi...>]

Hello Thomas,

during last 3 months I've noticed these problems that prevented 
RebuildSpamdb from successfully completing the rebuild on my system:

1.

2025/03/15 01:53:11 [Worker_10001] Error: rebuildspamdb failed - Corrupted storable string (binary v2.11) at /usr/lib/x86_64-linux-gnu/perl/5.32/Storable.pm line 471, at (eval 1160) line 75 thread 7.

- Storable has version 3.21
- the list of Perl modules in ASSP shows no problems.

2.

2025/03/09 22:55:12 [Worker_10001] BerkeleyDB-ERROR: in mounting BerkeleyDB /var/assp/tmpDB/rebuildDB/FileModel.bdb - Object #592128 should have been retrieved already at /usr/lib/x86_64-linux-gnu/perl/5.32/Storable.pm line 471, at (eval 986) line 75 thread 7.

- I could work around this problem by deleting the contents of 
tmpDB/rebuildDB/ and starting rebuild manually.

Please let me know what to do / if I can help debugging this.

Recent log:

2025/04/02 01:53:01 [Worker_10001] Info: found module /var/assp/lib/rebuildspamdb.pm version 8.24
2025/04/02 01:53:01 [Worker_10001] Rebuild debug output is enabled to /var/assp/rebuilddebug.txt
2025/04/02 01:53:01 [Worker_10001] RebuildSpamDB uses BerkeleyDB for temporary hashes
2025/04/02 01:53:01 [Worker_10001] RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte
2025/04/02 01:53:11 [Worker_10001] Internal FileModel has 119126 entries after maintenance
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB reloaded and uses the internal FileModel (BDB with 119126 entries) to speedup processing
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB-thread rebuildspamdb-version 8.24 started in ASSP version 2.8.2(25075)
2025/04/02 01:53:11 [Worker_10001] Detection of local disclaimers is enabled
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will create a Hidden Markov Model
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will include attachment-database-entries into spamdb
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will create unicode enabled databases
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will process all words as Sequence of UAX #29 Grapheme Clusters
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will normalize unicode characters
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will use the ASSP_WordStem engine
2025/04/02 01:53:11 [Worker_10001] RebuildSpamDB will create private spamdb entries for users email addresses and each local domain..
2025/04/02 01:53:11 [Worker_10001] Maxbytes: 12,000
2025/04/02 01:53:11 [Worker_10001] Maxfiles: 70,000
2025/04/02 01:53:11 [Worker_10001] RebuildFileTimeLimit: 2 7
2025/04/02 01:53:11 [Worker_10001] RebuildFileTimeLimit: files will be moved away from the corpus if their processing takes longer than 7 second(s)
2025/04/02 01:53:11 [Worker_10001] Info: Trashlist cleaning finished, 2 of 2 files deleted
2025/04/02 01:53:11 [Worker_10001] /var/assp/errors/spam
2025/04/02 01:53:11 [Worker_10001] File Count:  12,304
2025/04/02 01:53:11 [Worker_10001] Processing... errors/spam with 12,304 files
2025/04/02 01:53:11 [Worker_10001] Ignore and remove files older than 2021/12/19 00:53:11 in folder errors/spam
2025/04/02 01:53:12 [Worker_10001] Error: rebuildspamdb failed - Corrupted storable string (binary v2.11) at /usr/lib/x86_64-linux-gnu/perl/5.32/Storable.pm line 471, at (eval 1097) line 75 thread 7.
2025/04/02 01:53:13 [Worker_10001] Info: RebuildSpamdb Scheduler stopped
2025/04/02 01:53:13 [Worker_10001] Info: ReStart Scheduler stopped
2025/04/02 01:53:13 [Worker_10001] Info: starting RebuildSpamdb Scheduler with '53 1 * * 3,6' - next RebuildSpamdb is scheduled for Saturday 2025/04/05 01:53:00 +02:00
2025/04/02 01:53:13 [Worker_10001] Info: starting ReStart Scheduler with '33 1 * * 0,2,4' - next ASSP-ReStart is scheduled for Thursday 2025/04/03 01:33:00 +02:00

Thank you,
best regards,
Zrin

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25075

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25075:

- the output of the mail analyzer was partly destroyed, if a regex match 
was found for a HTML-tag

- if a host name or domain name that contained a dash was used in the 
'work with IP-addresses dialog',
  no or bad output was given by assp, because the dash was interpreted as 
a minus sign.
  Now a minus sign requires a leading space in every case in this dialog!

changed:

- not visible text is now removed from HTML content of a mail for bayesian 
and HMM analyzes (e.g. display: none; visibility: hidden;)

- 'removeDispositionNotification' headers are now also removed from 
noprocessing mails

-  'AddRegexHeader' is now a list option
   'AddRegexHeader','Add RegEx Match Header','0:no|1:incoming 
only|2:outgoing only|3:all'



Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25042

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25042:

- Requesting a change of the OpenSSL-securitiy-level for all or a specific 
SSL-context may caused the SSL-context creation to fail.
  In some cases the SMTP-Worker was crashing with a SEGV error on such a 
request, because assp was using a wrong context pointer.

- The resend of a SMIME signed mail may led into a failing SMIME-signature 
check at the users mail client, because of a wrong correction
  or setting of the mail end sequence [CRLF.CRLF]

- A zero byte long file in the resend folder caused an infinity looping of 
the Maintenance-Thread (10000)


changed:

'SSL_cipher_list' description added
... The openssl-security-level may be set using something like 
'DEFAULT:@SECLEVEL=0' 
(e.g. in case SSL/TLS is no longer working after upgrading openssl from 
V1.x.x to V3.x.x - 
read the openssl documentation for SSL_CTX_get_security_level and 
SSL_CTX_set_security_level ).
Setting the security level of openssl to zero is highly NOT recommended 
and should be only used as a temporary solution

Thomas

Re: [Assp-test] E-Mails in errors/spam/newManuallyAdded

[Thomas E. <tho...@th...>]

There are no changes in the code for processing this folder.
But - if the first processed file contains the unsupported charset 
'csEUCKR' , the rebuild thread dies.
The latest build 25014 solves this problem.

If you use a lower build, move all files out of this folder and restart 
assp.

Thomas



Von:    "Zrin Žiborski" <zri...@zi...>
An:     ass...@li...
Datum:  10.01.2025 14:25
Betreff:        [Assp-test] E-Mails in errors/spam/newManuallyAdded







Hi Thomas,

until recently ASSP would pick up and move the .eml files from 
errors/spam/newmanuallyAdded every few minutes or so.
It seems that doesn't work any more since recently.

I've checked the permissions on the file system level and checked that 
maillogExt ist still .eml.

What could be the cause / where to look next?

Thank you,
Zrin




_______________________________________________
Assp-test mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 25014

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 25014:

- If a server/client was sending large base64 encoded attachments using 
small SSL/TLS frames, the transmission of the mail may took
  too long, which may leaded in to an SMTP-timeout

- If a mail contained MIME-encoded 'csEUCKR' header lines
  like:  Subject: =?csEUCKR?B?KLGksO0pIMPWsO3AxyDB97/4?= PC 
=?csEUCKR?B?uPC0z8XNuLU=?=
  the rebuildspamdb task crashed on an error: Malformed UTF-8 character 
(fatal) at sub main::fixsub line 5
  This is caused by a missing codepage alias for 'csEUCKR' -> 'euc-kr' in 
the Encode module of perl.
  Because of this missing alias, the decoding of the header line to UTF8 
was incorrect and leaded in to malformed UTF8 characters
  If this alias is missing, it is added by assp at runtime.
 

Thomas

[Assp-test] E-Mails in errors/spam/newManuallyAdded

[Zrin Ž. <zri...@zi...>]

Hi Thomas,

until recently ASSP would pick up and move the .eml files from 
errors/spam/newmanuallyAdded every few minutes or so.
It seems that doesn't work any more since recently.

I've checked the permissions on the file system level and checked that 
maillogExt ist still .eml.

What could be the cause / where to look next?

Thank you,
Zrin

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 24340

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 24340:


- 'SpamLog' was not shown in the GUI left alphaindex - only 'spamlog' was 
shown

- because of changes in the time representation code in build 24291, 
blockreports were only done for the current day


Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 24334

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 24334:


- the timezone difference to GMT was only calculated once at start
  if assp was running while DST has changed - schedule times were wrong 
calculated

- if a mail contained more than one DKIM identity, only the first one was 
used in related features
 

changed:

- The following was added to the scheduler help text:

If your systems timezone is other than UTC and within the timezone 
daylight saving is used, there will be two special time ranges in a year.
1. the day were the time is set forward (e.g. march sunday last) - which 
leads in to a time range with invalid time values (e.g. CET->CEST 
02:00:00-02:59:59)
2. the day were the time is set backward (e.g. october sunday last) - 
which leads in to a time range with ambiguous time values (e.g. CEST->CET 
02:00:00-02:59:59)
If a scheduled time is in such a range, the execution time is set one hour 
later in case 1, in case 2 the first (lower epoch) time is used.
If in case 2 a (re)schedule is done within the first time and the new 
scheduled time is valid in the second time (higher epoch), the scheduled 
time is used.
For example:
a task scheduled with "30 2 * * *" will run at 3:30 in case 1 and at 2:30 
(DST on) in case 2.
a task scheduled with "30 * * * *" will run at 3:30 in case 1 and in case 
2 at 2:30 (DST on) and 2:30 (DST off).

The scheduler code was changed to provide this behavior.
Loglines that are related to the scheduler are now showing the timezone 
name (like CET) and the timezone difference to GMT (like +01:00)


- The added Authentication-Results: header contains now more details about 
SPF,DKIM and DMARC results


added:

'noSpoofingCheckDKIMIdentity','Don\'t do Spoofing Check for these DKIM 
Identities and DKIM From: and Sender:*'
 'If a mail is DKIM signed by a local domain and the DKIM identity or the 
header FROM or the header SENDER address matches, spoofing is not checked 
for any address.
 To make this feature working DoNoSpoofing4From has to be enabled!
 Accepts specific addresses (us...@ex...), user parts (user) or 
entire domains (@example.com). Wildcards are supported 
(fribo*@example.com).'
 

Thomas

Re: [Assp-test] DMARC record and DKIMcache (ASSP 2.8.2)

[Thomas E. <tho...@th...>]

>It seems to me that it does that and that it later complains that the 
incoming e-mails from that domain do not have DKIM-Signature:

ASSP creates a DKIMCache entry, if a valid DKIM-signature is found 
(DKIM-signature or ARC-result) - this has primary nothing to do with 
DMARC.
Any further mail has to be valid DKIM-signed - this is the default 
behavior for much more than a decade now.

This behavior can be changed

our $DKIMCacheStrict = 1;                # (0/1) if a DKIM signature is 
found for a domain - all other mails from this domain will require a DKIM 
signature to pass the Pre-DKIM-Check

by setting $DKIMCacheStrict to zero (use the commandline switch or 
lib/CorrectASSPcfg.pm to change this variable).

Nearly 100% of sending domains are constantly using DKIM or not. A valid 
DKIM-signature can't be faked by spammers. So knowing a domain uses DKIM 
(we saw it before) and now getting a mail from this domain without a 
DKIM-signature, is a very good indicator for SPAM!
Exceptions can be defined in the assp DKIM-check configuration and 
$DKIMCacheStrict.

Published DMARC policies are ignored by assp for the SPF and DKIM checks 
itself. They are only used to check/report DMARC.
So even the [p=] and/or [sp=] DMARC flags tells us to ignore mistakes - we 
do this for the DMARC check ---- a failed SPFcheck remains a miss and a 
failed DKIMcheck remains a miss!
Possibly we can make assp more strict. If a domain published a DMARC 
record, there is no doubt that a SPF record has to be defined a valid DKIM 
signature has to be included. If the 'aspf' and/or 'adkim' alignment rules 
are not defined, they are set to the default value 'r'.
Any miss or missmatch would be an SPF/DKIM error and a DMARC-alignment 
error. IMHO currently this would lead in to too many false detection and 
blocked good mails, because of bad configured DNS.



>Also, I've just noticed that ASSP checks and gives negative score for 
IP/HELLO for authenticated e-mail clients:

This is the case for ages  - I can't remember a time when this was not the 
case!
The last change was at 2015-05-10 in assp 2.4.4 build 15130
This feature was last touched in assp 2.8.2 *SPAM-Eliminator* build 24291 
to fix a punicode issue - where punicode domains.were unexpected scored.

The 'authenticated' flag is nowhere used in assp to make any check 
exception. If a client is connected to listenport2 and it is 
authenticated, then it is allowed to send outgoing mails (relayok) - this 
flag skips most checks (also IPinHeloOK).


Thomas




Von:    "Zrin" <zri...@zi...>
An:     ass...@li...
Datum:  07.11.2024 12:45
Betreff:        [Assp-test] DMARC record and DKIMcache (ASSP 2.8.2)







Hi Thomas,
does ASSP create DKIMcache entry for domains that have published a DMARC 
record (e.g. "v=DMARC1; p=none" under _dmarc.example.com) but do not have 
DKIM, i.e. there is no mail._domainkey.example.com?
It seems to me that it does that and that it later complains that the 
incoming e-mails from that domain do not have DKIM-Signature:
[scoring] DKIM domain mismatch - example.com found in DKIMCache, but no 
DKIM-Signature found in mail header (Cache)
Message-Score: added 25 (dkimValencePB) for DKIM domain mismatch - 
example.com found in DKIMCache, but no DKIM-Signature found in mail 
header, total score for this message is now 35
Message-Score: added -5 (spfpValencePB) for SPF pass, total score for this 
message is now 30
info: domain ziborski.net has published a DMARC record

Also, I've just noticed that ASSP checks and gives negative score for 
IP/HELLO for authenticated e-mail clients:
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 
<client externel IP>] <fr...@ex...> to: rec...@do... 
Originating IP/HELO:  <client ext IP> / [192.168.xxx.xxx]
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 
212.17.78.158] <fr...@ex...> to: rec...@do... Message-Score: 
added 5 (fiphValencePB) for Suspicious HELO - contains IP: 
'[192.168.xxx.xxx]', total score for this message is now 5
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 
212.17.78.158] <fr...@ex...> to: rec...@do... [scoring] 
(Suspicious HELO - contains IP: '[192.168.xxx.xxx]')
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 
212.17.78.158] <fr...@ex...> to: rec...@do... Message-Score: 
added 5 (fiphmValencePB) for IP in HELO '[192.168.xxx.xxx]' does not match 
IP in connection '<client ext ip>' , total score for this message is now 
10

It seems to me that this wasn't the case before update to 2.8.x (?)
Thank you in advance,
Zrin_______________________________________________
Assp-test mailing list
Ass...@li...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Assp-test] DMARC record and DKIMcache (ASSP 2.8.2)

[Zrin <zri...@zi...>]

Hi Thomas,

does ASSP create DKIMcache entry for domains that have published a DMARC 
record (e.g. "v=DMARC1; p=none" under _dmarc.example.com) but do not 
have DKIM, i.e. there is no mail._domainkey.example.com?

It seems to me that it does that and that it later complains that the 
incoming e-mails from that domain do not have DKIM-Signature:

[scoring] DKIM domain mismatch - example.com found in DKIMCache, but no DKIM-Signature found in mail header (Cache)
Message-Score: added 25 (dkimValencePB) for DKIM domain mismatch - example.com found in DKIMCache, but no DKIM-Signature found in mail header, total score for this message is now 35
Message-Score: added -5 (spfpValencePB) for SPF pass, total score for this message is now 30
info: domain ziborski.net has published a DMARC record

Also, I've just noticed that ASSP checks and gives negative score for 
IP/HELLO for authenticated e-mail clients:

2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: <client externel IP>]<fr...@ex...>  to:rec...@do...  Originating IP/HELO:  <client ext IP> / [192.168.xxx.xxx]
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 212.17.78.158]<fr...@ex...>  to:rec...@do...  Message-Score: added 5 (fiphValencePB) for Suspicious HELO - contains IP: '[192.168.xxx.xxx]', total score for this message is now 5
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 212.17.78.158]<fr...@ex...>  to:rec...@do...  [scoring] (Suspicious HELO - contains IP: '[192.168.xxx.xxx]')
2024/11/07 11:57:56 77076-69079 [Worker_1] [TLS-in] <ASSP IP> [OIP: 212.17.78.158]<fr...@ex...>  to:rec...@do...  Message-Score: added 5 (fiphmValencePB) for IP in HELO '[192.168.xxx.xxx]' does not match IP in connection '<client ext ip>' , total score for this message is now 10

It seems to me that this wasn't the case before update to 2.8.x (?)

Thank you in advance,
Zrin

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 24291

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 24291:

- If a client connected to assp by sending a HTML connection header, or 
invalid SMTP commands were used, or a sent SMTP command line or any data 
line was not
  correctly terminated by a linefeed - and the client stopped sending data 
after that (eg. never sent a LF) - the assp SMTP-timeout was never reached 
and the connection was
  never closed by assp. This case may led in to high CPU usage, high 
memory usage and stucking workers.

- improved output of connection timeout debug

- if a client HELO contained a punycode host or domain (like 
xn--80avu.042.xn--p1acf), assp possibly detected an IP address <> HELO 
missmatch,
  even the HELO contained no IP address

- under certain circumstances it was possible, that assp has an unicode 
error shown in the log while it was checking a mail for bomb... regular 
expressions

- under rare circumstances the subaddressing with 'SepChar' was not 
working for header addresses
  (introduced in build 24169)

 
Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 24289

[Thomas E. <tho...@th...>]

Hi all,


fixed in assp 2.8.2 *SPAM-Eliminator* build 24289

- build 24288 destroys mails - reverted back to 24282 as 24289:

I'm sorry !!!

Thomas

[Assp-test] fixes in assp 2.8.2 *SPAM-Eliminator* build 24288

[Thomas E. <tho...@th...>]

Hi all,

fixed in assp 2.8.2 *SPAM-Eliminator* build 24288:

- If a client connected to assp by sending a HTML connection header, or an 
invalid SMTP commands were used, or a sent SMTP command line or any data 
line was not
  correctly terminated by a linefeed - and the client stopped sending data 
after that (eg. never sent a LF) - the assp SMTP-timeout was never reached 
and the connection was
  never closed by assp. This case may led in to high CPU usage, high 
memory usage and stucking workers.


Thomas