Webshield does not retry temporary mail delivery failures
Status: Beta
Brought to you by:
waynemcdougall
Menu
▾
▴
#17 Webshield does not retry temporary mail delivery failures
Microsoft Exchange Server is sometimes configured to
bounce email, without retrying, if it receives a temporary
failure error when trying to deliver mail.
Fluffy is set by default to generate a temporary failure
error when a 'new' site attempts to send email - the
theory (and it works in practice) is that many spamming
systems to not retry (or are blacklisted by the time they
do retry) while genuine email from normal mail servers
will try again in a short period of time. This is known
as "tempfailing".
The fault is with the Exchange Server configuration, but
it is not always possible to get Exchange Server
administrators to correct their configuration.
If you know the Exchange Server you can whitelist it in
Fluffy - this prevents the application of any deferrment.
The question is: can we use a different response code
that will not trigger Exchange Servers immediate
rejection of the message, but will still force it to retry?
Attached are two files in Zip format. One is Flufyy
version 1.1.26 and the other is a .REG file which will add
a registry entry. This version of Fluffy will use this
registry entry (if present) as the response status code
(the default is 450). Interesting numbers to test with
Exchange Server are 451,452 (being common codes)
400,401 (the initial codes in the temporary failure
range), 410,411 (being codes that aren't at the very
start) 420,421 (where 421 is an existing known code
indicating that we have to shut down imminently - try
again later), and 499 (as the last code in the range).
Change the response code is down either by editing the
tempfauil.reg file (and adding it to the registry) or
directly editing the registry. If you are going to do these
testsm, and you aren't familiar with this process (to add
a .REG file to the registry just double click on it), please
feel free to ask.
It would also be interesting to see the actual bounce
message generated by Exchange as this may offer some
clues. And at log level 10, Fluffy will log the entire SMTP
conversation when Exchange tries to deliver the mail,
and this also may be informative.
I do not hold out much hope. There is no doubt in my
mind that thiis faulty behaviour by Exchange (retries=0),
but if someone is in a position to test and find a generic
workaround, so much the better.
This bug will remain open until someone has the chance
to run the tests outlined above and report back. Later
Fluffy versions will continue to support this response
code/registry lookup until the bug is closed.
Fluffy 1.1.26 and tempfail.reg
Logged In: YES
user_id=431787
I had the opportunity to discuss this with the sysadmin of a
mail server where this was an issue. He said that all mail was
sent from Exchange via Webshield. It was Webshield that
didn't retry. He told Exchange not to use Webshield for any
mail to my address, but he didn't fix Webshield - maybe it is a
bug in that software.
Logged In: YES
user_id=660239
Good spot Neil. Thanks.
My thinking (could be wrong) is this WebShield is now the
SMTP part of the GroupSheild programme from Network
Associates. There are various complaints around the net
about this problem, although from the sender (because people
complain mail is coming through late or not at all) rather than
the recipient. All the complains I saw referredto an old copy
ofGroupshield, so maybe it has been fixed. I don't know how
current the version being run at your Sysadmin's friend is, or
whether they've applied all the patches.
IF there was a way of recognising a WebShield/Groupshield
connection (a distinct pattern in the greeting) I could tune
Fluffy not to defer connections from such sites.
I can't really progress any further without a log (level 10) of
the SMTP conversation with such a site, or the IP address of
a site. I'll see if I can find someone running GroupShield.
Logged In: YES
user_id=431787
It is part of GroupShield. It looks like they might etry properly
with code response code 442. I am trying this response code
to see if it helps. I assume 'normal' servers will treat it in the
same way as 450?
I found this in Network Associates knowledge base:
Understanding SMTP error code 442
Solution ID : nai24630 Last Modified : 5MAR 2003
Goal and/or Problem Description
Understanding SMTP error code 442
SMTP Error 442
Error 442
Deferred mail
Mail is being deferred
------------------------------------------------------------
--------------------
Problem Environment
McAfee Webshield SMTP 4.5
Microsoft Windows NT
Microsoft Windows 2000
------------------------------------------------------------
--------------------
Changes affecting this problem
Change information is not available for this solution
------------------------------------------------------------
--------------------
Cause of this problem
442 is a Extended SMTP error code:
From SMTP RFC1893:
4.X.X - Persistent Transient Failure A persistent transient
failure is one in which the
message as sent is valid, but some temporary event prevents
the successful sending of the message. Sending in the future
may be successful.
4.4.X - Network and Routing Status The networking or routing
codes report status
about the delivery system itself. These system components
include any necessary
infrastructure such as directory and routing services. Network
issues are assumed
to be under the control of the destination or intermediate
system administrator.
4.4.2 - Bad connection The outbound connection was
established, but was otherwise unable to complete the
message transaction, either because of time-out, or
inadequate connection quality. This is useful only as a
persistent transient error.
------------------------------------------------------------
--------------------
Solution 1:
WebShield will defer emails with a 442 SMTP error and retry
delivery according to GUI Deferral settings.
Logged In: YES
user_id=660239
You are correct. A properly configured SMTP server will treat
any 4xx as a temporary failure and will retry. While it is
possible a badly written server would look specifically for 450
(and the RFCs now say to use the same code and the same
text as given in examples to try and cope with this), this
probably wouldn't be a problem, because they should also be
looking for other codes (eg 421) that indicate a temporary
failure, and furthermore if they don't understand a code they
should retry again later.
But you are incorrect to think that 442 is the SMTP response
code. The knowledge base is talking about enhanced status
codes. RFC2034. So Webshield would generate a 442 error
code if it got a response like:
450 4.4.2 Deferring mail
This doesn't make much sense thought, because RFC1893
defines x.4.2 as:
X.4.2 Bad connection
The outbound connection was established, but was
otherwise
unable to complete the message transaction, either
because
of time-out, or inadequate connection quality. This is
useful only as a persistent transient error.
And we don't want to say it is a persistent transient error.
This gives me a clue, though...
By default I send 450 4.7.0 Deferring mail message, and 4.7.0
means Other or undefined security status
Something related to security caused the message to
be
returned, and the problem cannot be well expressed
with any
of the other provided detail codes. This status code
may
also be used when the condition cannot be further
described
because of security policies in force.
It might be that GroupShield is misbehaving when it gets that
status code.
So leave it at 450 Neil, but try changing the 4.7.0 part and
see if that works:
You could try removing the status code (4.7.0) altogether
(it's not necessary) and see how that works. Then you could
try
4.6.0
4.3.2
4.3.0
4.2.1
4.2.0
4.2.5
4.2.999
(Since some mail servers support EnhancedStatusCodes I'm
really requried to send some code since I'm acting as a proxy
server - the codes are listed in descending order of
preference, by which I eamn the first listed is closest to the
meaning that I'm trying to report).
Sorry to lay this on you, but it would be much appreciated.
But just see if dropping the status code fixes things to begin
with. If it doesn't then no change will help.
Logged In: YES
user_id=660239
Further comment: I think the 4.4.2 status reported in the NAI
knowledgeabase refers to a message generated by Webshield
whenit defers mail it couldn't send, as opposed to reporting
an error emssage it gets back.
Still worth seeing what happens if you
a) drop the enhanced status code
b) try changing the 450
Logged In: YES
user_id=431787
OK. How do I drop the enhanced status code?
By the way, the tempfail.reg file you posted needs a CR at
the end for it to work - it doesn't change the registry even
though there is a successfully changed message after double
clicking it. It probably gets stripped in transit.
Logged In: YES
user_id=660239
How to drop the enhanced status code - it's expected at the
start of the message, so juyst keep adding extra text in your
registry entry. So if you enter
460 No enhanced status code here
it will get converted to
460 No enhanced status code here 4.6.0 We haven't seen
mail from your lately please try again in 5 minutes.
so if the enhanced status codes are the problem, hopefully it
wouldn't be looking for them in the middle of the message
(the standard requires them to follow the 3 digit SMTP
response code.
The current version of Fluffy (formal 1.3 release) is NOT
sending the enhanced status codes anyway (because I
dropped them as part of our direct testing, which I'm happy
to continue), but I will need to put them back soon, to be
compliant with mail servers using Enhanced status codes.
All clear?
I'm standing by for another test message to
webshield@codeworks.gen.nz
I will be sending a 460 response code, and no enhanced
status codes, so we will see how it looks that. If you don't
mind. At your convenience.
Logged In: YES
user_id=660239
Webshield will correctly retry a mail message after getting an
SMTP response code of 400. Attached is rempfail.reg which
will add a registry entry setting forcing Fluffy to use this
response code on temporary deferral SMTP response
messages. I will make this response of 400 the default on all
future releases of Fluffy. There is no reason to expect this will
affect any other mail server, as all mail servers are supposed
to treat any response code beginning with 4 the same way.
Of course, Webshield does not.
I will be contacting Network Associates to popint out the
flaws in their software. Other users of Webshield may ike to
do the same. The fault is that Websheild does not retry
sending mail messages after getting an SMTP response code
of 450.
I'm interested in the fact that code 400 does work. That
suggests either there is a bug in their faulty implementation.
I'd guess that the code might be:
if smtpresponse>400 and smtpresponse<600 then
reportmessagefailed
whereas it should read:
if smtpresponse>=400 and smtpresponse<600 then
reportmessagefailed
Still, a bug in their implementation of incorrect behaviour
means it does work corectly in the case of 400.
Of course the otehr possibility is that they tried to do it right,
but instead of the range 400-499 they only correctly process
400.
Any comments welcome.
Tempfail.reg to set Fluffy defer response compatible with Webshields faulty behaviour
Logged In: YES
user_id=660239
My grateful thanks to Neilx for his assistance in letting me do
the testing requried to find a workaround for the Webshield
design flaw.
Logged In: YES
user_id=660239
The problem is with Network Associates (McAfee)'s Webshield
product. The Webshield 4.5 MR1a version does not retry mail.
This is due to a timing failure - it does not handle dates
1,000,000,000 seconds after 1 January 1970, which occurred
in 1999.
There is a fix for this bug available from Network Associates
on request - Hotfix 8. It is also possible to find this Hotfix by
searching the web, eg on:
http://www.itap.purdue.edu/security/files/Wsmr1hf8.zip
This is not a Fluffy problem but a bug in Webshield. It will
affect any Webshield user trying to send mail, which is not
immediately delivered. Fluffy compounds the problem if (as it
does by default) it defers a connection from a new source.
Websheild then will not retry, but bounce the message back.
Reponses:
* the Webshield system should have the Hotfix patch applied
* there is no way of detecting whether the connecting server
to Fluffy is a bugg version of Webshield, or a Webshield
version at all
* we could add, to the defer notice, an advisory about
Webshield. In theory only Webshield users would see this.
* we could connect back to the sending server and if it is
Webshield, alter our advisory message (or not defer at all)
* we could scan through Fluffy's records for all deferred mail
connections that didn't retry, connect to them and see if
they are running Webshield, and if so, send their postmaster
and attempted sender and advisory on the bug
I'm open to suggestions on this.
Logged In: YES
user_id=660239
In the absence of any further comments, I will close this bug
report. The fault is an inherent one with Webshield and there
is a fix available from, the makers of Webshield. The fault isn't
just with Fluffy, but Fluffy's deferral option will cause the bug
in Webshield to manifest itself regularly.