Menu
▾
▴
dkim-milter-discuss
|
From: Jukka S. <j+d...@20...> - 2007-12-13 10:41:34
|
Hello, yesterday someone tried to send a message to my primary mail server running dkim-milter 2.4.0 and Postfix 2.4.5, but dkim-milter failed: Dec 12 20:30:55 grouper postfix/cleanup[6694]: warning: milter unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 [...] Dec 12 20:30:55 grouper postfix/cleanup[6694]: 23B0A24C85: milter-reject: END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - try again later; [...] [...] Dec 12 20:30:55 grouper dkim-milter[4895]: 23B0A24C85 "X-DKIM" header add failed One second later my backup MX which runs the same dkim-milter and Postfix versions successfully received the message and forwarded it to the primary mail server where it was received just fine. Unfortunately the message has been deleted in the meantime, but I'm quite sure it was [1]this message which seems to have a missing or null body. I haven't yet managed to reproduce the problem. Has anybody ever seen this? Regards, Jukka [1] http://www.suckless.org/pipermail/dwm/2007-December/004541.html -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Tony E. <to...@he...> - 2007-12-13 12:25:52
|
Jukka Salmi skrev, on 13-12-2007 11:41: > Hello, > > yesterday someone tried to send a message to my primary mail server > running dkim-milter 2.4.0 and Postfix 2.4.5, but dkim-milter failed: > > Dec 12 20:30:55 grouper postfix/cleanup[6694]: warning: milter unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 > [...] > Dec 12 20:30:55 grouper postfix/cleanup[6694]: 23B0A24C85: milter-reject: END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - try again later; [...] > [...] > Dec 12 20:30:55 grouper dkim-milter[4895]: 23B0A24C85 "X-DKIM" header add failed > > One second later my backup MX which runs the same dkim-milter and > Postfix versions successfully received the message and forwarded it > to the primary mail server where it was received just fine. > > Unfortunately the message has been deleted in the meantime, but I'm > quite sure it was [1]this message which seems to have a missing or > null body. > > I haven't yet managed to reproduce the problem. Has anybody ever seen > this? That's Postfix stuff from milter.c, so it's not very likely that Sendmail people here can answer. I haven't had that (Postfix 2.4.6, dkim-filter 2.4.0), but I occasionally get "can't read SMFIC_HEADER reply packet header: Connection reset by peer" when over-eager mailers send HUGE CCs or TOs to scores of recipients. It just means Postfix doesn't sign the message; it doesn't die, thank $DEITY, like it does if it can't contact amavisd-new or my dspam daemon. Could be worth taking up on the PF ML - I haven't bothered up to now. FWIW I gave up using domain sockets for Postfix milters long ago, only led to tears. Everything's exclusively inet, now. Best, --Tonni -- Tony Earnshaw Email: tonni at hetnet dot nl |
|
From: Murray S. K. <ms...@se...> - 2007-12-13 17:43:47
|
On Thu, 13 Dec 2007, Tony Earnshaw wrote: > That's Postfix stuff from milter.c, so it's not very likely that > Sendmail people here can answer. I haven't had that (Postfix 2.4.6, > dkim-filter 2.4.0), but I occasionally get "can't read SMFIC_HEADER > reply packet header: Connection reset by peer" when over-eager mailers > send HUGE CCs or TOs to scores of recipients. It just means Postfix > doesn't sign the message; it doesn't die, thank $DEITY, like it does if > it can't contact amavisd-new or my dspam daemon. Mark opened a bug against dkim-filter for rejecting messages with oversized headers (over 32k of total header size). This is actually intended as protection against a denial-of-service attack, but the hard-coding of the limit isn't especially friendly. The next release will make that limit configurable. The logging though is a little confusing; if mlfi_header() returns SMFIS_REJECT, the recovery of the sending MTA should be graceful. |
|
From: Tony E. <to...@he...> - 2007-12-13 18:40:29
|
Murray S. Kucherawy skrev, on 13-12-2007 18:43: >> That's Postfix stuff from milter.c, so it's not very likely that >> Sendmail people here can answer. I haven't had that (Postfix 2.4.6, >> dkim-filter 2.4.0), but I occasionally get "can't read SMFIC_HEADER >> reply packet header: Connection reset by peer" when over-eager mailers >> send HUGE CCs or TOs to scores of recipients. It just means Postfix >> doesn't sign the message; it doesn't die, thank $DEITY, like it does if >> it can't contact amavisd-new or my dspam daemon. > > Mark opened a bug against dkim-filter for rejecting messages with > oversized headers (over 32k of total header size). This is actually > intended as protection against a denial-of-service attack, but the > hard-coding of the limit isn't especially friendly. It's interesting that Mark has his own (Perl) amavisd-new (that's his own daemon glue for content that is primarily directed at extensions and AV, but has also glue for anti-spam and all sorts of other useful Postfix things, without which my life would be much more difficult) baby, a Perl-based DKIM milter implementation. It would be just as interesting to know how Mark has solved this problem. Recently Mark posted to the PF ML that he's fed up with Postfix DKIM milters and wishes to concentrate on his own own Perl-based amavisd-new-based solution. He had support from one of the Postfix developers. Me, I'm sticking to Sendmail DKIM milter. That having been said, he helped me (as did you), personally, over the Sendmail dkim-milter "doesn't work" to "works now" hump. I owe him much. > The next release will make that limit configurable. I'd prefer it to be dynamic ;) . How am I as mailadmin supposed to know in advance how many CCs or TOs my users are going to use? > The logging though is a little confusing; if mlfi_header() returns > SMFIS_REJECT, the recovery of the sending MTA should be graceful. As far as Postfix 2.4.6 goes it is already graceful, in as much as it doesn't die; it just doesn't sign the message (one in 2000?). --Tonni -- Tony Earnshaw Email: tonni at hetnet dot nl |
|
From: Mark M. <Mar...@ij...> - 2007-12-13 18:31:43
|
Murray, > Mark opened a bug against dkim-filter for rejecting messages with > oversized headers (over 32k of total header size). This is actually > intended as protection against a denial-of-service attack, but the > hard-coding of the limit isn't especially friendly. > > The next release will make that limit configurable. The limit is not my concern, I don't mind that verification (or signing) for such message does nothing. The issue is that a reject should not be possible at all, having action set to 'accept' for all situations, including internal or protocol failures (-C dns=a,int=a). A verifying milter has no right to reject a message if it isn't explicitly configured for rejection of non-valid messages. > The logging though is a little confusing; if mlfi_header() returns > SMFIS_REJECT, the recovery of the sending MTA should be graceful. It gracefully rejects the message. It must not do that. Mark |
|
From: Murray S. K. <ms...@se...> - 2007-12-13 19:48:58
|
On Thu, 13 Dec 2007, Mark Martinec wrote: > The issue is that a reject should not be possible at all, having action > set to 'accept' for all situations, including internal or protocol > failures (-C dns=a,int=a). Well the real problem is that the header size limit doesn't fall into any of the categories covered by "-C". I'll have to add another. > A verifying milter has no right to reject a message if it isn't > explicitly configured for rejection of non-valid messages. Does a receiving MTA have the right to reject a message with properties it considers to be a possible attack attempt? >> The logging though is a little confusing; if mlfi_header() returns >> SMFIS_REJECT, the recovery of the sending MTA should be graceful. > > It gracefully rejects the message. It must not do that. The earlier remarks in this thread (i.e. from Jukka) suggest the rejection may be causing some other mysterious symptoms. |
|
From: Mark M. <Mar...@ij...> - 2007-12-13 20:05:20
|
Murray, > Well the real problem is that the header size limit doesn't fall into any > of the categories covered by "-C". I'll have to add another. How about a general catch-all setting, so that instead of having to list each one in -C (including the new ones potentially introduced with later versions), one could specify only one. E.g. instead of "-C dns=a,int=a" one could have something like "-C default=a" (or equivalent in a configuration file). > > A verifying milter has no right to reject a message if it isn't > > explicitly configured for rejection of non-valid messages. > > Does a receiving MTA have the right to reject a message with properties > it considers to be a possible attack attempt? Yes, MTA (or its filters) has this right. But a dedicated filter which is intended to check exactly one aspect of a message has no right to extend its vocation and say: "although I can't say anything about signatures/ssp, I believe this message is harmful to your eyes so I'll just step in and reject it" > > It gracefully rejects the message. It must not do that. > > The earlier remarks in this thread (i.e. from Jukka) suggest the > rejection may be causing some other mysterious symptoms. This could be something else. The rejection didn't cause any problems with MTA in my case, it cleanly logged a reject request and rejected the message. Mark |
|
From: Murray S. K. <ms...@se...> - 2007-12-13 20:24:05
|
On Thu, 13 Dec 2007, Mark Martinec wrote: > How about a general catch-all setting, so that instead of having to > list each one in -C (including the new ones potentially introduced > with later versions), one could specify only one. > > E.g. instead of "-C dns=a,int=a" > one could have something like "-C default=a" > (or equivalent in a configuration file). Sounds reasonable. Can you make it a feature request on SourceForge? >> Does a receiving MTA have the right to reject a message with properties >> it considers to be a possible attack attempt? > > Yes, MTA (or its filters) has this right. > > But a dedicated filter which is intended to check exactly one > aspect of a message has no right to extend its vocation and say: > "although I can't say anything about signatures/ssp, I believe this > message is harmful to your eyes so I'll just step in and reject it" There's no "harmful to your eyes" logic in play here. The issue is the arbitrary trust of user-provided data. The fact that a user can connect to an MTA and feed an unbounded number of headers thus causing dkim-filter to become gigantic is what I'm trying to avert. You're right though that the default action should probably not be to reject the message. |
|
From: Mark M. <Mar...@ij...> - 2007-12-13 21:02:55
|
Murray, > > E.g. instead of "-C dns=a,int=a" > > one could have something like "-C default=a" > > (or equivalent in a configuration file). > > Sounds reasonable. Can you make it a feature request on SourceForge? I have now clarified my request in 1842970. Mark |
|
From: Mark M. <Mar...@ij...> - 2007-12-13 20:32:05
|
Jukka, > yesterday someone tried to send a message to my primary mail server > running dkim-milter 2.4.0 and Postfix 2.4.5, but dkim-milter failed: > > Dec 12 20:30:55 grouper postfix/cleanup[6694]: warning: milter > unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet > header: Undefined error: 0 [...] > Dec 12 20:30:55 grouper postfix/cleanup[6694]: 23B0A24C85: milter-reject: > END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - try again later; > [...] [...] > Dec 12 20:30:55 grouper dkim-milter[4895]: 23B0A24C85 "X-DKIM" header add > failed Search the log for all events associated with this smtp session, including the logged 'connect'. Then check timestamps - perhaps dkim-milter took long to respond and Postfix gave up waiting (although it would normally say so, but perhaps unix socket behaves differently than inet socket in this respect, I haven't tried). > One second later my backup MX which runs the same dkim-milter and > Postfix versions successfully received the message and forwarded > it to the primary mail server where it was received just fine. > Unfortunately the message has been deleted in the meantime > but I'm quite sure it was [1]this message which seems to have > a missing or null body. I haven't yet managed to reproduce > the problem. Has anybody ever seen this? If you manage to reproduce the problem, capturing a milter session with tcpdump (full size packets, binary) could be of help. Mark |
|
From: Jukka S. <j+d...@20...> - 2007-12-15 12:28:31
|
Mark Martinec --> dkim-milter-discuss (2007-12-13 21:31:50 +0100): > Search the log for all events associated with this smtp session, > including the logged 'connect'. Then check timestamps - perhaps > dkim-milter took long to respond and Postfix gave up waiting > (although it would normally say so, but perhaps unix socket behaves > differently than inet socket in this respect, I haven't tried). While dkim-milter responded to this particular SMTP session within a second, I just noticed that each time this problem happens (i.e. dkim-milter fails with `"X-DKIM" header add failed'), there are two additional open connections from other MTAs which are not yet handled by dkim-milter - they both time out after dkim-milter fails: Dec 13 18:38:15 grouper postfix/smtpd[1471]: connect from lists-outbound.sourceforge.net[66.35.250.225] Dec 13 18:38:15 grouper postfix/smtpd[19583]: connect from lists-outbound.sourceforge.net[66.35.250.225] Dec 13 18:38:16 grouper postfix/smtpd[1471]: 12D8824C82: client=lists-outbound.sourceforge.net[66.35.250.225] Dec 13 18:38:16 grouper postfix/cleanup[15386]: 12D8824C82: message-id=<E1J...@sc...> Dec 13 18:38:16 grouper postfix/smtpd[19583]: 9D3A824C84: client=lists-outbound.sourceforge.net[66.35.250.225] Dec 13 18:38:17 grouper postfix/cleanup[16795]: 9D3A824C84: message-id=<E1J...@sc...> [...] Dec 13 18:43:04 grouper postfix/smtpd[23256]: connect from russian-caravan.cloud9.net[168.100.1.4] Dec 13 18:43:04 grouper postfix/smtpd[23256]: D651024C85: client=russian-caravan.cloud9.net[168.100.1.4] Dec 13 18:43:05 grouper postfix/cleanup[4541]: D651024C85: message-id=<20071213184252.456f5834@pennywise> Dec 13 18:43:05 grouper dkim-milter[4895]: D651024C85 "X-DKIM" header add failed Dec 13 18:43:05 grouper postfix/cleanup[4541]: warning: milter unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 Dec 13 18:43:05 grouper postfix/cleanup[4541]: D651024C85: milter-reject: END-OF-MESSAGE from russian-caravan.cloud9.net[168.100.1.4]: 4.7.1 Service unavailable - try again later; [...] Dec 13 18:43:05 grouper postfix/smtpd[23256]: disconnect from russian-caravan.cloud9.net[168.100.1.4] [...] Dec 13 18:43:16 grouper postfix/cleanup[15386]: warning: milter unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet header: Operation timed out Dec 13 18:43:16 grouper postfix/cleanup[15386]: 12D8824C82: milter-reject: END-OF-MESSAGE from lists-outbound.sourceforge.net[66.35.250.225]: 4.7.1 Service unavailable - try again later; [...] Dec 13 18:43:16 grouper postfix/smtpd[1471]: disconnect from lists-outbound.sourceforge.net[66.35.250.225] Dec 13 18:43:17 grouper postfix/cleanup[16795]: warning: milter unix:/sockets/dkfilter/dkimvrfy: can't read SMFIC_BODYEOB reply packet header: Operation timed out Dec 13 18:43:17 grouper postfix/cleanup[16795]: 9D3A824C84: milter-reject: END-OF-MESSAGE from lists-outbound.sourceforge.net[66.35.250.225]: 4.7.1 Service unavailable - try again later; [...] Dec 13 18:43:17 grouper postfix/smtpd[19583]: disconnect from lists-outbound.sourceforge.net[66.35.250.225] However, as Tony wrote, this probably belongs to the Postfix ML... > If you manage to reproduce the problem, capturing a milter session > with tcpdump (full size packets, binary) could be of help. I'm using unix domain sockets - AFAIK it's not possible to capture a session in this case, is it? Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Mark M. <Mar...@ij...> - 2007-12-16 01:54:31
|
On Saturday 15 December 2007 13:28:22 Jukka Salmi wrote: > While dkim-milter responded to this particular SMTP session within a > second, I just noticed that each time this problem happens (i.e. > dkim-milter fails with `"X-DKIM" header add failed'), there are two > additional open connections from other MTAs which are not yet handled > by dkim-milter - they both time out after dkim-milter fails: Interesting association of events. Does your dkim-filter run with auto-starting option - or better, is the PID as reported by dkim-milter after these event (on a next normal session) the same as before the event? > However, as Tony wrote, this probably belongs to the Postfix ML... I'm not so sure. Until you capture and analyze a failing milter session you won't know whether the problem report should be directed to postfix or to dkim-milter, or maybe even to the OS or its sockets layer. > I'm using unix domain sockets - AFAIK it's not possible to > capture a session in this case, is it? I don't think that is possible. This is why I'd suggest you switch to inet socket on a loopback interface. All you need to do is to change the socket specification on a MTA side and on a command line to a dkim-filter. Something like -p inet:4444@127.0.0.1 on a dkim-filter, and a smtpd_milters=inet:127.0.0.1:4444 on the Postfix side. This also does away with a tricks you need to play with Unix socket ownership and protection to make it work. Mark |
|
From: Jukka S. <j+d...@20...> - 2007-12-16 15:27:34
|
Mark Martinec --> dkim-milter-discuss (2007-12-16 02:54:20 +0100): > On Saturday 15 December 2007 13:28:22 Jukka Salmi wrote: > > While dkim-milter responded to this particular SMTP session within a > > second, I just noticed that each time this problem happens (i.e. > > dkim-milter fails with `"X-DKIM" header add failed'), there are two > > additional open connections from other MTAs which are not yet handled > > by dkim-milter - they both time out after dkim-milter fails: > > Interesting association of events. Does your dkim-filter run > with auto-starting option - or better, is the PID as reported > by dkim-milter after these event (on a next normal session) > the same as before the event? Auto-restarting is not enabled, dkim-milter's pid does not change. > > However, as Tony wrote, this probably belongs to the Postfix ML... > > I'm not so sure. Until you capture and analyze a failing milter > session you won't know whether the problem report should be > directed to postfix or to dkim-milter, or maybe even to the > OS or its sockets layer. > > > I'm using unix domain sockets - AFAIK it's not possible to > > capture a session in this case, is it? > > I don't think that is possible. This is why I'd suggest you > switch to inet socket on a loopback interface. All you need > to do is to change the socket specification on a MTA side > and on a command line to a dkim-filter. Something like > -p inet:4444@127.0.0.1 on a dkim-filter, and a > smtpd_milters=inet:127.0.0.1:4444 on the Postfix side. > This also does away with a tricks you need to play with > Unix socket ownership and protection to make it work. Ok, as soon as I manage to reproduce the problem I'll switch to an inet socket, capture the session and report results here. Thanks for your help so far! Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Jukka S. <j+d...@20...> - 2008-01-15 10:40:46
|
Jukka Salmi --> dkim-milter-discuss (2007-12-16 16:27:26 +0100): > Mark Martinec --> dkim-milter-discuss (2007-12-16 02:54:20 +0100): [...] > > I'm not so sure. Until you capture and analyze a failing milter > > session you won't know whether the problem report should be > > directed to postfix or to dkim-milter, or maybe even to the > > OS or its sockets layer. > > > > > I'm using unix domain sockets - AFAIK it's not possible to > > > capture a session in this case, is it? > > > > I don't think that is possible. This is why I'd suggest you > > switch to inet socket on a loopback interface. All you need > > to do is to change the socket specification on a MTA side > > and on a command line to a dkim-filter. Something like > > -p inet:4444@127.0.0.1 on a dkim-filter, and a > > smtpd_milters=inet:127.0.0.1:4444 on the Postfix side. > > This also does away with a tricks you need to play with > > Unix socket ownership and protection to make it work. > > Ok, as soon as I manage to reproduce the problem I'll switch to an > inet socket, capture the session and report results here. Thanks for > your help so far! I didn't manage to reproduce the problem deliberately, but since it usually occurs every few days I switched to inet sockets anyway and had tcpdump write the whole milter communication to a file. Yesterday the problem reoccurred, but I guess I need some help interpreting the results... This is still with dkim-milter 2.4.0 and Postfix 2.4.5 on NetBSD/i386 3.1, BTW. So far the problem always happens more or less like this: - a remote MTA connects to my local MTA (mx1) - another remote MTA connects to mx1 - a third remote MTA connects to mx1 - the message from the third MTA is rejected, dkim-milter logs "X-DKIM" header add failed and the MTA logs can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 - the messages from first and second MTA are rejected, the MTA logs can't read SMFIC_BODYEOB reply packet header: Operation timed out - the remote systems either deliver the message to my backup MTA (mx2) which forwards it to mx1, or they try again some time later to mx1 which succeeds this time A complete MTA [1]log is available (I garbled some email adresses...), as is a tcpdump [2]record of about the same period and some [3]notes listing the MTA log <--> TCP session mappings. More context of the MTA log and tcpdump session record is available on request. I put online what I thought is relevant, but of course I might have missed some things... Help is appreciated! TIA, Jukka [1] http://salmi.ch/~jukka/dkim-milter/maillog [2] http://salmi.ch/~jukka/dkim-milter/miltersniff.39902-40249.gz [3] http://salmi.ch/~jukka/dkim-milter/notes.txt -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Jukka S. <j+d...@20...> - 2008-01-21 20:40:50
|
Jukka Salmi --> dkim-milter-discuss (2008-01-21 19:39:01 +0100): > Hi, > > SM --> dkim-milter-discuss (2008-01-21 08:19:51 -0800): > > I have a copy of the message. I don't think we have to pursue that > > angle as you have already tried it. > > could you please forward it to me off-list anyhow? TIA Thanks for the message, sm. As expected I could not trigger the failure by reinjecting it. > > The milter library you have is currently using select. Try using > > poll by defining > > > > APPENDDEF(`conf_libmilter_ENVDEF',`-DSM_CONF_POLL=1') > > > > in the site.config.m4 file. > > OK, I'll try this asap. Done; let's wait and see whether problem still exists... Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Jukka S. <j+d...@20...> - 2008-01-28 17:34:01
|
Jukka Salmi --> dkim-milter-discuss (2008-01-21 21:40:44 +0100): > Jukka Salmi --> dkim-milter-discuss (2008-01-21 19:39:01 +0100): > > Hi, > > > > SM --> dkim-milter-discuss (2008-01-21 08:19:51 -0800): [...] > > > The milter library you have is currently using select. Try using > > > poll by defining > > > > > > APPENDDEF(`conf_libmilter_ENVDEF',`-DSM_CONF_POLL=1') > > > > > > in the site.config.m4 file. > > > > OK, I'll try this asap. > > Done; let's wait and see whether problem still exists... It does: the problem just reoccurred using dkim-milter linked with a poll(2)ing libmilter... Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Jukka S. <j+d...@20...> - 2008-01-21 18:39:07
|
Hi, SM --> dkim-milter-discuss (2008-01-21 08:19:51 -0800): > I have a copy of the message. I don't think we have to pursue that > angle as you have already tried it. could you please forward it to me off-list anyhow? TIA > I don't think that would change the outcome. You could disable the > inclusion of the X-DKIM header. ...but then I'd possibly wouldn't notice the failure anymore ;-) > The milter library you have is currently using select. Try using > poll by defining > > APPENDDEF(`conf_libmilter_ENVDEF',`-DSM_CONF_POLL=1') > > in the site.config.m4 file. OK, I'll try this asap. Thanks, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: Tony E. <to...@he...> - 2008-01-21 11:54:20
|
Jukka Salmi skrev, on 21-01-2008 11:49:
>> What OS/distribution are you running again? I've been through your posts
>> in this thread again and can't see any mention.
>
> NetBSD/i386 3.1_STABLE (and Postfix 2.4.5, dkim-milter 2.4.0).
Ok, I have no practical experience of any BSD flavor; however, this does
isolate your OS from mine.
>> Apart from anything else, AFAICR, no client MTA or MUA has ever, ever
>> attempted to initiate a second connection for the same message while the
>> first connection has been open. I do read my main Postfix log and
>> pflogsumm output daily and have "always" done so. Likewise the system
>> bounce messages.
>
> Hmm, I did not say the remote MTA was trying to deliver the same message
> twice concurrently; I only noticed once that a remote MTA openened two
> concurrent connections and tried to deliver a message each, wich had
> identical Message-IDs.
Correct me if I'm wrong anybody, but a Message-ID is supposed to be
unique. See rfc2822 3.6.4. Identification fields:
The "Message-ID:" field contains a single unique message identifier.
The "References:" and "In-Reply-To:" field each contain one or more
unique message identifiers, optionally separated by CFWS.
The fact that the client is doing this (trying to send the same message
twice on two concurrent connections) I repeat I have never experienced.
>> Now that you're using an inet socket there's one less unknown; I'd like
>> to find out what causes the client to attempt a second connect while the
>> first one's still valid.
>
> IIRC that was a ML message from sf.net; I guess somebody just sent his
> message twice, and sf.net doesn't suppress duplicate messages.
If a client MUA or an MTA sends a message twice, both will have discrete
Message-IDs. Alternatively, either the client software is badly broken
or the initial connect isn't being reported. The likelihood is that
sf.net is using a reasonably recent version of Exim, which isn't known
for misbehaving like that.
Best,
--Tonni
--
Tony Earnshaw
Email: tonni at hetnet dot nl
|
|
From: Jukka S. <j+d...@20...> - 2008-01-21 12:25:40
|
Tony Earnshaw --> dkim-milter-discuss (2008-01-21 12:53:35 +0100): > Correct me if I'm wrong anybody, but a Message-ID is supposed to be > unique. See rfc2822 3.6.4. Identification fields: > > The "Message-ID:" field contains a single unique message identifier. > The "References:" and "In-Reply-To:" field each contain one or more > unique message identifiers, optionally separated by CFWS. That's correct, and two paragraphs later there's the following note: Note: There are many instances when messages are "changed", but those changes do not constitute a new instantiation of that message, and therefore the message would not get a new message identifier. For example, when messages are introduced into the transport system, they are often prepended with additional header fields such as trace fields (described in section 3.6.7) and resent fields (described in section 3.6.6). The addition of such header fields does not change the identity of the message and therefore the original "Message-ID:" field is retained. In all cases, it is the meaning that the sender of the message wishes to convey (i.e., whether this is the same message or a different message) that determines whether or not the "Message-ID:" field changes, not any particular syntactic difference that appears (or does not appear) in the message. And that's BTW exactly what e.g. mutt does when you bounce / resend a message: it retains the Message-ID and creates a new Resent-Message-ID. > If a client MUA or an MTA sends a message twice, both will have discrete > Message-IDs. Alternatively, either the client software is badly broken No. It's broken if it does change the Message-ID (see above). Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: SM <sm...@re...> - 2008-01-21 16:27:33
|
Hi Jukka, At 02:34 21-01-2008, Jukka Salmi wrote: >No, unfortunately the message has been deleted in the meantime. But I >tried this before, and could never trigger the failure. In fact, since >the message causing troubles can't be delivered, I can probably never >get hold of such a message ;-) I have a copy of the message. I don't think we have to pursue that angle as you have already tried it. >Hmm, maybe I should change dkim-milter to return SMFIS_ACCEPT instead >of SMFIS_TEMPFAIL in case dkimf_insheader(..., ..., XHEADERNAME, ...) >fails? I don't think that would change the outcome. You could disable the inclusion of the X-DKIM header. >I built [1]it using pkgsrc (2007Q3) which uses [2]this site.config.m4. >Nothing fancy AFAICT. The milter library you have is currently using select. Try using poll by defining APPENDDEF(`conf_libmilter_ENVDEF',`-DSM_CONF_POLL=1') in the site.config.m4 file. Regards, -sm |
|
From: Mike M. <mi...@ma...> - 2007-12-14 20:19:10
|
On Thu, Dec 13, 2007 at 12:24:00PM -0800, Murray S. Kucherawy <ms...@se...> wrote: > On Thu, 13 Dec 2007, Mark Martinec wrote: > > How about a general catch-all setting, so that instead of having to > > list each one in -C (including the new ones potentially introduced > > with later versions), one could specify only one. > > > > E.g. instead of "-C dns=a,int=a" > > one could have something like "-C default=a" > > (or equivalent in a configuration file). > > Sounds reasonable. Can you make it a feature request on SourceForge? Interesting timing, since we're running up against the 64-header-line limit in my employer's environment. Should that also be covered by the added -C option(s)? (I'd rather ask here first than kibitz on Mark's feature request) -- Mike Markley <mi...@ma...> You can get more with a kind word and a two-by-four than you can with a kind word. - Marcus Cole, "Ceremonies of Light & Dark" |
|
From: Murray S. K. <ms...@se...> - 2007-12-14 20:21:17
|
On Fri, 14 Dec 2007, Mike Markley wrote: > Interesting timing, since we're running up against the 64-header-line > limit in my employer's environment. Should that also be covered by the > added -C option(s)? (I'd rather ask here first than kibitz on Mark's > feature request) Actually I've opened your particular problem already as a separate bug report. The path to solving that problem is different (Mark's is in dkim-filter, yours is in libdkim). |
|
From: Jukka S. <j+d...@20...> - 2008-01-17 11:21:56
|
Jukka Salmi --> dkim-milter-discuss (2008-01-15 11:40:39 +0100): [...] > This is still with dkim-milter 2.4.0 and Postfix 2.4.5 on NetBSD/i386 > 3.1, BTW. > > So far the problem always happens more or less like this: > > - a remote MTA connects to my local MTA (mx1) > - another remote MTA connects to mx1 > - a third remote MTA connects to mx1 > - the message from the third MTA is rejected, dkim-milter logs > "X-DKIM" header add failed > and the MTA logs > can't read SMFIC_BODYEOB reply packet header: > Undefined error: 0 > - the messages from first and second MTA are rejected, the MTA logs > can't read SMFIC_BODYEOB reply packet header: > Operation timed out > - the remote systems either deliver the message to my backup MTA (mx2) > which forwards it to mx1, or they try again some time later to mx1 > which succeeds this time > > A complete MTA [1]log is available (I garbled some email adresses...), > as is a tcpdump [2]record of about the same period and some [3]notes > listing the MTA log <--> TCP session mappings. > > More context of the MTA log and tcpdump session record is available > on request. I put online what I thought is relevant, but of course I > might have missed some things... Yesterday I saw the problem again, but the order of events was slightly different: - A remote MTA (sf.net) connects to my MTA (mx1). - The same remote MTA (sf.net) opens a second connection to mx1 and tries to deliver a message with the same message id as in the first (still open) connection. - After 5 minutes (which is my Postfix milter_content_timeout setting) both connections are rejected: postfix/cleanup[6973]: warning: milter inet:localhost:1026: can't read SMFIC_BODYEOB reply packet header: Operation timed out postfix/cleanup[6973]: CDE4C24C7D: milter-reject: END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - [...] - Some minutes later another remote MTA (cloud9.net) connects to mx1. The message is rejected imediately: postfix/cleanup[14447]: warning: milter inet:localhost:1026: can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 postfix/cleanup[14447]: 39EA224C7B: milter-reject: END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - [...] dkim-milter[9162]: 39EA224C7B "X-DKIM" header add failed - The cloud9.net MTA connects to my backup MTA which runs exactly the same software versions (dkim-milter, Postfix, OS) as mx1, and delivers the message successfully. - After this there are two deliveries of DKIM signed messages to mx1, both of them successful. - Now the sf.net MTA connects again twice and tries to deliver the previously rejected messages. - Another MTA (soekris.com) connects, but its message is immediately rejected, as cloud9.net's message was (see above), and dkim-milter logs its usual "X-DKIM header add failed". - mx1 times out both connections from sf.net as before. - Some minutes later sf.net tries again, and this time succeeds. Two hours after Postfix times out a milter connection I see a TCP keep-alive packet from dkim-milter to the port Postfix used, which is answered with a TCP reset. I'm not sure how to interpret this, but I guess dkim-milter is still waiting for input... A slightly garbled MTA [4]log, a tcpdump [5]record and some [6]notes are available, and all of this with more context on request. Help is still appreciated! TIA, Jukka > [1] http://salmi.ch/~jukka/dkim-milter/maillog > [2] http://salmi.ch/~jukka/dkim-milter/miltersniff.39902-40249.gz > [3] http://salmi.ch/~jukka/dkim-milter/notes.txt [4] http://salmi.ch/~jukka/dkim-milter/maillog1 [5] http://salmi.ch/~jukka/dkim-milter/miltersniff.1.gz [6] http://salmi.ch/~jukka/dkim-milter/notes1.txt -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
|
From: SM <sm...@re...> - 2008-01-17 14:39:51
|
Hi Jukka, At 03:21 17-01-2008, Jukka Salmi wrote: >Yesterday I saw the problem again, but the order of events was slightly >different: > >- A remote MTA (sf.net) connects to my MTA (mx1). > >- The same remote MTA (sf.net) opens a second connection to mx1 and > tries to deliver a message with the same message id as in the first > (still open) connection. That shouldn't be a problem as the message-id is considered as a just a string by the milter. >- After 5 minutes (which is my Postfix milter_content_timeout setting) > both connections are rejected: > > postfix/cleanup[6973]: warning: milter inet:localhost:1026: > can't read SMFIC_BODYEOB reply packet header: > Operation timed out Would > postfix/cleanup[6973]: CDE4C24C7D: milter-reject: > END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - [...] > >- Some minutes later another remote MTA (cloud9.net) connects to mx1. > The message is rejected imediately: > > postfix/cleanup[14447]: warning: milter inet:localhost:1026: > can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 Can you enable logging of milter events? Regards, -sm |
|
From: Jukka S. <j+d...@20...> - 2008-01-18 17:30:12
|
Hi, SM --> dkim-milter-discuss (2008-01-17 06:38:25 -0800): > Hi Jukka, > At 03:21 17-01-2008, Jukka Salmi wrote: > >Yesterday I saw the problem again, but the order of events was slightly > >different: > > > >- A remote MTA (sf.net) connects to my MTA (mx1). > > > >- The same remote MTA (sf.net) opens a second connection to mx1 and > > tries to deliver a message with the same message id as in the first > > (still open) connection. > > That shouldn't be a problem as the message-id is considered as a just > a string by the milter. OK. > >- After 5 minutes (which is my Postfix milter_content_timeout setting) > > both connections are rejected: > > > > postfix/cleanup[6973]: warning: milter inet:localhost:1026: > > can't read SMFIC_BODYEOB reply packet header: > > Operation timed out > > Would ? > > postfix/cleanup[6973]: CDE4C24C7D: milter-reject: > > END-OF-MESSAGE from [...]: 4.7.1 Service unavailable - [...] > > > >- Some minutes later another remote MTA (cloud9.net) connects to mx1. > > The message is rejected imediately: > > > > postfix/cleanup[14447]: warning: milter inet:localhost:1026: > > can't read SMFIC_BODYEOB reply packet header: Undefined error: 0 > > Can you enable logging of milter events? Hmm, I'm trying to figure out how to do this with Postfix without specifying -v to smtpd (because this is probably a bit too verbose for this) and without recompiling (after defining msg_verbose in milter8.c)... Any hints? Ok, this is the wrong place for this question ;-) Regards, Jukka -- bashian roulette: $ ((RANDOM%6)) || rm -rf ~ |
Thanks for helping keep SourceForge clean.
X