From: Murray S. K. <ms...@se...> - 2006-04-19 18:56:42
|
I've opened this as an RFE against dkim-milter (#1473171), but I'd like to open up the possible solutions for discussion here. The issue: The asynchronous resolver's default timeout (set by the "-T" command line option) is ten seconds. However, this is also the default amount of time the MTA will wait for a reply from the filter. This means if you actually hit the full ten seconds, the MTA will time out waiting for a reply and give up even though the filter is about to send back a reply indicating it was giving up waiting for a DNS reply and requesting a temp-fail of the message. However, the MTA will always win this battle because the reply comes in just slightly more than ten seconds, by which point the MTA has already given up. The solutions: (1) Document the problem and explain that the simplest thing to do is increase the MTA's milter read timeout. This is configurable via the INPUT_MAIL_FILTER() m4 macro. (2) Document the problem and explain that the simplest thing to do is decrease the asynchronous resolver's timeout by running with a value of "-T" that's less than ten seconds. Even better, lower the default value as well. [In either case (1) or (2), document that the asynchronous resolver's timeout should always be less than the MTA's milter read timeout to avoid this problem.] (3) Modify dkim-filter's use of the asynchronous resolver such that it periodically sends a "still in progress" message back to the MTA using the smfi_progress() function and then goes back to waiting for the DNS reply up to the value of "-T", and otherwise leave the default as-is. (4) All three of these. (5) Something else I haven't thought of yet. Any feedback from the field? -MSK |
From: SM <sm...@re...> - 2006-04-21 05:56:09
|
Hi Murray, At 11:56 19-04-2006, Murray S. Kucherawy wrote: >The issue: The asynchronous resolver's default timeout (set by the >"-T" command line option) is ten seconds. However, this is also the >default amount of time the MTA will wait for a reply from the >filter. This means if you actually hit the full ten seconds, I have not hit that default timeout. >the MTA will time out waiting for a reply and give up even though >the filter is about to send back a reply indicating it was giving up >waiting for a DNS reply and requesting a temp-fail of the >message. However, the MTA will always win this battle because the >reply comes in just slightly more than ten seconds, by which point >the MTA has already given up. > >The solutions: >(1) Document the problem and explain that the simplest thing to do >is increase the MTA's milter read timeout. This is configurable via >the INPUT_MAIL_FILTER() m4 macro. The above explanation about timeouts should be in the documentation. >(2) Document the problem and explain that the simplest thing to do >is decrease the asynchronous resolver's timeout by running with a >value of "-T" that's less than ten seconds. Even better, lower the >default value as well. I suggest lowering the default value so that the MTA does not win the battle if the default timeout for the resolver is reached. >[In either case (1) or (2), document that the asynchronous >resolver's timeout should always be less than the MTA's milter read >timeout to avoid this problem.] Yes. >(3) Modify dkim-filter's use of the asynchronous resolver such that >it periodically sends a "still in progress" message back to the MTA >using the smfi_progress() function and then goes back to waiting for >the DNS reply up to the value of "-T", and otherwise leave the default as-is. The asynchronous resolver could called smfi_progress() every few seconds if the operation takes time as you explained. That would also work around the problem if the user increases the value of -T without increasing the milter read timeout. I gather that the above should also apply for dk-milter. Regards, -sm |
From: Murray S. K. <ms...@se...> - 2006-04-21 17:31:59
|
SM wrote: >> The issue: The asynchronous resolver's default timeout (set by the >> "-T" command line option) is ten seconds. However, this is also the >> default amount of time the MTA will wait for a reply from the filter. >> This means if you actually hit the full ten seconds, > > I have not hit that default timeout. It's not common. You'd have to have several unresponsive remote servers and no cached data, since retries are every five seconds and the timeout is ten. In my case it became visible because a packet filter was intercepting DNS replies, not a normal use case. >> (2) Document the problem and explain that the simplest thing to do is >> decrease the asynchronous resolver's timeout by running with a value >> of "-T" that's less than ten seconds. Even better, lower the default >> value as well. > > I suggest lowering the default value so that the MTA does not win the > battle if the default timeout for the resolver is reached. 10 seconds is already pretty short, since the retransmission timer default is five. This allows for only one retransmission and no margin for latency. > I gather that the above should also apply for dk-milter. And sid-milter, yes. |
From: Murray S. K. <ms...@se...> - 2006-04-21 18:08:36
|
I've added documentation now to all three packages, so that's a start. Since sid-milter maintains control over the DNS operations, it can easily be modified to send progress messages back to the MTA via smfi_progress(). However, arranging to do smfi_progress() calls is tricky in the DK and DKIM packages because the DNS queries are managed by the libraries and not the filters themselves. Those libraries are nicely MTA-agnostic right now and I don't really want to break that by making them milter-aware. Perhaps a callback of some kind... |
From: SM <sm...@re...> - 2006-04-21 19:52:17
|
Hi Murray, At 11:08 21-04-2006, Murray S. Kucherawy wrote: >Since sid-milter maintains control over the DNS operations, it can >easily be modified to send progress messages back to the MTA via >smfi_progress(). > >However, arranging to do smfi_progress() calls is tricky in the DK >and DKIM packages because the DNS queries are managed by the >libraries and not the filters themselves. Those libraries are >nicely MTA-agnostic right now and I don't really want to break that >by making them milter-aware. I forgot that these queries were done from the libraries. >Perhaps a callback of some kind... It could be done in a callback. I suggest keeping the libraries MTA-agnostic. I forgot that smfi_progress() is a FFR in sendmail 8.12. You might as well increase the timeout setting in the INPUT_MAIL_FILTER() in dkim-filter/Readme. Regards, -sm |
From: Murray S. K. <ms...@se...> - 2006-04-27 20:57:10
|
SM wrote: >> Perhaps a callback of some kind... > > It could be done in a callback. I suggest keeping the libraries > MTA-agnostic. I forgot that smfi_progress() is a FFR in sendmail 8.12. > You might as well increase the timeout setting in the > INPUT_MAIL_FILTER() in dkim-filter/Readme. I've added an experimental callback facility in libdkim for the next release, and dkim-filter will use it. |