fetchmail-users

[Fetchmail-users] Working around "Bad sender address syntax" error?

[Daniel B. <dba...@bl...>]

I use fetchmail to retrieve email from an IMAP server every five
minutes, passing it to a local postfix process on my home Linux box
for delivery. The problem: sometimes an email has an invalid "From"
line (usually spam), like this one with leading hyphens:

  From: "- Some Dumb Spammer" <-Du...@ex...>

In this case, when fetchmail retrieves the email, the local postfix
process raises an error:

  fetchmail: SMTP error: 501 5.1.7 Bad sender address syntax

This error causes fetchmail to leave the bad email sitting on the IMAP
server, even when my .fetchmailrc file specifies "nokeep". As a
result, fetchmail re-downloads it every five minutes. Repeat
forever... or until I manually delete the bad email from the IMAP
server.

What's the best way to break these loops automatically, either
deleting or delivering the bad email, without opening a security hole
(e.g., permitting leading hyphens)?

Thank you very much.

--
Dan Barrett
dba...@bl...

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Rob M. <rob...@gm...>]

On 9 September 2014 14:17, Daniel Barrett <dba...@bl...> wrote:
>
> I use fetchmail to retrieve email from an IMAP server every five
> minutes, passing it to a local postfix process on my home Linux box
> for delivery. The problem: sometimes an email has an invalid "From"
> line (usually spam), like this one with leading hyphens:
>
>   From: "- Some Dumb Spammer" <-Du...@ex...>
>
> In this case, when fetchmail retrieves the email, the local postfix
> process raises an error:
>
>   fetchmail: SMTP error: 501 5.1.7 Bad sender address syntax
>
> This error causes fetchmail to leave the bad email sitting on the IMAP
> server, even when my .fetchmailrc file specifies "nokeep". As a
> result, fetchmail re-downloads it every five minutes. Repeat
> forever... or until I manually delete the bad email from the IMAP
> server.
>
> What's the best way to break these loops automatically, either
> deleting or delivering the bad email, without opening a security hole
> (e.g., permitting leading hyphens)?

Check the anti-spam option:

http://www.fetchmail.info/fetchmail-man.html#29

If you set 501 as a spam code fetchmail will delete any email your
Postfix server returns that code for.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Matthias A. <mat...@gm...>]

Am 09.09.2014 um 15:17 schrieb Daniel Barrett:
> 
> I use fetchmail to retrieve email from an IMAP server every five
> minutes, passing it to a local postfix process on my home Linux box
> for delivery. The problem: sometimes an email has an invalid "From"
> line (usually spam), like this one with leading hyphens:
> 
>   From: "- Some Dumb Spammer" <-Du...@ex...>
> 
> In this case, when fetchmail retrieves the email, the local postfix
> process raises an error:
> 
>   fetchmail: SMTP error: 501 5.1.7 Bad sender address syntax
> 
> This error causes fetchmail to leave the bad email sitting on the IMAP
> server, even when my .fetchmailrc file specifies "nokeep". As a
> result, fetchmail re-downloads it every five minutes. Repeat
> forever... or until I manually delete the bad email from the IMAP
> server.

Daniel,

Older fetchmail versions promised to never delete mail from the server
that was not reported as delivered by its SMTP/LMTP server or MDA, and
since v6.3.10, fetchmail lives up to this promise.

> What's the best way to break these loops automatically, either
> deleting or delivering the bad email, without opening a security hole
> (e.g., permitting leading hyphens)?

I see at least three options:

1. Leading hyphens are only a security risk if you are using scripts
that would feed them to utilities that do not understand "--" to mark
the end of options, or without these "--".  If you're sure you're safe,
you can permit them.

2. If you prefer to delete permanently undeliverable messages (those
that elicit error codes 500 and above), use --nosoftbounce on the
command line, or "set no softbounce" in the rcfile.  Details in the
manual page.

3. If you prefer to delete messages with particular error codes, see the
manual page about the antispam option, see Rob MacGregor's message for a
reference.

Best,
Matthias

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Daniel B. <dba...@bl...>]

Thanks to Matthias and Rob for your suggestions to use the
nosoftbounce and antispam features!

Looks like the antispam feature can set "501" as a spam code (-Z 501),
but it does not accept the full "501 5.1.7" code reported in my error
message. Is this correct, and do you think this would be a worthwhile
enhancement request for fetchmail?

FYI, I asked my original question on StackOverflow last week and got
no responses. Feel free to add your answers and win some reputation
points! :-)

  http://stackoverflow.com/questions/25705002/postfix-bad-sender-address-syntax-causes-fetchmail-to-re-fetch-the-same-email

--
Dan Barrett
dba...@bl...

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Matthias A. <mat...@gm...>]

Am 10.09.2014 um 17:53 schrieb Daniel Barrett:
> 
> Thanks to Matthias and Rob for your suggestions to use the
> nosoftbounce and antispam features!
> 
> Looks like the antispam feature can set "501" as a spam code (-Z 501),
> but it does not accept the full "501 5.1.7" code reported in my error
> message. Is this correct, and do you think this would be a worthwhile
> enhancement request for fetchmail?

Thank you for the suggestion.  For now fetchmail only looks at the
actual SMTP/LMTP response code, rather than the RFC3463 extended status
code, and I wonder how broad we would need to make the RFC3463 stuff to
make it useful, rather than create user confusion because only some
parts of fetchmail might be looking at it.

If you have a use case where you need the 5.1.7 code or similar ones to
make a distinction beyond just the "501" in your case (or more
generally), then please file the use case along with the feature request
and include a reference to RFC3463.

> FYI, I asked my original question on StackOverflow last week and got
> no responses. Feel free to add your answers and win some reputation
> points! :-)

Certainly not.  While Stackoverflow is one of the forums that is getting
near the brink of being actually useful and halfway concise, I do not
believe in dispersing such knowledge outside the official and publicly
archived support channels, and SO is certainly way too late to the game.
By how many years does fetchmail predate it?

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[J. R. <jo...@an...>]

On Tuesday, September 09, 2014 10:59:51 PM Matthias Andree wrote:
> Am 09.09.2014 um 15:17 schrieb Daniel Barrett:
> > I use fetchmail to retrieve email from an IMAP server every five
> > minutes, passing it to a local postfix process on my home Linux box
> > for delivery. The problem: sometimes an email has an invalid "From"
> > 
> > line (usually spam), like this one with leading hyphens:
> >   From: "- Some Dumb Spammer" <-Du...@ex...>
> > 
> > In this case, when fetchmail retrieves the email, the local postfix
> > 
> > process raises an error:
> >   fetchmail: SMTP error: 501 5.1.7 Bad sender address syntax
> > 
> > This error causes fetchmail to leave the bad email sitting on the IMAP
> > server, even when my .fetchmailrc file specifies "nokeep". As a
> > result, fetchmail re-downloads it every five minutes. Repeat
> > forever... or until I manually delete the bad email from the IMAP
> > server.
> 
> Daniel,
> 
> Older fetchmail versions promised to never delete mail from the server
> that was not reported as delivered by its SMTP/LMTP server or MDA, and
> since v6.3.10, fetchmail lives up to this promise.
> 
> > What's the best way to break these loops automatically, either
> > deleting or delivering the bad email, without opening a security hole
> > (e.g., permitting leading hyphens)?
> 
> I see at least three options:
> 
> 1. Leading hyphens are only a security risk if you are using scripts
> that would feed them to utilities that do not understand "--" to mark
> the end of options, or without these "--".  If you're sure you're safe,
> you can permit them.
> 
> 2. If you prefer to delete permanently undeliverable messages (those
> that elicit error codes 500 and above), use --nosoftbounce on the
> command line, or "set no softbounce" in the rcfile.  Details in the
> manual page.
> 
> 3. If you prefer to delete messages with particular error codes, see the
> manual page about the antispam option, see Rob MacGregor's message for a
> reference.

I would prefer to see a 4th option:
Save the email as a TXT file in a configurable location to allow us to at 
least confirm it is spam and not a misconfigured email client. (I've 
encountered that particular situation not too long ago)

--
Joost

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[J. R. <jo...@an...>]

On Wednesday, September 10, 2014 08:04:04 AM Matthias Andree wrote:
> Am 10.09.2014 um 07:55 schrieb J. Roeleveld:
> > I would prefer to see a 4th option:
> > Save the email as a TXT file in a configurable location to allow us to at
> > least confirm it is spam and not a misconfigured email client. (I've
> > encountered that particular situation not too long ago)
> 
> Thanks for the feedback.
> 
> I do not think I will go for such features.
> 
> I want fetchmail to behave in a consistent and easily determined way,
> rather than along many ifs and buts, meaning in particular I want it to
> only ever use the one specific delivery path that is configured.
> 
> Another instance, disabling the "fallback" from SMTP to MDA was
> something that happened at my insisting already before I became a
> maintainer because it's just hard to track and tends to trigger false
> reports about misfiled/lost mail.
> 
> Feel free to file a feature request on the sourceforge.net project page
> (you need to be a registered user so I can ask questions and avoid
> hit-and-run reports) so this does not get lost, but it is very low
> priority for me and unlikely to happen in the next year.

Thank you for your reply. I will put this as a feature request.

I have not looked into the code yet, but if I were to provide a patch making 
this possible, would you be willing to work with me to get it included?

One other option I was thinking off would be to move the entire email into the 
body of a "new" email and add headers to have it delivered to the user as 
such. That should avoid any security issues.

If that is more acceptable, then please let me know and I will try to 
implement it like that.

Kind regards,

Joost

PS. Expect the feature request later today (if I can get a user account 
sorted)

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Matthias A. <mat...@gm...>]

Am 10.09.2014 um 08:22 schrieb J. Roeleveld:

> I have not looked into the code yet, but if I were to provide a patch making 
> this possible, would you be willing to work with me to get it included?
> 
> One other option I was thinking off would be to move the entire email into the 
> body of a "new" email and add headers to have it delivered to the user as 
> such. That should avoid any security issues.

Hi Joost,

The second proposal seems OK with me, but I would not enable it by
default.  I am planning to switch off the softbounce feature, so that
fetchmail would delete such nondeliverable mail by default, in fetchmail 7.

We can discuss integration, code, and thereabouts on the -devel@ mailing
list, and I will not add features to the legacy_63 branch - please make
sure you work against the master branch, which is currently a
7.0.0-alpha version.

Best,
Matthias

Re: [Fetchmail-users] Working around "Bad sender address syntax" error?

[Matthias A. <mat...@gm...>]

Am 10.09.2014 um 07:55 schrieb J. Roeleveld:

> I would prefer to see a 4th option:
> Save the email as a TXT file in a configurable location to allow us to at 
> least confirm it is spam and not a misconfigured email client. (I've 
> encountered that particular situation not too long ago)

Thanks for the feedback.

I do not think I will go for such features.

I want fetchmail to behave in a consistent and easily determined way,
rather than along many ifs and buts, meaning in particular I want it to
only ever use the one specific delivery path that is configured.

Another instance, disabling the "fallback" from SMTP to MDA was
something that happened at my insisting already before I became a
maintainer because it's just hard to track and tends to trigger false
reports about misfiled/lost mail.

Feel free to file a feature request on the sourceforge.net project page
(you need to be a registered user so I can ask questions and avoid
hit-and-run reports) so this does not get lost, but it is very low
priority for me and unlikely to happen in the next year.