Re: [fetchmail-users] Messages not downloaded

[Matthias A. <mat...@gm...>]

So, some more version-agnostic answers now that I've more time at my hands,
note that my answers will still assume you're using version 6.3.8 of fetchmail.

David Nash schrieb:
> I logged on to our domain's webmail service, and noticed that our multidrop inbox had a few thousand messages in it.  I don't think this used to happen, but they've started to collect there.  I cleaned them out, and we started to receive external mail instantaneously (obviously fetchmail was taking time to parse the thousands of messages).

Likely, to be confirmed or denied with "fetchmail -v" or perhaps "fetchmail
-vv".

> When new mail hits the multi-drop mailbox, its status is 'unread'.  When fetchmail downloads from the multidrop, the mail is removed from the multidrop mailbox, with the exception of some messages where the status changes to 'read' but it is not downloaded.  These are the ones that are building up.  Some of the recipients are genuine users on our system, but many are not.  From what I've been able to monitor, all these messages are SPAM.  I've looked at the headers, and they seem to be pretty much the same as genuine messages, but they have X-IMAIL-SPAM-xxx lines in their header, eg X-IMAIL-SPAM-VALFROM.  

> After all that explanation, I just wondered if fetchmail is looking at these header lines and choosing not to download the messages?  

No, it does not. If it did, you'd see that documented as a feature.
However, one of those spam messages may confuse your older version of
fetchmail, or perhaps (though less likely as I see it) your MDA (note I
wrote "may", not "does", I don't know trestlemail and this is not a
statement about trestlemail as such).

> I'm probably going to look at the 'flush' parameter to automatically delete them, once I've monitored it for a wee while to make sure that every message affected is genuinely SPAM.  I don't really want to change fetchmail versions if I can help it!  

It's a dangerous option to use, I'd rather run fetchmail once with that
option on the command line.

You really should update, not only fetchmail, but also your kernel, and
probably HEAPS of your system libraries and basic utilities. Your system is
way outdated and full of security-relevant bugs, and unless your fetchmail
version has been patched against known security holes, it's prone to code
injection.

I'd not be surprised at all if one of the bugs caused the messages to pile
up, by terminating fetchmail prematurely.

In a separate response to Frédéric I describe some workarounds to contain
the damage a bit - consider them regardless of if you're updating.

> Output of fetchmail -V is:
>  
> This is fetchmail release 5.8.17+SSL+NLS
> Linux scoms1.xxxxx.co.uk 2.4.13 #1 Thu Oct 31 02:32:23 EST 2002 i686 unknown
> Taking options from command line and /.fetchmailrc
> Poll interval is 300 seconds
> Idfile is //.fetchids
> Fetchmail will forward misaddressed multidrop messages to postmaster.
> Options for retrieving from xx...@xx...@xxx.net:
>   True name of server is xxx.net.
>   Protocol is POP3.
>   All available authentication methods will be tried.
>   Server nonresponse timeout is 300 seconds (default).
>   Default mailbox selected.
>   All messages will be retrieved (--all on).
>   Fetched messages will not be kept on the server (--keep off).
>   Old messages will not be flushed before message retrieval (--flush off).
>   Rewrite of server-local addresses is enabled (--norewrite off).
>   Carriage-return stripping is enabled (stripcr on).
>   Carriage-return forcing is disabled (forcecr off).
>   Interpretation of Content-Transfer-Encoding is enabled (pass8bits off).
>   MIME decoding is disabled (mimedecode off).
>   Idle after poll is disabled (idle off).
>   Nonempty Status lines will be kept (dropstatus off)
>   Delivered-To lines will be kept (dropdelivered off)
>   Messages will be delivered with "~/trestlemail/bin/trestlemail ~/trestlemail/trestlemail.rc -F %F -- %T".
>   Recognized listener spam block responses are: 571 550 501 554
>   Multi-drop mode: 1 local name(s) recognized.
>   DNS lookup for multidrop addresses is enabled.
>   Server aliases will be compared with multidrop addresses by name.
>   Envelope header is assumed to be: Received
>   Local domains: xxxx.co.uk
>   No UIDs saved from this host.

A general remark: DNS lookup for multidrop is a resource-consuming option
to use and based on the - usually false today - assumption that POP3 and
SMTP MX servers for the given domain are identical. This holds for none but
the smallest sites that run all services off the same computer. If there is
more than the polling domain, list it with "aka", then use the "no dns" option.

Hope that clarifies things a bit.

-- Matthias