[fetchmail-devel] Re: [fetchmail]Observations of a new user

[R H. B. <arg...@ya...>]

Matthias,

This is a follow-up post to my earlier thread on the Fetchmail-Friends list.

You replied to my original email:
> To: fet...@li...
> Subject: Re: [fetchmail]Observations of a new user
> From: Matthias Andree <ma...@dt...>
> Date: Mon, 08 Nov 2004 12:09:11 +0100
>
> R Hannes Beinert <arg...@ya...> writes:
>
>>  2) The documentation for FM is excellent, and sets a high bar for other
>>     OSS projects!  One small detail caused me to trip, initially, although
>>     (he says apologetically) this may have had more to do with the reader
>>     than the writer.  I found it might've helped me initially if there had
>>     been an explicit discussion of the "singledrop" versus "multidrop" mode
>>     of operation early in the man page.  Then, it might additionally have
>>     helped if each option had been explicitly labeled as having
>>     applicability for single- vs multidrop. [...]
>
> Well, the single- vs. multidrop issue is difficult; fetchmail implicitly
> derives this information from the "is ... here" stuff, and that is
> pretty unclear from the documentation.
>
> If you can suggest better wordings, perhaps even a patch, that would be
> most welcome - we have a fetchmail-devel list for such things, see
> http://developer.berlios.de/projects/fetchmail/

I have attached a patch on trunk/fetchmail.man (rev 3876).  Hopefully I
have managed to put it into a form which is useful to you.  I also hope
I have not made any technical errors, however I am by no means a
Fetchmail guru.

There are two basic changes I'm suggesting.  First, an explanation of
Fetchmail's modes of operation under the man-page DESCRIPTION heading.
Second, I've added some annotations to various options when the option
only applies to multidrop.  I've also added an additional column to the
keyword/option summary with this information, as well (I hope I was
able to find all the relevant instances).

>>  3) On the subject of logging - I find myself missing MTA-style logging, ie,
>>     timestamp, mailserver, MAIL FROM, RCPT TO, message size, and receiving
>>     MTA queueid.  This would make tracking email through the various logs
>>     much easier.  The standard FM log is too terse to include this
>>     information, yet while the "single-v" logging contains the information
>>     in question, it is probably more intended for debugging than logging of
>>     the sort I'm looking for.  Also, in the non-v logging, I miss having
>>     information on when a server with no messages was polled.  I do realize,
>>     though, that the non-v logging was intended to be very economical.  I've
>>     been using -v logging, and I'm happy.  Disks are cheap.  ;-)
>
> If you can offer a scheme what information exactly should be printed,
> that would help. MTA-style logging might be useful for fetchmail indeed.

I'm sure there are many details I've neglected which might be helpful,
but here is a sketch of what I have in mind.  All messages should be
prefixed by a time/date stamp which can be easily sorted.

At daemon start:

04/12/08@04:03:00[pid]: Starting fetchmail 6.2.3 daemon (/etc/fetchmailrc)
04/12/08@04:03:01[pid]: Postmaster=pos...@ho...m; Poll-Interval=1234 secs

When a poll starts.  Should only specify information for one of single
or multidrop modes.  Message line specifies the number of messages and
the size in bytes/octets:

04/12/08@04:03:10[pid]: Poll started; US...@ma... [1.2.3.4] (POP3); Interval=1
04/12/08@04:03:11[pid]: Multidrop; Fetchdomains DOM1, DOM2, DOM3, ...
04/12/08@04:03:12[pid]: Multidrop; Users <USER1>, <USER2>, <USER3>, ...
04/12/08@04:03:13[pid]: Singledrop; Recipient=<me...@ho...m>
04/12/08@04:03:14[pid]: Messages new 254 (1780000) / old 40 (9589) / total 294 (1789589)

If a poll specification includes an INTERVAL keyword which calls
for the server to be skipped on the current cycle:

04/12/08@04:03:10[pid]: Poll skipped; US...@ma... [1.2.3.4] (POP3); Interval=2

If a server poll is attempted, yet unsuccessful:

04/12/08@04:03:20[pid]: Poll started; US...@ma... [1.2.3.4] (POP3)
04/12/08@04:03:21[pid]: Server unreachable (specific error message)
04/12/08@04:03:22[pid]: Poll completed; US...@ma... [1.2.3.4] (POP3)

For each retrieved message.  "Resolved" refers to the envelope recipient which
has been derived by Fetchmail.  The parenthetical number is the sequence number
of the message in the mail account (or any other quasi-unique identifier).  The
Delivered/Rejected line comes from the reply code to the DATA & <CRLF>.<CRLF>
sequence (RFC2821 sec 4.2.5).  A positive completion status often contains the
queue-ID for the SMTP server, and could be included in the parentheses.  It is
my hope that this information is sufficient to track a particular message from
the retrieval server, through Fetchmail, and into the SMTP server:

04/12/08@04:03:30[pid]: (001) Message-Id=<612...@dr...>
04/12/08@04:03:31[pid]: (001) From=<ad...@dr...>; Size=38830
04/12/08@04:03:32[pid]: (001) To=<ad...@fo...>; Resolved=<admin@localhost>
04/12/08@04:03:33[pid]: (001) Delivered; SMTP.RELAYHOST.DOM:PORT (250 Ok: queued as 1B8D615F8F)
04/12/08@04:03:34[pid]: (001) Rejected; SMTP.RELAYHOST.DOM:PORT (540 Service unavailable)

If Fetchmail needs to break a server session due to a resource limit:

04/12/08@04:03:40[pid]: Poll fetchlimit reached (100); US...@ma... [1.2.3.4] (POP3)
04/12/08@04:03:41[pid]: Poll restarted; US...@ma... [1.2.3.4] (POP3)
04/12/08@04:03:42[pid]: Messages new 254 (1780000) / old 40 (9589) / total 294 (1789589)

Finally, when the daemon terminates a polling cycle:

04/12/08@04:03:50[pid]: Poll completed; US...@ma... [1.2.3.4] (POP3)
04/12/08@04:03:51[pid]: Sleeping until 04/12/08@05:03:40

>>  4) Closely releted to item #3, I also found it might be useful to be able
>>     to control logging from the fetchmailrc file, so that system start-up
>>     scripts wouldn't need to be touched.  So, eg, one could conceivably
>>     use global options such as "set verbose 2", or "set logging-level 1"
>>     in one's FMrc configuration to change the logging verbosity.
>
> Hm. That should be doable for the release after the next. The next one
> is for bugfixes, since we've had it pending for over a year now.
>
> Again, detailed suggestions on fetchmail-devel (see above) will be
> appreciated. Patches even more so. :)

I wish I could oblige with some patches to the code, but I don't have the
time to familiarize myself with the codebase at the moment.  In the meantime
I hope this email might be a little helpful.

Many grateful regards,

Hannes.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com