fetchmail-users — User questions and discussion

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Hans C. <fo...@gm...>]

On Mon, 1 Oct 2018, Matthias Andree wrote:

> I am somewhat inclined to take a variant of your (Hans's) patch and make
> it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity of
> 2:05 A and 2:05 B in the night when the DST is switched off), and that's
> it.  The alternative would be a plain timestamp (seconds since UTC Epoch
> or something)

Would it be unreasonable to include both ISO8601 and time_t? eg.

   2018-10-01T22:50:44+00:00 (1538434244)

The time_t would simply be easier for an external script to parse.

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Monday 01 October 2018 14:00:03 Matthias Andree wrote:

> Am 01.10.18 um 19:19 schrieb Hans Carlson:
> > Someone a bit more enterprising than me could probably submit a
> > patch with some kind of config file option (eg. log_timestamp) to do
> > something similar.  It would be wise to also add a config file
> > option that specified the format for the timestamp, with the default
> > timestamp something more universally acceptable and standard than
> > the format I used.
>
> Hi Gene, Hans, everybody else,
>
> I'd be bold (as the maintainer) and share the insight that adding
> options is an expensive process. It requires options parsing, twice
> (command line, and rcfile), option dumping, at that, documentation,
> testing, string translations, future compatibility considerations -
> for a few lines of code around strftime() and gmtime() calls we'd
> easily add 100 lines over the place for making things configurable.
>
> I am somewhat inclined to take a variant of your (Hans's) patch and
> make it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity
> of 2:05 A and 2:05 B in the night when the DST is switched off), and
> that's it.

I'd not have a problem if it was in UTC.

The manpage should clearly reflect that however.

> The alternative would be a plain timestamp (seconds since 
> UTC Epoch or something) that would require further formatting through
> an external program---but then again a program that translated ISO8601
> UTC timestamps to an arbitrary local time zone wouldn't have to be
> much more than a few lines of Perl.
>
> The consideration is whether this could go into 6.4, or would have to
> wait for 7.0. The log file format is not really specified, so changing
> it isn't breaking any documented interface, just breaking past
> practice, so it might cause some astonishment. Opinions?
>
> Best regards,
> Matthias
>
>
>
>
> _______________________________________________
> Fetchmail-users mailing list
> Fet...@li...
> https://lists.sourceforge.net/lists/listinfo/fetchmail-users



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[<kj...@o2...>]

Matthias Andree <mat...@gm...> writes:

[..]
>
> The consideration is whether this could go into 6.4, or would have to
> wait for 7.0. The log file format is not really specified, so changing
> it isn't breaking any documented interface, just breaking past practice,
> so it might cause some astonishment. Opinions?

Regardless of these questions I would as NOT to change logging via
syslog.
KJ

-- 
http://wolnelektury.pl/wesprzyj/teraz/
Do not remove tag under penalty of law.

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Matthias A. <mat...@gm...>]

Am 01.10.18 um 19:19 schrieb Hans Carlson:
> Someone a bit more enterprising than me could probably submit a patch
> with some kind of config file option (eg. log_timestamp) to do
> something similar.  It would be wise to also add a config file option
> that specified the format for the timestamp, with the default
> timestamp something more universally acceptable and standard than the
> format I used.


Hi Gene, Hans, everybody else,

I'd be bold (as the maintainer) and share the insight that adding
options is an expensive process. It requires options parsing, twice
(command line, and rcfile), option dumping, at that, documentation,
testing, string translations, future compatibility considerations - for
a few lines of code around strftime() and gmtime() calls we'd easily add
100 lines over the place for making things configurable.

I am somewhat inclined to take a variant of your (Hans's) patch and make
it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity of
2:05 A and 2:05 B in the night when the DST is switched off), and that's
it.  The alternative would be a plain timestamp (seconds since UTC Epoch
or something) that would require further formatting through an external
program---but then again a program that translated ISO8601 UTC
timestamps to an arbitrary local time zone wouldn't have to be much more
than a few lines of Perl.

The consideration is whether this could go into 6.4, or would have to
wait for 7.0. The log file format is not really specified, so changing
it isn't breaking any documented interface, just breaking past practice,
so it might cause some astonishment. Opinions?

Best regards,
Matthias

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Hans C. <fo...@gm...>]

On Sun, 30 Sep 2018, grarpamp wrote:

>> This makes the log entries look like this:
>>
>>    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
>>    fetchmail [2018/09/29 17:04:42]: reading message ...
>>
>> Here's the patch I used for 6.3.26...
>
> Try reading and adopting a valid iso8601 standard format for timestamps 
> instead of the above which are nonconformant and ambiguous in multiple 
> ways.

Feel free to change the format however you like.  This patch was a "quick
fix" for something I found annoying with the fetchmail logs... nothing
more than that.  I saw someone else with a similar need and provided an 
option.

I understand your point and it's a very good one, but I intentionally 
selected that format because it's what I want.  It gives me exactly the 
information I need and no more.  It is not ambiguous (to ME, for MY 
purposes) in any way.

Different people have different needs... so I'll amend my original 
suggestion a bit...

Someone a bit more enterprising than me could probably submit a patch with 
some kind of config file option (eg. log_timestamp) to do something 
similar.  It would be wise to also add a config file option that specified 
the format for the timestamp, with the default timestamp something more 
universally acceptable and standard than the format I used.

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Matthias A. <mat...@gm...>]

Am 28.09.18 um 18:27 schrieb Gene Heskett:
> Greetings, long time fetchmail user here;
>
> I was subjected to a phishing attack yesterday and again today, and it 
> would have been a lot easier to correlate the messages if fetchmail was 
> logging the time, preferably the date/time at the end of fetching of 
> fetching each mail. Something it is not now, and never has done.  And I 
> just reread the man pages without finding a clue.
>
> How do I enable this so a date/time exists in the fetchmail.log?
>
Does it have to be fetchmail logging by itself, or have you considered
logging through syslog? Would the latter be an option? Syslog
implementations usually timestamp their records.

Re: [Fetchmail-users] how to log date/time in daemon mode?

[grarpamp <gra...@gm...>]

> And whats wrong with the above format?

For that, refer to the International Standard 8601:2004
which you can find pdf from any search.
A number of rationale are also included in it.
Also note,
- spaces are forbidden separator, format is a single field string
- slashes have special purpose interval, and are not a separator
- lower case is forbidden where upper case available
- etc, see the standard, ignore such at peril of rockets crash

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Ralph C. <ra...@in...>]

Hi Gene,

> > > > >    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> > >
> > > And whats wrong with the above format? And since I know thats my
> > > machines local time in a valid 24 hour format, I do not see any
> > > ambiguity.
>
> And that is essentially the same with an added -0400 which is the
> local EDT time zone offset from GMT.

It is still ambiguous.  :-)

    $ TZ=America/New_York date -d 2018-11-04T05:30:00Z +'%Y/%m/%d %H:%M:%S'
    2018/11/04 01:30:00
    $ TZ=America/New_York date -d 2018-11-04T06:30:00Z +'%Y/%m/%d %H:%M:%S'
    2018/11/04 01:30:00
    $

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Sunday 30 September 2018 06:53:53 Peter Pentchev wrote:

> On Sun, Sep 30, 2018 at 04:51:51AM -0400, Gene Heskett wrote:
> > On Sunday 30 September 2018 00:58:21 grarpamp wrote:
> > > > This makes the log entries look like this:
> > > >
> > > >    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> > > >    fetchmail [2018/09/29 17:04:42]: reading message ...
> > > >
> > > > Here's the patch I used for 6.3.26...
> > >
> > > Try reading and adopting a valid iso8601 standard format for
> > > timestamps instead of the above which are nonconformant
> > > and ambiguous in multiple ways.
> >
> > And whats wrong with the above format? And since I know thats my
> > machines local time in a valid 24 hour format, I do not see any
> > ambiguity.
>
> Problems will arise once you try to coordinate logs from different
> machines or send these logs to somebody else who has to match them
> against logs from their mailserver which may be in a different
> timezone - and, unless you tell them what your timezone is, there is
> no way they know what time your 17:04:41 is on their side.  I have to
> deal with something like this with syslog files from machines in
> different timezones almost every week :(  (and yes, most of the time
> I know which timezone which log comes from, but it's still much harder
> than it needs to be to combine and process them automatically)
>
> Another way this is ambiguous is made a lot better by the use of
> four digits for the year, but the fact still remains that there are
> some areas in the world which actually use a YYYY/DD/MM format, just
> as there are still some (very, very weird from my point of view, but
> you get the point) areas in the world which still use a MM/DD format
> :) The four-digit year removes the MM/DD/YY vs DD/MM/YY vs YY/MM/DD
> ambiguity (which I've had to deal with in the past and it was awful),
> but, very slim as the chances are for a YYYY/DD/MM vs YYYY/MM/DD one,
> it is still better to use a well-known (and recognizable) format to
> avoid *any* ambiguities.
>
> So, yes, I accept the criticism, since in my code snippets I did not
> use an ISO 8601 format either; if you really want your logs to be
> easy to use for matching against other logs, possibly from other
> servers, something like a `date --iso-8601=seconds` (with GNU date) or
> passing %Y%m%dT%H:%M:%S%z as a format string to a strftime-like
> formatting function would be much preferable.
>
> G'luck,
> Peter

And that is essentially the same with an added -0400 which is the local 
EDT time zone offset from GMT.  But since the only person interested in 
my fetchmail.log is me, it definitely comes under the category of "good 
enough for the girls I go with." ;-)

Patches welcome however.

Take care Peter, and thanks for the explanation.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Peter P. <ro...@ri...>]

On Sun, Sep 30, 2018 at 04:51:51AM -0400, Gene Heskett wrote:
> On Sunday 30 September 2018 00:58:21 grarpamp wrote:
> 
> > > This makes the log entries look like this:
> > >
> > >    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> > >    fetchmail [2018/09/29 17:04:42]: reading message ...
> > >
> > > Here's the patch I used for 6.3.26...
> >
> > Try reading and adopting a valid iso8601 standard format for
> > timestamps instead of the above which are nonconformant
> > and ambiguous in multiple ways.
> 
> And whats wrong with the above format? And since I know thats my machines 
> local time in a valid 24 hour format, I do not see any ambiguity. 

Problems will arise once you try to coordinate logs from different
machines or send these logs to somebody else who has to match them
against logs from their mailserver which may be in a different
timezone - and, unless you tell them what your timezone is, there is
no way they know what time your 17:04:41 is on their side.  I have to
deal with something like this with syslog files from machines in
different timezones almost every week :(  (and yes, most of the time
I know which timezone which log comes from, but it's still much harder
than it needs to be to combine and process them automatically)

Another way this is ambiguous is made a lot better by the use of
four digits for the year, but the fact still remains that there are
some areas in the world which actually use a YYYY/DD/MM format, just
as there are still some (very, very weird from my point of view, but
you get the point) areas in the world which still use a MM/DD format :)
The four-digit year removes the MM/DD/YY vs DD/MM/YY vs YY/MM/DD
ambiguity (which I've had to deal with in the past and it was awful),
but, very slim as the chances are for a YYYY/DD/MM vs YYYY/MM/DD one,
it is still better to use a well-known (and recognizable) format to
avoid *any* ambiguities.

So, yes, I accept the criticism, since in my code snippets I did not
use an ISO 8601 format either; if you really want your logs to be
easy to use for matching against other logs, possibly from other
servers, something like a `date --iso-8601=seconds` (with GNU date) or
passing %Y%m%dT%H:%M:%S%z as a format string to a strftime-like
formatting function would be much preferable.

G'luck,
Peter

-- 
Peter Pentchev  roam@{ringlet.net,debian.org,FreeBSD.org} pp...@st...
PGP key:        http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Sunday 30 September 2018 00:58:21 grarpamp wrote:

> > This makes the log entries look like this:
> >
> >    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> >    fetchmail [2018/09/29 17:04:42]: reading message ...
> >
> > Here's the patch I used for 6.3.26...
>
> Try reading and adopting a valid iso8601 standard format for
> timestamps instead of the above which are nonconformant
> and ambiguous in multiple ways.

And whats wrong with the above format? And since I know thats my machines 
local time in a valid 24 hour format, I do not see any ambiguity. 

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[grarpamp <gra...@gm...>]

> This makes the log entries look like this:
>
>    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
>    fetchmail [2018/09/29 17:04:42]: reading message ...
>
> Here's the patch I used for 6.3.26...

Try reading and adopting a valid iso8601 standard format for
timestamps instead of the above which are nonconformant
and ambiguous in multiple ways.

Re: [Fetchmail-users] [BULK] Re: how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Saturday 29 September 2018 21:36:37 Hans Carlson wrote:

> I ended up adding a few lines to the fetchmail source (see below).
> Someone a bit more enterprising than me could probably submit a patch
> with some kind of config file option (eg. log_timestamp) to do
> something similar.
>
> This makes the log entries look like this:
>
>    fetchmail [2018/09/29 17:04:41]: 2 messages for ...
>    fetchmail [2018/09/29 17:04:42]: reading message ...
>
> Here's the patch I used for 6.3.26...
>
> diff -ur fetchmail-6.3.26/report.c fetchmail-6.3.26-new/report.c
> --- fetchmail-6.3.26/report.c	2013-04-23 13:00:45.000000000 -0700
> +++ fetchmail-6.3.26-new/report.c	2017-09-28 23:25:10.443642824
> -0700
> @@ -125,13 +125,20 @@
>       else /* i. e. not using syslog */
>   #endif
>       {
> +	time_t      now;
> +	static char timebuf[20];
> +
>   	if ( *message == '\n' )
>   	{
>   	    fputc( '\n', errfp );
>   	    ++message;
>   	}
>   	if (!partial_suppress_tag)
> -		fprintf (errfp, "%s: ", program_name);
> +	{
> +		time(&now);
> +		strftime (timebuf, sizeof(timebuf), "%Y/%m/%d %H:%M:%S",
> localtime(&now)); +		fprintf (errfp, "%s [%s]: ", program_name,
> timebuf);
> +	}
>   	partial_suppress_tag = 0;
>
>   #ifdef VA_START

Works like a charm, thank you.

> On Sat, 29 Sep 2018, Gene Heskett wrote:
> > On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
> >> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> >>> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> >>>> Hi Gene,
> >>>>
> >>>>> I was subjected to a phishing attack yesterday and again today,
> >>>>> and it would have been a lot easier to correlate the messages if
> >>>>> fetchmail was logging the time, preferably the date/time at the
> >>>>> end of fetching of fetching each mail. Something it is not now,
> >>>>> and never has done.  And I just reread the man pages without
> >>>>> finding a clue.
> >>>>>
> >>>>> How do I enable this so a date/time exists in the fetchmail.log?
> >>>>
> >>>> I'm using the following two commands to log the start and end of
> >>>> request cyle:
> >>>>
> >>>> preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> >>>> %H:%M:%S (%Z)'`"
> >>>>
> >>>> postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> >>>> %H:%M:%S (%Z)'` for xxx seconds
> >>>
> >>> THis would triple the size of the log in terms of line count. What
> >>> I would really like to do is convert the single log line:
> >>>
> >>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1
> >>> of 1 (8044 octets) flushed
> >>>
> >>> to:
> >>>
> >>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1
> >>> of 1 (8044 octets) flushed 09/27/18 14;51.28
> >>
> >> This could be done if you pipe fetchmail's output through some
> >> other program that puts a timestamp on each line; multilog, the log
> >> processor from the daemontools package, comes to mind, but in a
> >> pinch this might be done with a small dedicated program, maybe
> >> something like this:
> >>
> >> #!/usr/bin/perl
> >>
> >> use 5.012;
> >> use strict;
> >> use warnings;
> >>
> >> use POSIX qw(strftime);
> >>
> >> while (<>) {
> >> 	print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> >> }
> >>
> >> ...or even something like:
> >>
> >> #!/bin/sh
> >>
> >> while read line; do
> >> 	printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> >> done
> >
> > I see, this last looks to be more promising. Looks like the
> > date/time would be prepended, the src of the line would be
> > intercepted so the extra data can be prepended.
> >
> >> But, yes, I do see your point that fetchmail could also do it by
> >> itself, although sometimes chaining a couple of tools is a good
> >> approach.
> >
> > Been known to do that many times in the past, but time has taken its
> > toll on my thinker.
> >
> >> G'luck,
> >> Peter
> >
> > Thank you Peter.



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:

> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> > On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> > > Hi Gene,
> > >
> > > > I was subjected to a phishing attack yesterday and again today,
> > > > and it would have been a lot easier to correlate the messages if
> > > > fetchmail was logging the time, preferably the date/time at the
> > > > end of fetching of fetching each mail. Something it is not now,
> > > > and never has done.  And I just reread the man pages without
> > > > finding a clue.
> > > >
> > > > How do I enable this so a date/time exists in the fetchmail.log?
> > >
> > > I'm using the following two commands to log the start and end of
> > > request cyle:
> > >
> > > preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'`"
> > >
> > > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'` for xxx seconds
> >
> > THis would triple the size of the log in terms of line count. What I
> > would really like to do is convert the single log line:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed
> >
> > to:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed 09/27/18 14;51.28
>
> This could be done if you pipe fetchmail's output through some other
> program that puts a timestamp on each line; multilog, the log
> processor from the daemontools package, comes to mind,
Got that, but will try to understand the manpage tomorrow

Thank you.

> but in a pinch 
> this might be done with a small dedicated program, maybe something
> like this:
>
> #!/usr/bin/perl
>
> use 5.012;
> use strict;
> use warnings;
>
> use POSIX qw(strftime);
>
> while (<>) {
> 	print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> }
>
> ...or even something like:
>
> #!/bin/sh
>
> while read line; do
> 	printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> done
>
> But, yes, I do see your point that fetchmail could also do it by
> itself, although sometimes chaining a couple of tools is a good
> approach.
>
> G'luck,
> Peter



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Hans C. <fo...@gm...>]

I ended up adding a few lines to the fetchmail source (see below). 
Someone a bit more enterprising than me could probably submit a patch with 
some kind of config file option (eg. log_timestamp) to do something 
similar.

This makes the log entries look like this:

   fetchmail [2018/09/29 17:04:41]: 2 messages for ...
   fetchmail [2018/09/29 17:04:42]: reading message ...

Here's the patch I used for 6.3.26...

diff -ur fetchmail-6.3.26/report.c fetchmail-6.3.26-new/report.c
--- fetchmail-6.3.26/report.c	2013-04-23 13:00:45.000000000 -0700
+++ fetchmail-6.3.26-new/report.c	2017-09-28 23:25:10.443642824 
-0700
@@ -125,13 +125,20 @@
      else /* i. e. not using syslog */
  #endif
      {
+	time_t      now;
+	static char timebuf[20];
+
  	if ( *message == '\n' )
  	{
  	    fputc( '\n', errfp );
  	    ++message;
  	}
  	if (!partial_suppress_tag)
-		fprintf (errfp, "%s: ", program_name);
+	{
+		time(&now);
+		strftime (timebuf, sizeof(timebuf), "%Y/%m/%d %H:%M:%S", localtime(&now));
+		fprintf (errfp, "%s [%s]: ", program_name, timebuf);
+	}
  	partial_suppress_tag = 0;

  #ifdef VA_START


On Sat, 29 Sep 2018, Gene Heskett wrote:

> On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
>
>> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
>>> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
>>>> Hi Gene,
>>>>
>>>>> I was subjected to a phishing attack yesterday and again today,
>>>>> and it would have been a lot easier to correlate the messages if
>>>>> fetchmail was logging the time, preferably the date/time at the
>>>>> end of fetching of fetching each mail. Something it is not now,
>>>>> and never has done.  And I just reread the man pages without
>>>>> finding a clue.
>>>>>
>>>>> How do I enable this so a date/time exists in the fetchmail.log?
>>>>
>>>> I'm using the following two commands to log the start and end of
>>>> request cyle:
>>>>
>>>> preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
>>>> %H:%M:%S (%Z)'`"
>>>>
>>>> postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
>>>> %H:%M:%S (%Z)'` for xxx seconds
>>>
>>> THis would triple the size of the log in terms of line count. What I
>>> would really like to do is convert the single log line:
>>>
>>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
>>> 1 (8044 octets) flushed
>>>
>>> to:
>>>
>>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
>>> 1 (8044 octets) flushed 09/27/18 14;51.28
>>
>> This could be done if you pipe fetchmail's output through some other
>> program that puts a timestamp on each line; multilog, the log
>> processor from the daemontools package, comes to mind, but in a pinch
>> this might be done with a small dedicated program, maybe something
>> like this:
>>
>> #!/usr/bin/perl
>>
>> use 5.012;
>> use strict;
>> use warnings;
>>
>> use POSIX qw(strftime);
>>
>> while (<>) {
>> 	print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
>> }
>>
>> ...or even something like:
>>
>> #!/bin/sh
>>
>> while read line; do
>> 	printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
>> done
>>
> I see, this last looks to be more promising. Looks like the date/time
> would be prepended, the src of the line would be intercepted so the
> extra data can be prepended.
>
>> But, yes, I do see your point that fetchmail could also do it by
>> itself, although sometimes chaining a couple of tools is a good
>> approach.
>
> Been known to do that many times in the past, but time has taken its toll
> on my thinker.
>>
>> G'luck,
>> Peter
>
> Thank you Peter.
>
>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:

> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> > On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> > > Hi Gene,
> > >
> > > > I was subjected to a phishing attack yesterday and again today,
> > > > and it would have been a lot easier to correlate the messages if
> > > > fetchmail was logging the time, preferably the date/time at the
> > > > end of fetching of fetching each mail. Something it is not now,
> > > > and never has done.  And I just reread the man pages without
> > > > finding a clue.
> > > >
> > > > How do I enable this so a date/time exists in the fetchmail.log?
> > >
> > > I'm using the following two commands to log the start and end of
> > > request cyle:
> > >
> > > preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'`"
> > >
> > > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'` for xxx seconds
> >
> > THis would triple the size of the log in terms of line count. What I
> > would really like to do is convert the single log line:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed
> >
> > to:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed 09/27/18 14;51.28
>
> This could be done if you pipe fetchmail's output through some other
> program that puts a timestamp on each line; multilog, the log
> processor from the daemontools package, comes to mind, but in a pinch
> this might be done with a small dedicated program, maybe something
> like this:
>
> #!/usr/bin/perl
>
> use 5.012;
> use strict;
> use warnings;
>
> use POSIX qw(strftime);
>
> while (<>) {
> 	print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> }
>
> ...or even something like:
>
> #!/bin/sh
>
> while read line; do
> 	printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> done
>
I see, this last looks to be more promising. Looks like the date/time 
would be prepended, the src of the line would be intercepted so the 
extra data can be prepended.

> But, yes, I do see your point that fetchmail could also do it by
> itself, although sometimes chaining a couple of tools is a good
> approach.

Been known to do that many times in the past, but time has taken its toll 
on my thinker.
>
> G'luck,
> Peter

Thank you Peter.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Peter P. <ro...@ri...>]

On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> 
> > Hi Gene,
> >
> > > I was subjected to a phishing attack yesterday and again today, and
> > > it would have been a lot easier to correlate the messages if
> > > fetchmail was logging the time, preferably the date/time at the end
> > > of fetching of fetching each mail. Something it is not now, and
> > > never has done.  And I just reread the man pages without finding a
> > > clue.
> > >
> > > How do I enable this so a date/time exists in the fetchmail.log?
> >
> > I'm using the following two commands to log the start and end of
> > request cyle:
> >
> > preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > %H:%M:%S (%Z)'`"
> >
> > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > %H:%M:%S (%Z)'` for xxx seconds
> >
> THis would triple the size of the log in terms of line count. What I 
> would really like to do is convert the single log line:
> 
> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 
> (8044 octets) flushed
> 
> to:
> 
> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 
> (8044 octets) flushed 09/27/18 14;51.28

This could be done if you pipe fetchmail's output through some other
program that puts a timestamp on each line; multilog, the log processor
from the daemontools package, comes to mind, but in a pinch this might
be done with a small dedicated program, maybe something like this:

#!/usr/bin/perl

use 5.012;
use strict;
use warnings;

use POSIX qw(strftime);

while (<>) {
	print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
}

...or even something like:

#!/bin/sh

while read line; do
	printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
done

But, yes, I do see your point that fetchmail could also do it by itself,
although sometimes chaining a couple of tools is a good approach.

G'luck,
Peter

-- 
Peter Pentchev  roam@{ringlet.net,debian.org,FreeBSD.org} pp...@st...
PGP key:        http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:

> Hi Gene,
>
> > I was subjected to a phishing attack yesterday and again today, and
> > it would have been a lot easier to correlate the messages if
> > fetchmail was logging the time, preferably the date/time at the end
> > of fetching of fetching each mail. Something it is not now, and
> > never has done.  And I just reread the man pages without finding a
> > clue.
> >
> > How do I enable this so a date/time exists in the fetchmail.log?
>
> I'm using the following two commands to log the start and end of
> request cyle:
>
> preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> %H:%M:%S (%Z)'`"
>
> postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> %H:%M:%S (%Z)'` for xxx seconds
>
THis would triple the size of the log in terms of line count. What I 
would really like to do is convert the single log line:

fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 
(8044 octets) flushed

to:

fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 
(8044 octets) flushed 09/27/18 14;51.28

Against this server, flush doesn't work as its a dovecot intended as an 
imap server, but accessed by pop3, so the del command is ignored until I 
log in with firefox, then I can delete the contents, but fetchmail 
can't.

> Further details can be found here:
>
> http://www.fetchmail.info/fetchmail-man.html

containing virtually zero about action timestamps.

Perhaps this should be turned into a wishlist for the next release of 
fetchmail?

Procmail, which I use as the MTA/MDA does this nicely. And doing it in 
both, I can get a better idea of how much time it takes procmail to run 
everything thru all the filters.

Thanks.

> Regards
> Juergen



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] how to log date/time in daemon mode?

[Juergen E. <fli...@te...>]

Hi Gene,

> I was subjected to a phishing attack yesterday and again today, and it 
> would have been a lot easier to correlate the messages if fetchmail was 
> logging the time, preferably the date/time at the end of fetching of 
> fetching each mail. Something it is not now, and never has done.  And I 
> just reread the man pages without finding a clue.
> 
> How do I enable this so a date/time exists in the fetchmail.log?

I'm using the following two commands to log the start and end of
request cyle:

preconnect  "echo 'fetchmail: awakened at '`date +'%a, %d %b %G %H:%M:%S
(%Z)'`"

postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G %H:%M:%S
(%Z)'` for xxx seconds

Further details can be found here:

http://www.fetchmail.info/fetchmail-man.html

Regards
Juergen
-- 
Mail: fli...@te...

[Fetchmail-users] how to log date/time in daemon mode?

[Gene H. <ghe...@sh...>]

Greetings, long time fetchmail user here;

I was subjected to a phishing attack yesterday and again today, and it 
would have been a lot easier to correlate the messages if fetchmail was 
logging the time, preferably the date/time at the end of fetching of 
fetching each mail. Something it is not now, and never has done.  And I 
just reread the man pages without finding a clue.

How do I enable this so a date/time exists in the fetchmail.log?

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] tls bug and probably patch

[Matthias A. <mat...@gm...>]

Am 17.09.18 um 07:13 schrieb Kamil Jońca:
> Recently I got in fetchmail logs, something like this;
>
> --8<---------------cut here---------------start------------->8---
> Missing trust anchor certificate: /OU=No SNI provided; please fix your client./CN=invalid2.invalid
> --8<---------------cut here---------------end--------------->8---
> some googling, gives me:
> https://bugzilla.redhat.com/attachment.cgi?id=1472812&action=diff
>
> and applying this change in 7.0.0-alpha6+SSL+NLS. in socket.c 
> around line 1035 helped (i think)

What do I have to do to get imap.gmail.com to complain without SNI?


Anyways, as per my comments in that bugtracker entry and on the
fetchmail developer's mailing list, that patch is inadequate as it lacks
error checking/reporting.

Try the master branch from Git, this morning I have cherry-picked the
relevant patch from the legacy_64 branch, which has been in the latest
6.4 beta releases already.

Cheers,
Matthias

[Fetchmail-users] tls bug and probably patch

[<kj...@o2...>]

Recently I got in fetchmail logs, something like this;

--8<---------------cut here---------------start------------->8---
Missing trust anchor certificate: /OU=No SNI provided; please fix your client./CN=invalid2.invalid
--8<---------------cut here---------------end--------------->8---
some googling, gives me:
https://bugzilla.redhat.com/attachment.cgi?id=1472812&action=diff

and applying this change in 7.0.0-alpha6+SSL+NLS. in socket.c 
around line 1035 helped (i think)
KJ

-- 
http://stopstopnop.pl/stop_stopnop.pl_o_nas.html
Trying to define yourself is like trying to bite your own teeth.
		-- Alan Watts

Re: [Fetchmail-users] Fetchmail only writes the most recent mail

[Matthias A. <mat...@gm...>]

Am 09.08.2018 um 13:21 schrieb daniel_1983--- via Fetchmail-users:
> Hello !
>
> I'm trying fetchmail on the command line, here's what I came up with
>
> fetchmail -p IMAP \ # use the IMAP protocol
> -r "INBOX.Brigands" \ # to fetch mail from the Brigands folder
> -k \ # keep mail on the server
> -a \ # retrieve seen and unseen mail
> -u user@domain mail.server.com \ # my username
> --mda "/bin/sh -c 'cat > /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new/$(date +%''s_%N)'" # my MDA

The problem is in your --mda option - you need >>, not >, for output
redirection. Read any decent manual about Bourne or POSIX-like shells.

Re: [Fetchmail-users] Fetchmail only writes the most recent mail

[Ralph C. <ra...@in...>]

Hi Daniel,

> --mda "/bin/sh -c 'cat > /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new/$(date +%''s_%N)'"

This is wrong.  The $() is expanded once by your shell before fetchmail
is run because it's in a ""-quoted string.  Put a backslash before
the `$'.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [Fetchmail-users] Fetchmail only writes the most recent mail

[<dan...@pr...>]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On August 9, 2018 12:27 PM, Ralph Corderoy <ra...@in...> wrote:

> Hi Daniel,
>
> > --mda "/bin/sh -c 'cat > /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new/$(date +%''s_%N)'"
>
> This is wrong. The $() is expanded once by your shell before fetchmail
> is run because it's in a ""-quoted string. Put a backslash before
> the `$'.
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy

My highest of fives to you, good sir !