Menu
▾
▴
fetchmail-users
|
From: Juergen E. <fli...@te...> - 2018-09-29 17:27:36
|
Hi Gene, > I was subjected to a phishing attack yesterday and again today, and it > would have been a lot easier to correlate the messages if fetchmail was > logging the time, preferably the date/time at the end of fetching of > fetching each mail. Something it is not now, and never has done. And I > just reread the man pages without finding a clue. > > How do I enable this so a date/time exists in the fetchmail.log? I'm using the following two commands to log the start and end of request cyle: preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G %H:%M:%S (%Z)'`" postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G %H:%M:%S (%Z)'` for xxx seconds Further details can be found here: http://www.fetchmail.info/fetchmail-man.html Regards Juergen -- Mail: fli...@te... |
|
From: Gene H. <ghe...@sh...> - 2018-09-29 18:18:04
|
On Saturday 29 September 2018 13:10:49 Juergen Edner wrote: > Hi Gene, > > > I was subjected to a phishing attack yesterday and again today, and > > it would have been a lot easier to correlate the messages if > > fetchmail was logging the time, preferably the date/time at the end > > of fetching of fetching each mail. Something it is not now, and > > never has done. And I just reread the man pages without finding a > > clue. > > > > How do I enable this so a date/time exists in the fetchmail.log? > > I'm using the following two commands to log the start and end of > request cyle: > > preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G > %H:%M:%S (%Z)'`" > > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G > %H:%M:%S (%Z)'` for xxx seconds > THis would triple the size of the log in terms of line count. What I would really like to do is convert the single log line: fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 (8044 octets) flushed to: fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1 (8044 octets) flushed 09/27/18 14;51.28 Against this server, flush doesn't work as its a dovecot intended as an imap server, but accessed by pop3, so the del command is ignored until I log in with firefox, then I can delete the contents, but fetchmail can't. > Further details can be found here: > > http://www.fetchmail.info/fetchmail-man.html containing virtually zero about action timestamps. Perhaps this should be turned into a wishlist for the next release of fetchmail? Procmail, which I use as the MTA/MDA does this nicely. And doing it in both, I can get a better idea of how much time it takes procmail to run everything thru all the filters. Thanks. > Regards > Juergen -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page <http://geneslinuxbox.net:6309/gene> |
|
From: Peter P. <ro...@ri...> - 2018-09-29 23:49:28
Attachments:
signature.asc
|
On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
>
> > Hi Gene,
> >
> > > I was subjected to a phishing attack yesterday and again today, and
> > > it would have been a lot easier to correlate the messages if
> > > fetchmail was logging the time, preferably the date/time at the end
> > > of fetching of fetching each mail. Something it is not now, and
> > > never has done. And I just reread the man pages without finding a
> > > clue.
> > >
> > > How do I enable this so a date/time exists in the fetchmail.log?
> >
> > I'm using the following two commands to log the start and end of
> > request cyle:
> >
> > preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > %H:%M:%S (%Z)'`"
> >
> > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > %H:%M:%S (%Z)'` for xxx seconds
> >
> THis would triple the size of the log in terms of line count. What I
> would really like to do is convert the single log line:
>
> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1
> (8044 octets) flushed
>
> to:
>
> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of 1
> (8044 octets) flushed 09/27/18 14;51.28
This could be done if you pipe fetchmail's output through some other
program that puts a timestamp on each line; multilog, the log processor
from the daemontools package, comes to mind, but in a pinch this might
be done with a small dedicated program, maybe something like this:
#!/usr/bin/perl
use 5.012;
use strict;
use warnings;
use POSIX qw(strftime);
while (<>) {
print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
}
...or even something like:
#!/bin/sh
while read line; do
printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
done
But, yes, I do see your point that fetchmail could also do it by itself,
although sometimes chaining a couple of tools is a good approach.
G'luck,
Peter
--
Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} pp...@st...
PGP key: http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13
|
|
From: Gene H. <ghe...@sh...> - 2018-09-30 03:49:10
|
On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> > On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> > > Hi Gene,
> > >
> > > > I was subjected to a phishing attack yesterday and again today,
> > > > and it would have been a lot easier to correlate the messages if
> > > > fetchmail was logging the time, preferably the date/time at the
> > > > end of fetching of fetching each mail. Something it is not now,
> > > > and never has done. And I just reread the man pages without
> > > > finding a clue.
> > > >
> > > > How do I enable this so a date/time exists in the fetchmail.log?
> > >
> > > I'm using the following two commands to log the start and end of
> > > request cyle:
> > >
> > > preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'`"
> > >
> > > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'` for xxx seconds
> >
> > THis would triple the size of the log in terms of line count. What I
> > would really like to do is convert the single log line:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed
> >
> > to:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed 09/27/18 14;51.28
>
> This could be done if you pipe fetchmail's output through some other
> program that puts a timestamp on each line; multilog, the log
> processor from the daemontools package, comes to mind,
Got that, but will try to understand the manpage tomorrow
Thank you.
> but in a pinch
> this might be done with a small dedicated program, maybe something
> like this:
>
> #!/usr/bin/perl
>
> use 5.012;
> use strict;
> use warnings;
>
> use POSIX qw(strftime);
>
> while (<>) {
> print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> }
>
> ...or even something like:
>
> #!/bin/sh
>
> while read line; do
> printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> done
>
> But, yes, I do see your point that fetchmail could also do it by
> itself, although sometimes chaining a couple of tools is a good
> approach.
>
> G'luck,
> Peter
--
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
|
|
From: Gene H. <ghe...@sh...> - 2018-09-30 00:00:50
|
On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> > On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> > > Hi Gene,
> > >
> > > > I was subjected to a phishing attack yesterday and again today,
> > > > and it would have been a lot easier to correlate the messages if
> > > > fetchmail was logging the time, preferably the date/time at the
> > > > end of fetching of fetching each mail. Something it is not now,
> > > > and never has done. And I just reread the man pages without
> > > > finding a clue.
> > > >
> > > > How do I enable this so a date/time exists in the fetchmail.log?
> > >
> > > I'm using the following two commands to log the start and end of
> > > request cyle:
> > >
> > > preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'`"
> > >
> > > postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> > > %H:%M:%S (%Z)'` for xxx seconds
> >
> > THis would triple the size of the log in terms of line count. What I
> > would really like to do is convert the single log line:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed
> >
> > to:
> >
> > fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
> > 1 (8044 octets) flushed 09/27/18 14;51.28
>
> This could be done if you pipe fetchmail's output through some other
> program that puts a timestamp on each line; multilog, the log
> processor from the daemontools package, comes to mind, but in a pinch
> this might be done with a small dedicated program, maybe something
> like this:
>
> #!/usr/bin/perl
>
> use 5.012;
> use strict;
> use warnings;
>
> use POSIX qw(strftime);
>
> while (<>) {
> print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> }
>
> ...or even something like:
>
> #!/bin/sh
>
> while read line; do
> printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> done
>
I see, this last looks to be more promising. Looks like the date/time
would be prepended, the src of the line would be intercepted so the
extra data can be prepended.
> But, yes, I do see your point that fetchmail could also do it by
> itself, although sometimes chaining a couple of tools is a good
> approach.
Been known to do that many times in the past, but time has taken its toll
on my thinker.
>
> G'luck,
> Peter
Thank you Peter.
--
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
|
|
From: Hans C. <fo...@gm...> - 2018-09-30 01:54:36
|
I ended up adding a few lines to the fetchmail source (see below).
Someone a bit more enterprising than me could probably submit a patch with
some kind of config file option (eg. log_timestamp) to do something
similar.
This makes the log entries look like this:
fetchmail [2018/09/29 17:04:41]: 2 messages for ...
fetchmail [2018/09/29 17:04:42]: reading message ...
Here's the patch I used for 6.3.26...
diff -ur fetchmail-6.3.26/report.c fetchmail-6.3.26-new/report.c
--- fetchmail-6.3.26/report.c 2013-04-23 13:00:45.000000000 -0700
+++ fetchmail-6.3.26-new/report.c 2017-09-28 23:25:10.443642824
-0700
@@ -125,13 +125,20 @@
else /* i. e. not using syslog */
#endif
{
+ time_t now;
+ static char timebuf[20];
+
if ( *message == '\n' )
{
fputc( '\n', errfp );
++message;
}
if (!partial_suppress_tag)
- fprintf (errfp, "%s: ", program_name);
+ {
+ time(&now);
+ strftime (timebuf, sizeof(timebuf), "%Y/%m/%d %H:%M:%S", localtime(&now));
+ fprintf (errfp, "%s [%s]: ", program_name, timebuf);
+ }
partial_suppress_tag = 0;
#ifdef VA_START
On Sat, 29 Sep 2018, Gene Heskett wrote:
> On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
>
>> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
>>> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
>>>> Hi Gene,
>>>>
>>>>> I was subjected to a phishing attack yesterday and again today,
>>>>> and it would have been a lot easier to correlate the messages if
>>>>> fetchmail was logging the time, preferably the date/time at the
>>>>> end of fetching of fetching each mail. Something it is not now,
>>>>> and never has done. And I just reread the man pages without
>>>>> finding a clue.
>>>>>
>>>>> How do I enable this so a date/time exists in the fetchmail.log?
>>>>
>>>> I'm using the following two commands to log the start and end of
>>>> request cyle:
>>>>
>>>> preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
>>>> %H:%M:%S (%Z)'`"
>>>>
>>>> postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
>>>> %H:%M:%S (%Z)'` for xxx seconds
>>>
>>> THis would triple the size of the log in terms of line count. What I
>>> would really like to do is convert the single log line:
>>>
>>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
>>> 1 (8044 octets) flushed
>>>
>>> to:
>>>
>>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1 of
>>> 1 (8044 octets) flushed 09/27/18 14;51.28
>>
>> This could be done if you pipe fetchmail's output through some other
>> program that puts a timestamp on each line; multilog, the log
>> processor from the daemontools package, comes to mind, but in a pinch
>> this might be done with a small dedicated program, maybe something
>> like this:
>>
>> #!/usr/bin/perl
>>
>> use 5.012;
>> use strict;
>> use warnings;
>>
>> use POSIX qw(strftime);
>>
>> while (<>) {
>> print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
>> }
>>
>> ...or even something like:
>>
>> #!/bin/sh
>>
>> while read line; do
>> printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
>> done
>>
> I see, this last looks to be more promising. Looks like the date/time
> would be prepended, the src of the line would be intercepted so the
> extra data can be prepended.
>
>> But, yes, I do see your point that fetchmail could also do it by
>> itself, although sometimes chaining a couple of tools is a good
>> approach.
>
> Been known to do that many times in the past, but time has taken its toll
> on my thinker.
>>
>> G'luck,
>> Peter
>
> Thank you Peter.
>
>
|
|
From: Gene H. <ghe...@sh...> - 2018-09-30 04:18:42
|
On Saturday 29 September 2018 21:36:37 Hans Carlson wrote:
> I ended up adding a few lines to the fetchmail source (see below).
> Someone a bit more enterprising than me could probably submit a patch
> with some kind of config file option (eg. log_timestamp) to do
> something similar.
>
> This makes the log entries look like this:
>
> fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> fetchmail [2018/09/29 17:04:42]: reading message ...
>
> Here's the patch I used for 6.3.26...
>
> diff -ur fetchmail-6.3.26/report.c fetchmail-6.3.26-new/report.c
> --- fetchmail-6.3.26/report.c 2013-04-23 13:00:45.000000000 -0700
> +++ fetchmail-6.3.26-new/report.c 2017-09-28 23:25:10.443642824
> -0700
> @@ -125,13 +125,20 @@
> else /* i. e. not using syslog */
> #endif
> {
> + time_t now;
> + static char timebuf[20];
> +
> if ( *message == '\n' )
> {
> fputc( '\n', errfp );
> ++message;
> }
> if (!partial_suppress_tag)
> - fprintf (errfp, "%s: ", program_name);
> + {
> + time(&now);
> + strftime (timebuf, sizeof(timebuf), "%Y/%m/%d %H:%M:%S",
> localtime(&now)); + fprintf (errfp, "%s [%s]: ", program_name,
> timebuf);
> + }
> partial_suppress_tag = 0;
>
> #ifdef VA_START
Works like a charm, thank you.
> On Sat, 29 Sep 2018, Gene Heskett wrote:
> > On Saturday 29 September 2018 19:30:41 Peter Pentchev wrote:
> >> On Sat, Sep 29, 2018 at 02:17:55PM -0400, Gene Heskett wrote:
> >>> On Saturday 29 September 2018 13:10:49 Juergen Edner wrote:
> >>>> Hi Gene,
> >>>>
> >>>>> I was subjected to a phishing attack yesterday and again today,
> >>>>> and it would have been a lot easier to correlate the messages if
> >>>>> fetchmail was logging the time, preferably the date/time at the
> >>>>> end of fetching of fetching each mail. Something it is not now,
> >>>>> and never has done. And I just reread the man pages without
> >>>>> finding a clue.
> >>>>>
> >>>>> How do I enable this so a date/time exists in the fetchmail.log?
> >>>>
> >>>> I'm using the following two commands to log the start and end of
> >>>> request cyle:
> >>>>
> >>>> preconnect "echo 'fetchmail: awakened at '`date +'%a, %d %b %G
> >>>> %H:%M:%S (%Z)'`"
> >>>>
> >>>> postconnect "echo 'fetchmail: sleeping at '`date +'%a, %d %b %G
> >>>> %H:%M:%S (%Z)'` for xxx seconds
> >>>
> >>> THis would triple the size of the log in terms of line count. What
> >>> I would really like to do is convert the single log line:
> >>>
> >>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1
> >>> of 1 (8044 octets) flushed
> >>>
> >>> to:
> >>>
> >>> fetchmail: reading message ghe...@sh...@pop.shentel.net:1
> >>> of 1 (8044 octets) flushed 09/27/18 14;51.28
> >>
> >> This could be done if you pipe fetchmail's output through some
> >> other program that puts a timestamp on each line; multilog, the log
> >> processor from the daemontools package, comes to mind, but in a
> >> pinch this might be done with a small dedicated program, maybe
> >> something like this:
> >>
> >> #!/usr/bin/perl
> >>
> >> use 5.012;
> >> use strict;
> >> use warnings;
> >>
> >> use POSIX qw(strftime);
> >>
> >> while (<>) {
> >> print strftime('%Y-%m-%d %H:%M:%S', localtime)." $_";
> >> }
> >>
> >> ...or even something like:
> >>
> >> #!/bin/sh
> >>
> >> while read line; do
> >> printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$line"
> >> done
> >
> > I see, this last looks to be more promising. Looks like the
> > date/time would be prepended, the src of the line would be
> > intercepted so the extra data can be prepended.
> >
> >> But, yes, I do see your point that fetchmail could also do it by
> >> itself, although sometimes chaining a couple of tools is a good
> >> approach.
> >
> > Been known to do that many times in the past, but time has taken its
> > toll on my thinker.
> >
> >> G'luck,
> >> Peter
> >
> > Thank you Peter.
--
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
|
|
From: grarpamp <gra...@gm...> - 2018-09-30 04:58:29
|
> This makes the log entries look like this: > > fetchmail [2018/09/29 17:04:41]: 2 messages for ... > fetchmail [2018/09/29 17:04:42]: reading message ... > > Here's the patch I used for 6.3.26... Try reading and adopting a valid iso8601 standard format for timestamps instead of the above which are nonconformant and ambiguous in multiple ways. |
|
From: Gene H. <ghe...@sh...> - 2018-09-30 08:52:14
|
On Sunday 30 September 2018 00:58:21 grarpamp wrote: > > This makes the log entries look like this: > > > > fetchmail [2018/09/29 17:04:41]: 2 messages for ... > > fetchmail [2018/09/29 17:04:42]: reading message ... > > > > Here's the patch I used for 6.3.26... > > Try reading and adopting a valid iso8601 standard format for > timestamps instead of the above which are nonconformant > and ambiguous in multiple ways. And whats wrong with the above format? And since I know thats my machines local time in a valid 24 hour format, I do not see any ambiguity. -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page <http://geneslinuxbox.net:6309/gene> |
|
From: grarpamp <gra...@gm...> - 2018-09-30 18:57:01
|
> And whats wrong with the above format? For that, refer to the International Standard 8601:2004 which you can find pdf from any search. A number of rationale are also included in it. Also note, - spaces are forbidden separator, format is a single field string - slashes have special purpose interval, and are not a separator - lower case is forbidden where upper case available - etc, see the standard, ignore such at peril of rockets crash |
|
From: Peter P. <ro...@ri...> - 2018-09-30 10:54:09
Attachments:
signature.asc
|
On Sun, Sep 30, 2018 at 04:51:51AM -0400, Gene Heskett wrote:
> On Sunday 30 September 2018 00:58:21 grarpamp wrote:
>
> > > This makes the log entries look like this:
> > >
> > > fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> > > fetchmail [2018/09/29 17:04:42]: reading message ...
> > >
> > > Here's the patch I used for 6.3.26...
> >
> > Try reading and adopting a valid iso8601 standard format for
> > timestamps instead of the above which are nonconformant
> > and ambiguous in multiple ways.
>
> And whats wrong with the above format? And since I know thats my machines
> local time in a valid 24 hour format, I do not see any ambiguity.
Problems will arise once you try to coordinate logs from different
machines or send these logs to somebody else who has to match them
against logs from their mailserver which may be in a different
timezone - and, unless you tell them what your timezone is, there is
no way they know what time your 17:04:41 is on their side. I have to
deal with something like this with syslog files from machines in
different timezones almost every week :( (and yes, most of the time
I know which timezone which log comes from, but it's still much harder
than it needs to be to combine and process them automatically)
Another way this is ambiguous is made a lot better by the use of
four digits for the year, but the fact still remains that there are
some areas in the world which actually use a YYYY/DD/MM format, just
as there are still some (very, very weird from my point of view, but
you get the point) areas in the world which still use a MM/DD format :)
The four-digit year removes the MM/DD/YY vs DD/MM/YY vs YY/MM/DD
ambiguity (which I've had to deal with in the past and it was awful),
but, very slim as the chances are for a YYYY/DD/MM vs YYYY/MM/DD one,
it is still better to use a well-known (and recognizable) format to
avoid *any* ambiguities.
So, yes, I accept the criticism, since in my code snippets I did not
use an ISO 8601 format either; if you really want your logs to be
easy to use for matching against other logs, possibly from other
servers, something like a `date --iso-8601=seconds` (with GNU date) or
passing %Y%m%dT%H:%M:%S%z as a format string to a strftime-like
formatting function would be much preferable.
G'luck,
Peter
--
Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} pp...@st...
PGP key: http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13
|
|
From: Gene H. <ghe...@sh...> - 2018-09-30 13:14:50
|
On Sunday 30 September 2018 06:53:53 Peter Pentchev wrote: > On Sun, Sep 30, 2018 at 04:51:51AM -0400, Gene Heskett wrote: > > On Sunday 30 September 2018 00:58:21 grarpamp wrote: > > > > This makes the log entries look like this: > > > > > > > > fetchmail [2018/09/29 17:04:41]: 2 messages for ... > > > > fetchmail [2018/09/29 17:04:42]: reading message ... > > > > > > > > Here's the patch I used for 6.3.26... > > > > > > Try reading and adopting a valid iso8601 standard format for > > > timestamps instead of the above which are nonconformant > > > and ambiguous in multiple ways. > > > > And whats wrong with the above format? And since I know thats my > > machines local time in a valid 24 hour format, I do not see any > > ambiguity. > > Problems will arise once you try to coordinate logs from different > machines or send these logs to somebody else who has to match them > against logs from their mailserver which may be in a different > timezone - and, unless you tell them what your timezone is, there is > no way they know what time your 17:04:41 is on their side. I have to > deal with something like this with syslog files from machines in > different timezones almost every week :( (and yes, most of the time > I know which timezone which log comes from, but it's still much harder > than it needs to be to combine and process them automatically) > > Another way this is ambiguous is made a lot better by the use of > four digits for the year, but the fact still remains that there are > some areas in the world which actually use a YYYY/DD/MM format, just > as there are still some (very, very weird from my point of view, but > you get the point) areas in the world which still use a MM/DD format > :) The four-digit year removes the MM/DD/YY vs DD/MM/YY vs YY/MM/DD > ambiguity (which I've had to deal with in the past and it was awful), > but, very slim as the chances are for a YYYY/DD/MM vs YYYY/MM/DD one, > it is still better to use a well-known (and recognizable) format to > avoid *any* ambiguities. > > So, yes, I accept the criticism, since in my code snippets I did not > use an ISO 8601 format either; if you really want your logs to be > easy to use for matching against other logs, possibly from other > servers, something like a `date --iso-8601=seconds` (with GNU date) or > passing %Y%m%dT%H:%M:%S%z as a format string to a strftime-like > formatting function would be much preferable. > > G'luck, > Peter And that is essentially the same with an added -0400 which is the local EDT time zone offset from GMT. But since the only person interested in my fetchmail.log is me, it definitely comes under the category of "good enough for the girls I go with." ;-) Patches welcome however. Take care Peter, and thanks for the explanation. -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page <http://geneslinuxbox.net:6309/gene> |
|
From: Ralph C. <ra...@in...> - 2018-09-30 13:49:54
|
Hi Gene,
> > > > > fetchmail [2018/09/29 17:04:41]: 2 messages for ...
> > >
> > > And whats wrong with the above format? And since I know thats my
> > > machines local time in a valid 24 hour format, I do not see any
> > > ambiguity.
>
> And that is essentially the same with an added -0400 which is the
> local EDT time zone offset from GMT.
It is still ambiguous. :-)
$ TZ=America/New_York date -d 2018-11-04T05:30:00Z +'%Y/%m/%d %H:%M:%S'
2018/11/04 01:30:00
$ TZ=America/New_York date -d 2018-11-04T06:30:00Z +'%Y/%m/%d %H:%M:%S'
2018/11/04 01:30:00
$
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
|
|
From: Hans C. <fo...@gm...> - 2018-10-01 17:44:22
|
On Sun, 30 Sep 2018, grarpamp wrote: >> This makes the log entries look like this: >> >> fetchmail [2018/09/29 17:04:41]: 2 messages for ... >> fetchmail [2018/09/29 17:04:42]: reading message ... >> >> Here's the patch I used for 6.3.26... > > Try reading and adopting a valid iso8601 standard format for timestamps > instead of the above which are nonconformant and ambiguous in multiple > ways. Feel free to change the format however you like. This patch was a "quick fix" for something I found annoying with the fetchmail logs... nothing more than that. I saw someone else with a similar need and provided an option. I understand your point and it's a very good one, but I intentionally selected that format because it's what I want. It gives me exactly the information I need and no more. It is not ambiguous (to ME, for MY purposes) in any way. Different people have different needs... so I'll amend my original suggestion a bit... Someone a bit more enterprising than me could probably submit a patch with some kind of config file option (eg. log_timestamp) to do something similar. It would be wise to also add a config file option that specified the format for the timestamp, with the default timestamp something more universally acceptable and standard than the format I used. |
|
From: Matthias A. <mat...@gm...> - 2018-10-01 18:00:23
|
Am 01.10.18 um 19:19 schrieb Hans Carlson: > Someone a bit more enterprising than me could probably submit a patch > with some kind of config file option (eg. log_timestamp) to do > something similar. It would be wise to also add a config file option > that specified the format for the timestamp, with the default > timestamp something more universally acceptable and standard than the > format I used. Hi Gene, Hans, everybody else, I'd be bold (as the maintainer) and share the insight that adding options is an expensive process. It requires options parsing, twice (command line, and rcfile), option dumping, at that, documentation, testing, string translations, future compatibility considerations - for a few lines of code around strftime() and gmtime() calls we'd easily add 100 lines over the place for making things configurable. I am somewhat inclined to take a variant of your (Hans's) patch and make it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity of 2:05 A and 2:05 B in the night when the DST is switched off), and that's it. The alternative would be a plain timestamp (seconds since UTC Epoch or something) that would require further formatting through an external program---but then again a program that translated ISO8601 UTC timestamps to an arbitrary local time zone wouldn't have to be much more than a few lines of Perl. The consideration is whether this could go into 6.4, or would have to wait for 7.0. The log file format is not really specified, so changing it isn't breaking any documented interface, just breaking past practice, so it might cause some astonishment. Opinions? Best regards, Matthias |
|
From: Gene H. <ghe...@sh...> - 2018-10-01 22:50:16
|
On Monday 01 October 2018 14:00:03 Matthias Andree wrote: > Am 01.10.18 um 19:19 schrieb Hans Carlson: > > Someone a bit more enterprising than me could probably submit a > > patch with some kind of config file option (eg. log_timestamp) to do > > something similar. It would be wise to also add a config file > > option that specified the format for the timestamp, with the default > > timestamp something more universally acceptable and standard than > > the format I used. > > Hi Gene, Hans, everybody else, > > I'd be bold (as the maintainer) and share the insight that adding > options is an expensive process. It requires options parsing, twice > (command line, and rcfile), option dumping, at that, documentation, > testing, string translations, future compatibility considerations - > for a few lines of code around strftime() and gmtime() calls we'd > easily add 100 lines over the place for making things configurable. > > I am somewhat inclined to take a variant of your (Hans's) patch and > make it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity > of 2:05 A and 2:05 B in the night when the DST is switched off), and > that's it. I'd not have a problem if it was in UTC. The manpage should clearly reflect that however. > The alternative would be a plain timestamp (seconds since > UTC Epoch or something) that would require further formatting through > an external program---but then again a program that translated ISO8601 > UTC timestamps to an arbitrary local time zone wouldn't have to be > much more than a few lines of Perl. > > The consideration is whether this could go into 6.4, or would have to > wait for 7.0. The log file format is not really specified, so changing > it isn't breaking any documented interface, just breaking past > practice, so it might cause some astonishment. Opinions? > > Best regards, > Matthias > > > > > _______________________________________________ > Fetchmail-users mailing list > Fet...@li... > https://lists.sourceforge.net/lists/listinfo/fetchmail-users -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page <http://geneslinuxbox.net:6309/gene> |
|
From: <kj...@o2...> - 2018-10-01 18:09:35
|
Matthias Andree <mat...@gm...> writes: [..] > > The consideration is whether this could go into 6.4, or would have to > wait for 7.0. The log file format is not really specified, so changing > it isn't breaking any documented interface, just breaking past practice, > so it might cause some astonishment. Opinions? Regardless of these questions I would as NOT to change logging via syslog. KJ -- http://wolnelektury.pl/wesprzyj/teraz/ Do not remove tag under penalty of law. |
|
From: Hans C. <fo...@gm...> - 2018-10-01 22:54:40
|
On Mon, 1 Oct 2018, Matthias Andree wrote: > I am somewhat inclined to take a variant of your (Hans's) patch and make > it emit ISO8601 formatted time stamps, in UTC (to avoid ambiguity of > 2:05 A and 2:05 B in the night when the DST is switched off), and that's > it. The alternative would be a plain timestamp (seconds since UTC Epoch > or something) Would it be unreasonable to include both ISO8601 and time_t? eg. 2018-10-01T22:50:44+00:00 (1538434244) The time_t would simply be easier for an external script to parse. |
|
From: grarpamp <gra...@gm...> - 2018-10-02 06:26:21
|
> Would it be unreasonable to include both ISO8601 and time_t? eg. > > 2018-10-01T22:50:44+00:00 (1538434244) > > The time_t would simply be easier for an external script to parse. Some apps have user configurable global choice to make happy / useful... a) iso8601 (default) b) $(date) c) epoch d) strftime format string, ie above: "%Y-%m-%dT%H:%M:%S%z (%s)" Rightly leaving the box, localtime, and TZ env, to handle the timezones and such. As to when / if anything, 7.0 would seem best oppurtunity, as there will be some change and other new things and oppurtunities there anyways. |
|
From: Carlos E. R. <rob...@te...> - 2018-10-02 17:34:27
Attachments:
signature.asc
|
On 01/10/2018 20.09, Kamil Jońca wrote: > Matthias Andree <mat...@gm...> writes: > > [..] >> >> The consideration is whether this could go into 6.4, or would have to >> wait for 7.0. The log file format is not really specified, so changing >> it isn't breaking any documented interface, just breaking past practice, >> so it might cause some astonishment. Opinions? > > Regardless of these questions I would as NOT to change logging via > syslog. Timestamping via syslog is done by the syslog daemon, not by the application that sends the messages. -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.0 (Legolas)) |
|
From: <kj...@o2...> - 2018-10-02 18:10:33
|
"Carlos E. R." <rob...@te...> writes: > On 01/10/2018 20.09, Kamil Jońca wrote: >> Matthias Andree <mat...@gm...> writes: >> >> [..] >>> >>> The consideration is whether this could go into 6.4, or would have to >>> wait for 7.0. The log file format is not really specified, so changing >>> it isn't breaking any documented interface, just breaking past practice, >>> so it might cause some astonishment. Opinions? >> >> Regardless of these questions I would as NOT to change logging via >> syslog. > > Timestamping via syslog is done by the syslog daemon, not by the > application that sends the messages. And? KJ -- http://stopstopnop.pl/stop_stopnop.pl_o_nas.html You climb to reach the summit, but once there, discover that all roads lead down. -- Stanislaw Lem, "The Cyberiad" |
|
From: Carlos E. R. <rob...@te...> - 2018-10-02 18:56:11
Attachments:
signature.asc
|
On 02/10/2018 20.03, Kamil Jońca wrote: > "Carlos E. R." <rob...@te...> writes: > >> On 01/10/2018 20.09, Kamil Jońca wrote: >>> Matthias Andree <mat...@gm...> writes: >>> >>> [..] >>>> >>>> The consideration is whether this could go into 6.4, or would have to >>>> wait for 7.0. The log file format is not really specified, so changing >>>> it isn't breaking any documented interface, just breaking past practice, >>>> so it might cause some astonishment. Opinions? >>> >>> Regardless of these questions I would as NOT to change logging via >>> syslog. >> >> Timestamping via syslog is done by the syslog daemon, not by the >> application that sends the messages. Don't you understand? Fetchmail can not change logging via syslog. -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.0 (Legolas)) |
|
From: <kj...@o2...> - 2018-10-02 19:24:01
|
"Carlos E. R." <rob...@te...> writes: > On 02/10/2018 20.03, Kamil Jońca wrote: >> "Carlos E. R." <rob...@te...> writes: >> >>> On 01/10/2018 20.09, Kamil Jońca wrote: >>>> Matthias Andree <mat...@gm...> writes: >>>> >>>> [..] >>>>> >>>>> The consideration is whether this could go into 6.4, or would have to >>>>> wait for 7.0. The log file format is not really specified, so changing >>>>> it isn't breaking any documented interface, just breaking past practice, >>>>> so it might cause some astonishment. Opinions? >>>> >>>> Regardless of these questions I would as NOT to change logging via >>>> syslog. >>> >>> Timestamping via syslog is done by the syslog daemon, not by the >>> application that sends the messages. > > Don't you understand? Fetchmail can not change logging via syslog. Please correct me if I am wrong. 1. So far fetchmail produces "log messages" 2. these messages are identical, in case when logging to file/stderr or syslog. 3. syslog adds timestamp to messages received from fetchmail 4. now we want to extend tetchmail log message with timestamps. 5. so syslog will get messages with timestamps, and adds its own timestamp. 6. so, after adding timestamp to fetchmail messages and logging to syslog we will have TWO timestamps in logs. I think this is ugly, and I kindly ask Mathhias, not to change messages when someone pick "set syslog" option. KJ -- http://stopstopnop.pl/stop_stopnop.pl_o_nas.html If you don't know what game you're playing, don't ask what the score is. |
|
From: Carlos E. R. <rob...@te...> - 2018-10-02 22:42:18
Attachments:
signature.asc
|
On 02/10/2018 21.23, Kamil Jońca wrote: > "Carlos E. R." <> writes: > >> On 02/10/2018 20.03, Kamil Jońca wrote: >>> "Carlos E. R." <> writes: >>> >>>> On 01/10/2018 20.09, Kamil Jońca wrote: >>>>> Matthias Andree <mat...@gm...> writes: >>>>> >>>>> [..] >>>>>> >>>>>> The consideration is whether this could go into 6.4, or would have to >>>>>> wait for 7.0. The log file format is not really specified, so changing >>>>>> it isn't breaking any documented interface, just breaking past practice, >>>>>> so it might cause some astonishment. Opinions? >>>>> >>>>> Regardless of these questions I would as NOT to change logging via >>>>> syslog. >>>> >>>> Timestamping via syslog is done by the syslog daemon, not by the >>>> application that sends the messages. >> >> Don't you understand? Fetchmail can not change logging via syslog. > > Please correct me if I am wrong. > 1. So far fetchmail produces "log messages" > 2. these messages are identical, in case when logging to file/stderr or > syslog. > 3. syslog adds timestamp to messages received from fetchmail > 4. now we want to extend tetchmail log message with timestamps. > 5. so syslog will get messages with timestamps, and adds its own > timestamp. > 6. so, after adding timestamp to fetchmail messages and logging to > syslog we will have TWO timestamps in logs. I think this is ugly, and I > kindly ask Mathhias, not to change messages when someone pick "set > syslog" option. Well, I have never seen a programmer make that error in a released version. I'm sure they know. The message is formed, sent to syslog, or some extra entries are added, like a timestapmp, then printed (or written to file). -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.0 (Legolas)) |
|
From: Matthias A. <mat...@gm...> - 2018-10-03 00:17:23
|
Am 03.10.18 um 00:42 schrieb Carlos E. R.: > On 02/10/2018 21.23, Kamil Jońca wrote: >> "Carlos E. R." <> writes: >> >>> On 02/10/2018 20.03, Kamil Jońca wrote: >>>> "Carlos E. R." <> writes: >>>> >>>>> On 01/10/2018 20.09, Kamil Jońca wrote: >>>>>> Matthias Andree <mat...@gm...> writes: >>>>>> >>>>>> [..] >>>>>>> The consideration is whether this could go into 6.4, or would have to >>>>>>> wait for 7.0. The log file format is not really specified, so changing >>>>>>> it isn't breaking any documented interface, just breaking past practice, >>>>>>> so it might cause some astonishment. Opinions? >>>>>> Regardless of these questions I would as NOT to change logging via >>>>>> syslog. >>>>> Timestamping via syslog is done by the syslog daemon, not by the >>>>> application that sends the messages. >>> Don't you understand? Fetchmail can not change logging via syslog. >> Please correct me if I am wrong. >> 1. So far fetchmail produces "log messages" >> 2. these messages are identical, in case when logging to file/stderr or >> syslog. >> 3. syslog adds timestamp to messages received from fetchmail >> 4. now we want to extend tetchmail log message with timestamps. >> 5. so syslog will get messages with timestamps, and adds its own >> timestamp. >> 6. so, after adding timestamp to fetchmail messages and logging to >> syslog we will have TWO timestamps in logs. I think this is ugly, and I >> kindly ask Mathhias, not to change messages when someone pick "set >> syslog" option. > Well, I have never seen a programmer make that error in a released > version. I'm sure they know. The message is formed, sent to syslog, or > some extra entries are added, like a timestapmp, then printed (or > written to file). > > To calm your concerns, 1. I thank Kamil for the hint to make sure we don't bluntly apply timestamping all over the map, to avoid double logging in syslog; 2. I thank Carlos for the trust. Carlos, rest assured that oversights and mistakes do happen, and if the code that formats the message is shared between syslog and fetchmail code, that concern of Kamil's is indeed rather substantiated. And the formatting itself _is_ shared code. O:-) See report.c for the function report(). Hans's patch, however, only changed the file output, not the syslog output, and generated an additional system call, and would not change the log buffer, so syslog output would remain unadorned with the new logfile time stamp. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X