log4perl-devel — Ongoing log4perl development

Re: [log4perl-devel] log4perl appenders with warp_message=0

[Mike S. <m...@pe...>]

On Tue, 13 May 2008, Erskine, Thomas (IT) wrote:

> While warp_message is indeed an appender setting, if you have two
> appenders, one with warp_message=0 and one without, then you'll get
> the mess I got: the one with will work fine and will have access to
> extra args separately, but the one without will mash the args into
> a single message.
>
> Perhaps I ought to have said that, while it's possible to have appenders
> with warp_message=0 and those without in the same setup, it's not
> _useful_ as the result will be a mess.

It's a dilemma: We're hiding probes in the code that should work
for all appenders, and yet some appenders interpret them differently.
Note that the Log4perl configuration is supposed to be independent of
the code. Any person, even those unfamiliar with your code, should be
able to define a Log4perl config file from scratch to run it with.
I'm admitting that this is a stretch (and what L4p does deviates
from the strict Log4j standard) but we've found it useful for database
appenders, for example. I'll be the first to admit, though, that this
isn't a 100% clean solution.

I guess you could find arguments either way, but I feel that mashing
arguments as a default is reasonable and provides predictable results.

> On a completely different topic, while making a Null appender (which
> discards everything), I noticed that it doesn't seem possible to make
> the initializer shut up about it not having a layout.  Even though
> I added in code to the new to cause it to inflict the NoopLayout to
> itself.  It just seemed that I shouldn't have to provide configuration
> for a Null appender, since it isn't doing anything.  Instead, I have
> to say:
>   log4perl.xxx.appender=Null
>   log4perl.xxx.layout=NoopLayout
>   log4perl.xxx.warp_message=0
> I understand that you're just (properly) checking that people haven't
> made a stupid mistake in forgetting the layout.

Exactly, catching stupid errors has precedence over making slick code
short. :)

-- Mike

Mike Schilli
m...@pe...

Re: [log4perl-devel] setting defaults for appenders

[Jonathan S. <sw...@po...>]

> On Fri, 9 May 2008, Jonathan Swartz wrote:
>
>> I've got a growing number of log files for different categories  
>> that I
>> would like to all have the same layout and pattern, e.g.
>
> Hi Jonathan,
>
> this is one of the items on my todo-list: To have an inheritance
> mechanism that lets you say something like
>
>    log4perl.appender.file.inherits_from = \
>        log4perl.appender.file_base
>
> and any property you don't specify in 'file' is inherited from
> file_base. It's not in L4p yet, but hopefully soon! ;)
>

That would be great!

In the meantime, I might try to do this myself by creating my own  
Appender subclass:

     # My::Log4perl::Appender::File defaults to my preferred layout  
and layout.ConversionPattern
     log4perl.appender.file1 = My::Log4perl::Appender::File
     log4perl.appender.file2 = My::Log4perl::Appender::File

Do you have a vague idea of how I might approach this? Or is it too  
diffiicult, and I should just suck up the repetition until the  
inheritance feature? :)

Jon

[log4perl-devel] Understanding the config

[Bill M. <mo...@ha...>]

I inherited an existing Log4perl configuration, and I think I'm
missing some key point.

The config file starts out:

log4perl.rootLogger = INFO, syslogAppender, screenAppender

>From the examples I've seen and the ::Config docs I thought it should
be "log4perl.logger" not "log4perl.rootLogger".

Yet, logging is working as expected.  So, I'm curious about the
difference.

In ::Config it also give these examples:

    log4perl.logger.Bar.Twix = DEBUG, A1


    log4perl.category.Bar.Twix      = WARN, Screen

Why is it "category" instead of "logger"?


Finally, for a config like this in the example, what would
be the perl code equivalent if not using a config file?


    log4perl.logger.Bar.Twix = DEBUG, A1
    log4perl.appender.A1=Log::Log4perl::Appender::File
    log4perl.appender.A1.filename=test.log
    log4perl.appender.A1.mode=append
    log4perl.appender.A1.layout = \
        Log::Log4perl::Layout::PatternLayout
    log4perl.appender.A1.layout.ConversionPattern = %d %m %n


Like this?

    use Log::Log4perl qw(:levels);
    use Log::Log4perl::Appender::File;

    my $appender = Log::Log4perl::Appender::File->new(
      filename  => 'test.log',
      mode      => 'append',
      layout    => Log::Log4perl::Layout::PatternLayout->new( '%d %m %n'),
    );

    my $logger = Log::Log4perl->get_logger( 'Bar::Twix' );

    $logger->level( $DEBUG );
    $logger->add_appender( $appender );






-- 
Bill Moseley
mo...@ha...

Re: [log4perl-devel] log4perl appenders with warp_message=0

[Erskine, T. \(IT\) <Tho...@mo...>]

Hi Mike.

While warp_message is indeed an appender setting, if you have two
appenders, one with warp_message=0 and one without, then you'll get the
mess I got: the one with will work fine and will have access to extra
args separately, but the one without will mash the args into a single
message.  

Perhaps I ought to have said that, while it's possible to have appenders
with warp_message=0 and those without in the same setup, it's not
_useful_ as the result will be a mess.

The hackery with warp_mesage=sub{return $_[0]} has to be applied to the
fancy appender to override its wish to mash all the args together, not
the special appender.  This is the part which just feels wrong.  I made
the Netcool appender, which I acknowledge is special and I don't mind
having to put special setup into it.  What seems wrong is having to
modify the setup for the other appender to make them both play nice.
I'd like to be able to keep the special handling all within the special
appender.  Just me being fussy.

On a completely different topic, while making a Null appender (which
discards everything), I noticed that it doesn't seem possible to make
the initializer shut up about it not having a layout.  Even though I
added in code to the new to cause it to inflict the NoopLayout to
itself.  It just seemed that I shouldn't have to provide configuration
for a Null appender, since it isn't doing anything.  Instead, I have to
say:

  log4perl.xxx.appender=Null
  log4perl.xxx.layout=NoopLayout
  log4perl.xxx.warp_message=0

I understand that you're just (properly) checking that people haven't
made a stupid mistake in forgetting the layout.

Thomas Erskine
Consultant | Technology
2000 Barrington St | Suite 300 | Floor 04
Halifax, NS B3J3K1
Phone: +1 902 442-4709
Tho...@mo...
 

> -----Original Message-----
> From: Mike Schilli [mailto:m...@pe...] 
> Sent: Tuesday, May 13, 2008 12:43 PM
> To: Erskine, Thomas (IT)
> Cc: log...@li...
> Subject: RE: log4perl appenders with warp_message=0
> 
> On Tue, 13 May 2008, Erskine, Thomas (IT) wrote:
> 
> > I do indeed want
> >
> > 	$logger->log($level, $msg, $key)
> >
> > to be dealt with differently depending on the appender it 
> ends up on.
> > That's part of the point of having different appenders; 
> they ought to 
> > be independant.  The problem is that it doesn't seem 
> possible to use 
> > appenders simultaneously which set (and require) warp_message=0 and 
> > ones which don't.
> 
> I don't think that this is true, warp_message is a 
> per-appender setting and can be set individually (and 
> differently) per appender. The restriction is that the you 
> need to use the NoopLayout with the appender that has 
> warp_message set.
> 
> > I've just thought of another way to do this.  If I set 
> > warp_message=sub{return $_[0]} on the file appender, then things 
> > should just work.  I don't like putting code in the log 
> configuration 
> > file, but it's worth a shot.  Just a sec...  Yup.  It works.
> 
> That's exactly right -- and since it's per-appender, it won't 
> affect the other 'fancy' appender. But, the ugly part is that 
> with warp_message turned on, the appender's log() method gets 
> an a reference to an array of message chunks, which the file 
> appender can't deal with.
> 
> > The problem with using the MDC is that this isn't something 
> which is 
> > intended to persist for a while.  The whole point is to be 
> able to set 
> > the Netcool AlertKey on a per-message basis.  Of course I 
> _could_ use 
> > the MDC, but my usage would look like:
> >
> >   Log::Log4perl::MDC->put("netcool-key", $key);
> >   $logger->log($level, $msg);
> >   Log::Log4perl::MDC->put("netcool-key", undef);
> >
> > which isn't exactly handy.
> 
> ... or you define a wrapper around that:
> 
>     http://log4perl.sourceforge.net/d/Log/Log4perl.html#6acb7
> 
> -- Mike
> 
> Mike Schilli
> m...@pe...
> 
> 
> >
> > > -----Original Message-----
> > > From: Mike Schilli [mailto:m...@pe...]
> > > Sent: Tuesday, May 13, 2008 2:26 AM
> > > To: Erskine, Thomas (IT)
> > > Cc: log...@pe...
> > > Subject: Re: log4perl appenders with warp_message=0
> > >
> > > On Fri, 9 May 2008, Erskine, Thomas (IT) wrote:
> > >
> > > > The other logs an event to Netcool (specifying
> > > warp_message=0) and it
> > > > works fine.  It allows you to call log like:
> > > >
> > > >     $logger->log($level, $msg, $key)
> > > >
> > > > so that you can specify a Netcool alert key.
> > > >
> > > > Then comes the problem: I made a configuration file 
> including both 
> > > > appenders.  Now I lose.  The Netcool appender continues to
> > > work nicely.
> > > > However, the file appender mashes all the args together if
> > > I specify
> > > > the key.  Which is the expected behaviour for it, but then
> > > I can't use
> > > > the key feature with netcool.  I could make it work by setting 
> > > > warp_message=0 on the file appender, but to do that I have
> > > to set the
> > > > layout to NoopLayout, so I can't get nicely formatted log
> > > entries there.
> > > >
> > > > It seems as if there is no way to use both types of 
> appender (with 
> > > > warp_message=0 and without) in the same config file.  Or am
> > > I missing
> > > > something?
> > >
> > > Hi Thomas,
> > >
> > > It seems like you want a statement like
> > >
> > >     $logger->log($level, $msg, $key)
> > >
> > > to have two different meanings, depending on which appender the 
> > > message ends up at.
> > >
> > > On the Netcool appender, you want it to use the $key for 
> a special 
> > > purpose, on the file appender, you want it to sweep $key 
> under the 
> > > carpet -- seems inconsistent to me, as it's not clear 
> what another 
> > > random new appender would do with this.
> > >
> > > For cases like this, I would stuff the Netcool key in the MDC:
> > >
> > >     http://log4perl.sourceforge.net/d/Log/Log4perl/MDC.html
> > >
> > > So instead of saying
> > >
> > >     $logger->log($level, $msg, $key);
> > >
> > > you would say
> > >
> > >     Log::Log4perl::MDC->put("netcool-key", $key);
> > >     $logger->log($level, $msg);
> > >
> > > and the Netcool appender would either fetch the the key 
> from the MDC 
> > > in its log() method and do something with it or you could 
> access it 
> > > as %X{netcool-key} in the layout:
> > >
> > >
> > > http://log4perl.sourceforge.net/d/Log/Log4perl/Layout/PatternL
> > > ayout.html
> > >
> > > Would that work for you?
> > >
> > > Can I post this message to the mailing list? Looks like 
> this could 
> > > be useful for other folks as well.
> > >
> > > -- Mike
> > >
> > > Mike Schilli
> > > m...@pe...
> > >
> > --------------------------------------------------------
> >
> > NOTICE: If received in error, please destroy and notify 
> sender. Sender does not intend to waive confidentiality or 
> privilege. Use of this email is prohibited when received in error.
> >
>
--------------------------------------------------------

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.

Re: [log4perl-devel] log4perl appenders with warp_message=0

[Mike S. <m...@pe...>]

On Tue, 13 May 2008, Erskine, Thomas (IT) wrote:

> I do indeed want
>
> 	$logger->log($level, $msg, $key)
>
> to be dealt with differently depending on the appender it ends up on.
> That's part of the point of having different appenders; they ought to be
> independant.  The problem is that it doesn't seem possible to use
> appenders simultaneously which set (and require) warp_message=0 and ones
> which don't.

I don't think that this is true, warp_message is a per-appender setting
and can be set individually (and differently) per appender. The
restriction is that the you need to use the NoopLayout with the appender
that has warp_message set.

> I've just thought of another way to do this.  If I set
> warp_message=sub{return $_[0]} on the file appender, then things
> should just work.  I don't like putting code in the log configuration
> file, but it's worth a shot.  Just a sec...  Yup.  It works.

That's exactly right -- and since it's per-appender, it won't affect the
other 'fancy' appender. But, the ugly part is that with warp_message
turned on, the appender's log() method gets an a reference to an array
of message chunks, which the file appender can't deal with.

> The problem with using the MDC is that this isn't something which is
> intended to persist for a while.  The whole point is to be able to set
> the Netcool AlertKey on a per-message basis.  Of course I _could_ use
> the MDC, but my usage would look like:
>
>   Log::Log4perl::MDC->put("netcool-key", $key);
>   $logger->log($level, $msg);
>   Log::Log4perl::MDC->put("netcool-key", undef);
>
> which isn't exactly handy.

... or you define a wrapper around that:

    http://log4perl.sourceforge.net/d/Log/Log4perl.html#6acb7

-- Mike

Mike Schilli
m...@pe...


>
> > -----Original Message-----
> > From: Mike Schilli [mailto:m...@pe...]
> > Sent: Tuesday, May 13, 2008 2:26 AM
> > To: Erskine, Thomas (IT)
> > Cc: log...@pe...
> > Subject: Re: log4perl appenders with warp_message=0
> >
> > On Fri, 9 May 2008, Erskine, Thomas (IT) wrote:
> >
> > > The other logs an event to Netcool (specifying
> > warp_message=0) and it
> > > works fine.  It allows you to call log like:
> > >
> > >     $logger->log($level, $msg, $key)
> > >
> > > so that you can specify a Netcool alert key.
> > >
> > > Then comes the problem: I made a configuration file including both
> > > appenders.  Now I lose.  The Netcool appender continues to
> > work nicely.
> > > However, the file appender mashes all the args together if
> > I specify
> > > the key.  Which is the expected behaviour for it, but then
> > I can't use
> > > the key feature with netcool.  I could make it work by setting
> > > warp_message=0 on the file appender, but to do that I have
> > to set the
> > > layout to NoopLayout, so I can't get nicely formatted log
> > entries there.
> > >
> > > It seems as if there is no way to use both types of appender (with
> > > warp_message=0 and without) in the same config file.  Or am
> > I missing
> > > something?
> >
> > Hi Thomas,
> >
> > It seems like you want a statement like
> >
> >     $logger->log($level, $msg, $key)
> >
> > to have two different meanings, depending on which appender
> > the message ends up at.
> >
> > On the Netcool appender, you want it to use the $key for a
> > special purpose, on the file appender, you want it to sweep
> > $key under the carpet -- seems inconsistent to me, as it's
> > not clear what another random new appender would do with this.
> >
> > For cases like this, I would stuff the Netcool key in the MDC:
> >
> >     http://log4perl.sourceforge.net/d/Log/Log4perl/MDC.html
> >
> > So instead of saying
> >
> >     $logger->log($level, $msg, $key);
> >
> > you would say
> >
> >     Log::Log4perl::MDC->put("netcool-key", $key);
> >     $logger->log($level, $msg);
> >
> > and the Netcool appender would either fetch the the key from
> > the MDC in its log() method and do something with it or you
> > could access it as %X{netcool-key} in the layout:
> >
> >
> > http://log4perl.sourceforge.net/d/Log/Log4perl/Layout/PatternL
> > ayout.html
> >
> > Would that work for you?
> >
> > Can I post this message to the mailing list? Looks like this
> > could be useful for other folks as well.
> >
> > -- Mike
> >
> > Mike Schilli
> > m...@pe...
> >
> --------------------------------------------------------
>
> NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
>

Re: [log4perl-devel] log4perl appenders with warp_message=0 (fwd)

[Mike S. <m...@pe...>]

On Fri, 9 May 2008, Erskine, Thomas (IT) wrote:

> The other logs an event to Netcool (specifying warp_message=0) and it
> works fine.  It allows you to call log like:
>
>     $logger->log($level, $msg, $key)
>
> so that you can specify a Netcool alert key.
>
> Then comes the problem: I made a configuration file including both
> appenders.  Now I lose.  The Netcool appender continues to work nicely.
> However, the file appender mashes all the args together if I specify the
> key.  Which is the expected behaviour for it, but then I can't use the
> key feature with netcool.  I could make it work by setting
> warp_message=0 on the file appender, but to do that I have to set the
> layout to NoopLayout, so I can't get nicely formatted log entries there.
>
> It seems as if there is no way to use both types of appender (with
> warp_message=0 and without) in the same config file.  Or am I missing
> something?

Hi Thomas,

It seems like you want a statement like

    $logger->log($level, $msg, $key)

to have two different meanings, depending on which appender the message
ends up at.

On the Netcool appender, you want it to use the $key for a special
purpose, on the file appender, you want it to sweep $key under the
carpet -- seems inconsistent to me, as it's not clear what another
random new appender would do with this.

For cases like this, I would stuff the Netcool key in the MDC:

    http://log4perl.sourceforge.net/d/Log/Log4perl/MDC.html

So instead of saying

    $logger->log($level, $msg, $key);

you would say

    Log::Log4perl::MDC->put("netcool-key", $key);
    $logger->log($level, $msg);

and the Netcool appender would either fetch the the key from the
MDC in its log() method and do something with it or you could access it
as %X{netcool-key} in the layout:

    http://log4perl.sourceforge.net/d/Log/Log4perl/Layout/PatternLayout.html

Would that work for you?

Can I post this message to the mailing list? Looks like this could be
useful for other folks as well.

-- Mike

Mike Schilli
m...@pe...

[log4perl-devel] log4perl appenders with warp_message=0 (fwd)

[Mike S. <m...@pe...>]

(forwarded with permission)

Hi Mike.

I've been enjoying Log::Log4perl (using 1.13).  Thank-you.  Until I
decided to write a pair of appenders.

One of them is a fancy file appender and it works nicely.

The other logs an event to Netcool (specifying warp_message=0) and it
works fine.  It allows you to call log like:

    $logger->log($level, $msg, $key)

so that you can specify a Netcool alert key.

Then comes the problem: I made a configuration file including both
appenders.  Now I lose.  The Netcool appender continues to work nicely.
However, the file appender mashes all the args together if I specify the
key.  Which is the expected behaviour for it, but then I can't use the
key feature with netcool.  I could make it work by setting
warp_message=0 on the file appender, but to do that I have to set the
layout to NoopLayout, so I can't get nicely formatted log entries there.

It seems as if there is no way to use both types of appender (with
warp_message=0 and without) in the same config file.  Or am I missing
something?

Thomas Erskine
Consultant | Technology
2000 Barrington St | Suite 300 | Floor 04
Halifax, NS B3J3K1
Phone: +1 902 442-4709
Tho...@mo...
--------------------------------------------------------

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.

Re: [log4perl-devel] setting defaults for appenders

[Bill M. <mo...@ha...>]

On Mon, May 12, 2008 at 02:21:23PM -0700, Jonathan Swartz wrote:
> I've got a growing number of log files for different categories that I  
> would like to all have the same layout and pattern, e.g.

You can use "Variable Substitution" as described in the man page,
but that still seems awkward.


-- 
Bill Moseley
mo...@ha...

Re: [log4perl-devel] Watchdog reporting

[Mike S. <m...@pe...>]

On Wed, 7 May 2008, Bill Moseley wrote:

> What I'm after is to have an email sent *as soon* as an error is
> detected.  But, then to collect any more errors for a few minutes
> before sending another email again to keep from flooding with email.
> (I also want to report to Nagios, but that's later.)
> Log::Dispatch::Email has a "buffered" option but I'm not sure that
> fits with the above.  Any existing solutions for the above?

The way this works in the buffered email appender is actually quite
different: It collects messages *until* a threshold is reached, then it
sends an email:

  http://log4perl.sourceforge.net/d/Log/Log4perl/FAQ.html#a0f6e

> What I'm attempting to do instead is have a separate File appender
> that only logs ERROR and above to a file, then watch that file.

If you're running a single process (and won't need to keep state on
restarts) there's no need to do this in a file appender. You could
define your own appender which maintains a (memory) buffer.

> So I have these settings for my appender:
>
>     .recreate = 1
>     .recreate_check_interval = 0
>
> But, that is not working -- the move is not detected.

Aha, that's a bug -- I've fixed it in 1.16 in CVS, to be released soon
-- thanks!

> Finally, another question.  I've got three appenders.  One for syslog,
> one for stderr, and one to a file.
>
> Is it possible to set a pattern that will apply to all?  Or do I have
> to specify a pattern for each appender?

Variable substitution should help with this:

   http://log4perl.sourceforge.net/d/Log/Log4perl.html#bd632

-- Mike

Mike Schilli
m...@pe...

Re: [log4perl-devel] setting defaults for appenders

[Mike S. <m...@pe...>]

On Fri, 9 May 2008, Jonathan Swartz wrote:

> I've got a growing number of log files for different categories that I
> would like to all have the same layout and pattern, e.g.

Hi Jonathan,

this is one of the items on my todo-list: To have an inheritance
mechanism that lets you say something like

    log4perl.appender.file.inherits_from = \
        log4perl.appender.file_base

and any property you don't specify in 'file' is inherited from
file_base. It's not in L4p yet, but hopefully soon! ;)

-- Mike

Mike Schilli
m...@pe...

>
>     log4perl.appender.file1                            =
> Log::Log4perl::Appender::File
>     log4perl.appender.file1.filename                   = /path/to/
> logfile/1
>     log4perl.appender.file1.layout                     = PatternLayout
>     log4perl.appender.file1.layout.ConversionPattern   = %d{dd/MMM/
    > yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n
    >
    >     log4perl.appender.file2                            =
    > Log::Log4perl::Appender::File
    >     log4perl.appender.file2.filename                   = /path/to/
    > logfile/2
    >     log4perl.appender.file2.layout                     = PatternLayout
    >     log4perl.appender.file2.layout.ConversionPattern   = %d{dd/MMM/
	> yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n
	>
	>     log4perl.appender.file3                            =
	> Log::Log4perl::Appender::File
	>     log4perl.appender.file3.filename                   = /path/to/
	> logfile/3
	>     log4perl.appender.file3.layout                     = PatternLayout
	>     log4perl.appender.file3.layout.ConversionPattern   = %d{dd/MMM/
	    > yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n
	    >
	    > In the spirit of DRY, I'd like to eliminate those repeated layout and
	    > layout.ConversionPattern lines.
	    >
	    > What's the best way to do this? I'm happy to create whatever
	    > subclasses necessary, but looking at the Appender, Appender::File,
	    > Layout, and Layout::PatternLayout classes, it wasn't clear to me what
	    > to subclass.
	    >
	    > Thanks
	    > Jon
	    >
	    >
	    > -------------------------------------------------------------------------
	    > This SF.net email is sponsored by: Microsoft
	    > Defy all challenges. Microsoft(R) Visual Studio 2008.
	    > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
	    > _______________________________________________
	    > log4perl-devel mailing list
	    > log...@li...
	    > https://lists.sourceforge.net/lists/listinfo/log4perl-devel
	    >

Re: [log4perl-devel] setting defaults for appenders

[Bill M. <mo...@ha...>]

On Mon, May 12, 2008 at 02:21:23PM -0700, Jonathan Swartz wrote:
> I've got a growing number of log files for different categories that I  
> would like to all have the same layout and pattern, e.g.

You can use "Variable Substitution" as described in the man page,
but that still seems awkward.


-- 
Bill Moseley
mo...@ha...

[log4perl-devel] setting defaults for appenders

[Jonathan S. <sw...@po...>]

I've got a growing number of log files for different categories that I  
would like to all have the same layout and pattern, e.g.

   log4perl.appender.file1                            =  
Log::Log4perl::Appender::File
   log4perl.appender.file1.filename                   = /path/to/ 
logfile/1
   log4perl.appender.file1.layout                     = PatternLayout
   log4perl.appender.file1.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

   log4perl.appender.file2                            =  
Log::Log4perl::Appender::File
   log4perl.appender.file2.filename                   = /path/to/ 
logfile/2
   log4perl.appender.file2.layout                     = PatternLayout
   log4perl.appender.file2.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

   log4perl.appender.file3                            =  
Log::Log4perl::Appender::File
   log4perl.appender.file3.filename                   = /path/to/ 
logfile/3
   log4perl.appender.file3.layout                     = PatternLayout
   log4perl.appender.file3.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

In the spirit of DRY, I'd like to eliminate those repeated layout and  
layout.ConversionPattern lines.

What's the best way to do this? I'm happy to create whatever  
subclasses necessary, but looking at the Appender, Appender::File,  
Layout, and Layout::PatternLayout classes, it wasn't clear to me what  
to subclass.

Thanks
Jon

[log4perl-devel] setting defaults for appenders

[Jonathan S. <sw...@po...>]

I've got a growing number of log files for different categories that I  
would like to all have the same layout and pattern, e.g.

    log4perl.appender.file1                            =  
Log::Log4perl::Appender::File
    log4perl.appender.file1.filename                   = /path/to/ 
logfile/1
    log4perl.appender.file1.layout                     = PatternLayout
    log4perl.appender.file1.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

    log4perl.appender.file2                            =  
Log::Log4perl::Appender::File
    log4perl.appender.file2.filename                   = /path/to/ 
logfile/2
    log4perl.appender.file2.layout                     = PatternLayout
    log4perl.appender.file2.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

    log4perl.appender.file3                            =  
Log::Log4perl::Appender::File
    log4perl.appender.file3.filename                   = /path/to/ 
logfile/3
    log4perl.appender.file3.layout                     = PatternLayout
    log4perl.appender.file3.layout.ConversionPattern   = %d{dd/MMM/ 
yyyy:HH:mm:ss.SS} [%p] %c - %m - %F:%L %P-%n

In the spirit of DRY, I'd like to eliminate those repeated layout and  
layout.ConversionPattern lines.

What's the best way to do this? I'm happy to create whatever  
subclasses necessary, but looking at the Appender, Appender::File,  
Layout, and Layout::PatternLayout classes, it wasn't clear to me what  
to subclass.

Thanks
Jon

[log4perl-devel] Watchdog reporting

[Bill M. <mo...@ha...>]

Hello all,

I'd like to check if there's already a solution to my need, since I
assume it probably reasonably common need.

And second, I'd like to confirm a bug before submitting a ticket.


What I'm after is to have an email sent *as soon* as an error is
detected.  But, then to collect any more errors for a few minutes
before sending another email again to keep from flooding with email.

(I also want to report to Nagios, but that's later.)


Log::Dispatch::Email has a "buffered" option but I'm not sure that
fits with the above.

Any existing solutions for the above?


What I'm attempting to do instead is have a separate File appender
that only logs ERROR and above to a file, then watch that file.

The idea is to have a cron job run often and look for the file to not
be empty.  If found not to be empty do an atomic rename, sleep a few
seconds, then email the file.

Any new log message will be written to a new file.

Then when the cron script runs again look for that *new* file name and
exit if it's found and less than X number of minutes old.  Once it is
X minutes old then look again for a non-empty log file and repeat the
process.



Now, for that to work it's important that the File appender notices
that the file has changed once moved.  The File appender docs say for
"recreate_check_interval":

    This obviously means that the appender will continue writing to a
    moved file until the next check occurs, in the worst case this
    will happen recreate_check_interval seconds after the file has
    been moved or deleted. If this is undesirable, setting
    recreate_check_interval to 0 will have the appender check
    the file with every call to log().

So I have these settings for my appender:

    .recreate = 1
    .recreate_check_interval = 0

But, that is not working -- the move is not detected.

I suspect it's this code in A::File that is checking if
recreate_check_interval is true or not, not if it's defined:

    if($self->{recreate}) {
        $self->{watcher} = Log::Log4perl::Config::Watch->new(
            file           => $self->{filename},
            ($self->{recreate_check_interval} ?
              (check_interval => $self->{recreate_check_interval}) : ()),
            ($self->{recreate_check_signal} ?
              (signal => $self->{recreate_check_signal}) : ()),
        );
    }

and thus Log::Log4perl::Config::Watch falls back to the default of 30
seconds.  Am I configuring this incorrectly or is this a bug?


As a workaround I can set recreate_check_interval to 1, then just make
sure I sleep to catch anything else that might get written to the
renamed file.



Finally, another question.  I've got three appenders.  One for syslog,
one for stderr, and one to a file.

Is it possible to set a pattern that will apply to all?  Or do I have
to specify a pattern for each appender?


log4perl.rootLogger = INFO, syslogAppender, screenAppender, errorNotify

## syslog ##
log4perl.appender.syslogAppender = Log::Dispatch::Syslog
log4perl.appender.syslogAppender.utf8     = 1
log4perl.appender.syslogAppender.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.syslogAppender.layout.ConversionPattern \
    = [%p] "%m" pid:%P [%X{ip}] "%X{uri}"


## screen ##
log4perl.appender.screenAppender = Log::Log4perl::Appender::Screen
log4perl.appender.screenAppender.utf8     = 1
log4perl.appender.screenAppender.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.screenAppender.layout.ConversionPattern \
    = %d{MMM dd HH:mm:ss} [%p] "%m" pid:%P [%X{ip}] "%X{uri}"%n



## File -- Just ERRORs ##
log4perl.appender.screenAppender.layout.ConversionPattern \
    = %d{MMM dd HH:mm:ss} [%p] "%m" pid:%P [%X{ip}] "%X{uri}"%n


log4perl.appender.errorNotify = Log::Log4perl::Appender::File
log4perl.appender.errorNotify.filename = error.log
log4perl.appender.errorNotify.mode = append

# We want this to recreate the file if it's moved
log4perl.appender.errorNotify.recreate = 1
## This does not work
log4perl.appender.errorNotify.recreate_check_interval = 0

# And only log errors
log4perl.appender.errorNotify.Threshold  = ERROR

log4perl.appender.errorNotify.utf8     = 1
log4perl.appender.errorNotify.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.errorNotify.layout.ConversionPattern \
    = %d{MMM dd HH:mm:ss} [%p] "%m" pid:%P [%X{ip}] "%X{uri}"%n







I



-- 
Bill Moseley
mo...@ha...

Re: [log4perl-devel] OLG compliant logs

[Merijn B. <me...@il...>]

Quoting Mike Schilli (m...@pe...):
> 
> By the way, is OLF a common format? Wikipedia doesn't think so:
> 
>     http://en.wikipedia.org/wiki/OLF
> 
Google neither, and this blogger is not to happy with it:

http://raffy.ch/blog/2007/09/14/open-log-format-what-a-great-standard-not/

Cheers
-- 
Merijn Broeren | We take risks, we know we take them. Therefore, when things
               | come out against us, we have no cause for complaint.
               | - Scott, last journal entry, march 1912

Re: [log4perl-devel] OLG compliant logs

[Kevin M. G. <cp...@go...>]

Mike Schilli wrote:

> By the way, is OLF a common format? Wikipedia doesn't think so:
> 
>     http://en.wikipedia.org/wiki/OLF

A critique of OLF:
http://raffy.ch/blog/2007/09/14/open-log-format-what-a-great-standard-not/

If any of his critiques are true, I'm sure not very impressed by it
(tab-delimited?).  I wouldn't know though, you have to register to see a
copy the standard.

http://www.openlogformat.org/download/olf_download.shtml

Re: [log4perl-devel] OLG compliant logs

[Mike S. <m...@pe...>]

On Tue, 29 Apr 2008, Denis Lambret wrote:

> My question today is quite simple : What about OLF support ?
>
> I would like to use log4perl in order to generate logs compliant with Open
> Log Format definition.
>
> Well the OLF specifications gives the following description for
> a compliant OLF header : #Software: eIQ Open Log Format (OLF)
> #Version: 1.1 #Date: 02-18-2007 12:14:25 300 #Fields: date time
> gmt-offset internalIP externalIP virtualdevice

There's two parts to that:

1) The layout of each individual message logged can be set in Log4perl
   with the PatternLayout layout:

    http://log4perl.com/d/Log/Log4perl.html#21b9f

   If you have a lot of semi-static entries that aren't changing by
   message, look into using MDC:

   http://log4perl.com/d/Log/Log4perl.html#4b5b3

2) If you want Log4perl to write a header at the beginning of each newly
   created logfile, you need to write your own file appender that does
   that. This seems to be a reasonably generic requirement, though, so
   let me look into how this could be added to
   Log::Log4perl::Appender::File.

By the way, is OLF a common format? Wikipedia doesn't think so:

    http://en.wikipedia.org/wiki/OLF

-- Mike

Mike Schilli
m...@pe...


>
> device-id type subtype priority prioritycode direction
>
> username eventcode eventcategory protocol rule status count action
>
> sent-bytes recvd-bytes src-info dst-info vpn-info virus-info
>
> attack-info webfilter-info spam-info config-info message
>
> Custom-data Nativelog
>
>
>
> The #Version line gives the version of OLF, which should always be 1.1.
> NOTE: Software that
>
> processes OLF should check this line, and reject the file if the version is
> not one it understands.
>
>
>
> The #Date line gives the date and time the log file was started and the time
> zone (in GMT minutes).
>
>
>
> The #Fields line lists the fields of each line, as a reminder to the reader.
> NOTE: The fields may
>
>
>
> How can I manage these requirements with log4perl?
>
> If not available yet, did you plan to support this format in further version
> of your lib?
>
>
>
> Thank you by advance for your help,
    >
    > And one more time, thank you for this very handy and flexible lib.
    >
    >
    >
    > Denis LAMBRET
    >
    >
    >
    > Den...@gm...
    >
    >

[log4perl-devel] OLG compliant logs

[Denis L. <den...@gm...>]

Hi there,

 

Well, I was just beginning playing around with log4perl one week ago.

I used to work with my own perl native libs for logging. It was not so
efficient than yours but brings a solution to most of my developments.

I found the log4perl very handy and convenient for my dev that I can't keep
playing with it now!

 

My question today is quite simple : What about OLF support ?

 

I would like to use log4perl in order to generate logs compliant with Open
Log Format definition. 

Currently, I use a dedicated layout in order to format my log records. It
works fine but I have only one thing I can not solve for the moment.

Who can I generate an OLF compliant header for my log ?

 

Well the OLF specifications gives the following description for a compliant
OLF header :

 

#Software: eIQ Open Log Format (OLF)

#Version: 1.1

#Date: 02-18-2007 12:14:25 300

#Fields: date time gmt-offset internalIP externalIP virtualdevice

device-id type subtype priority prioritycode direction

username eventcode eventcategory protocol rule status count action

sent-bytes recvd-bytes src-info dst-info vpn-info virus-info

attack-info webfilter-info spam-info config-info message

Custom-data Nativelog

 

The #Version line gives the version of OLF, which should always be 1.1.
NOTE: Software that

processes OLF should check this line, and reject the file if the version is
not one it understands.

 

The #Date line gives the date and time the log file was started and the time
zone (in GMT minutes).

 

The #Fields line lists the fields of each line, as a reminder to the reader.
NOTE: The fields may

 

How can I manage these requirements with log4perl?

If not available yet, did you plan to support this format in further version
of your lib?

 

Thank you by advance for your help,

And one more time, thank you for this very handy and flexible lib.

 

Denis LAMBRET

 

Den...@gm...

Re: [log4perl-devel] Is it safe to use log4perl under Apache (multi worker mode)

[Mike S. <m...@pe...>]

On Mon, 10 Mar 2008, howard chen wrote:

> Since I would like to log message to text file, so I want to ask:
>
> Is it safe to use log4perl under Apache (multi worker mode), without
> file locking problem? If not, any recommendations for file logging
> under Apache.

Logging from different processes to the same file requires some
precautions, all listed in this FAQ:

   http://log4perl.sourceforge.net/d/Log/Log4perl/FAQ.html#23804

-- Mike

Mike Schilli
m...@pe...

[log4perl-devel] Is it safe to use log4perl under Apache (multi worker mode)

[howard c. <how...@gm...>]

Hi

Since I would like to log message to text file, so I want to ask:

Is it safe to use log4perl under Apache (multi worker mode), without
file locking problem?
If not, any recommendations for file logging under Apache.


Thanks.
Howard

Re: [log4perl-devel] Small issue with Log::Log4perl 1.15

[Mike S. <m...@pe...>]

On Thu, 21 Feb 2008, Emil Janev wrote:

> The problem certainly can be fixed with providing a default value of
> this parameter of an empty string,
> if an 'undef' value gets trough.

Passing on the arg list from the wrapper to the Log4perl log call is
probably the easiest solution:

    sub wrapper_get_logger {
        ...
        my $logger = get_logger(@_);
        ...
    }

> My question is also if there is any planned future change, where undef
> case will be treated in the same way as the one with an empty string,
> or an empty array.

It's not really obvious to me what 'undef' should stand for -- maybe
we'll leave it undefined until someone has a smart idea.

> p.s. As a coincidence, these days reading the Linux Pro Magazine on the
> subway ride to/from work,
>         I came upon the 2007-12 article "I think in Perl' :)

It's even available online if anybody is interested what I'm doing when
I'm not working on Log4perl :)

    http://w3.linux-magazine.com/issue/85/Perlmeister_Michael_Schilli.pdf

-- Mike

Mike Schilli
m...@pe...

> >> my $logger = get_logger(undef);
> >>
> >
> > That's an interesting case. While
> >
> >       # get a logger for the current package
> >     get_logger()
> >
> > and
> >
> >       # get the root logger
> >     get_logger("")
> >
> > are defined cases, using 'undef' as in
> >
> >     get_logger( undef )
> >
> > isn't defined. There was a change in 1.15 that now brings this undefined
> > behavior to light (unfortunately for you :).
> >
> > If you want to use a variable to toggle between "" and the empty list,
> > use an array:
> >
> >     my @array = ();
> >     my $logger = get_logger(@array);
> >
> >     # vs.
> >
> >     my @array = ("");
> >     my $logger = get_logger(@array);
> >
> > Does that work for you?
> >
> > -- Mike
> >
> > Mike Schilli
> > m...@pe...
> >
>
>
> --
> Emil Janev
>
>

Re: [log4perl-devel] Small issue with Log::Log4perl 1.15

[Mike S. <m...@pe...>]

On Wed, 20 Feb 2008, Emil Janev wrote:

> my $logger = get_logger(undef);

That's an interesting case. While

      # get a logger for the current package
    get_logger()

and

      # get the root logger
    get_logger("")

are defined cases, using 'undef' as in

    get_logger( undef )

isn't defined. There was a change in 1.15 that now brings this undefined
behavior to light (unfortunately for you :).

If you want to use a variable to toggle between "" and the empty list,
use an array:

    my @array = ();
    my $logger = get_logger(@array);

    # vs.

    my @array = ("");
    my $logger = get_logger(@array);

Does that work for you?

-- Mike

Mike Schilli
m...@pe...

[log4perl-devel] Small issue with Log::Log4perl 1.15

[Emil J. <ej...@gm...>]

Hi,

With the latest version 1.15, in a specific scenario, warning messages
are printed on the standard error.
It was not happening with 1.14 and before.

Here is a test case:

------------------------------------------------------------------------------------
$ cat tst_log.pl
#!/usr/bin/perl

use strict;
use Log::Log4perl qw(get_logger :levels);

my $config_file = 'logger.conf';

Log::Log4perl->init($config_file);

my $logger = get_logger(undef);

$logger->info('info_message_1');

exit;
$ tst_log.pl
Use of uninitialized value in string eq at
/usr/local/share/perl/5.8.8/Log/Log4perl.pm line 360.
Use of uninitialized value in string eq at
/usr/local/share/perl/5.8.8/Log/Log4perl/Logger.pm line 560.
$ cat t.log
2008/02/20 16:04:11 INFO> tst_log.pl:12 main:: - info_message_1
$
------------------------------------------------------------------------------------

Test script and the configuration file are attached.

Aside the messages on the STDERR, functionality is not affected.

Thanks in advance,
Emil

-- 
Emil Janev

Re: [log4perl-devel] Can't call method "log" on an undefined value...Appender.pm line 189 during global destruction.

[Mike S. <m...@pe...>]

On Thu, 14 Feb 2008, Darren Ball wrote:

> I am getting this message consistently.
>
> (in cleanup) Can't call method "log" on an undefined value at
> /usr/lib/perl5/site_perl/5.8/Log/Log4perl/Appender.pm line 189 during global
> destruction.

Hmm, to help me track this down, can you provide a condensed code
example that I can reproduce the error with?

-- Mike

Mike Schilli
m...@pe...

>
>
> I have a series of modules that are descendants of other packages.
> Scripts using the objects are using log4perl, and the packages also use
> log4perl.
>
> Everything seems to work - I initiate the logger in the 'script' that uses
> the libraries,
> and when the script ends, it calls a library function (i.e. pass, fail,
> error).
>
> These methods call the exit function, which in turn invoke the DESTROY
> routine for each object.
> It seems as though Log4perl is being destroyed prior to other objects being
> cleaned up (which use the logger in their DESTROY routines)
> or so I think.
>
> Can anyone help me try to figure out how to stop this, and ensure all
> logging occurs right to the end of the script.
>
> Any help would be appreciated.
>
> I instantiated the logger with a simple screen appender.
>
> my $conf = qq |
>     log4perl.logger                      = $acceptedLevel, ScreenApp
>     log4perl.appender.ScreenApp          = Log::Log4perl::Appender::Screen
>     log4perl.appender.ScreenApp.stderr   = 0
>     log4perl.appender.ScreenApp.layout   = PatternLayout
>     log4perl.appender.ScreenApp.layout.ConversionPattern = %d %-5p %-4L %m%n
>     |;
>
> But I always get :
> (in cleanup) Can't call method "log" on an undefined value at
> /usr/lib/perl5/site_perl/5.8/Log/Log4perl/Appender.pm line 189 during global
> destruction.
>
> at the end of the script.
>
> Somethings I am missing log statements as well.
>

[log4perl-devel] Can't call method "log" on an undefined value...Appender.pm line 189 during global destruction.

[Darren B. <bal...@gm...>]

I am getting this message consistently.

(in cleanup) Can't call method "log" on an undefined value at
/usr/lib/perl5/site_perl/5.8/Log/Log4perl/Appender.pm line 189 during global
destruction.


I have a series of modules that are descendants of other packages.
Scripts using the objects are using log4perl, and the packages also use
log4perl.

Everything seems to work - I initiate the logger in the 'script' that uses
the libraries,
and when the script ends, it calls a library function (i.e. pass, fail,
error).

These methods call the exit function, which in turn invoke the DESTROY
routine for each object.
It seems as though Log4perl is being destroyed prior to other objects being
cleaned up (which use the logger in their DESTROY routines)
or so I think.

Can anyone help me try to figure out how to stop this, and ensure all
logging occurs right to the end of the script.

Any help would be appreciated.

I instantiated the logger with a simple screen appender.

my $conf = qq |
    log4perl.logger                      = $acceptedLevel, ScreenApp
    log4perl.appender.ScreenApp          = Log::Log4perl::Appender::Screen
    log4perl.appender.ScreenApp.stderr   = 0
    log4perl.appender.ScreenApp.layout   = PatternLayout
    log4perl.appender.ScreenApp.layout.ConversionPattern = %d %-5p %-4L %m%n
    |;

But I always get :
(in cleanup) Can't call method "log" on an undefined value at
/usr/lib/perl5/site_perl/5.8/Log/Log4perl/Appender.pm line 189 during global
destruction.

at the end of the script.

Somethings I am missing log statements as well.