log4perl-devel — Ongoing log4perl development

Re: [log4perl-devel] Message Duplication

[Kevin G. <ke...@go...>]

Manju:

We had a discussion about this topic back around the end of November 
(threads are "Same appender attached to different loggers" and "one 
appender, many categories").  The upshot was that we fixed the behavior 
of log4perl to match that of log4j, which does what you observed: if you 
have an appender attached to category "tree" and also attached to 
"tree.branch.leaf", messages sent to "tree.branch.leaf" will show up in 
that appender twice.

However, forseeing objections like yours, we did log4j one better and 
added an as-yet undocumented feature, you can set

	log4perl.oneMessagePerAppender = 1

in your config file (for log4perl v 0.27).

Unless there are any objections, I think we should consider this a 
motion to add oneMessagePerAppender as a bona-fide feature.




Manju Ramanathpura wrote:
> Hello there,
> 
> I have attatched a sample configuration file and a test perl script 
> along with the email.
> 
> Could you please explain me why is the logger logging the messages twice??
> 
> This example is very similar to the one you have explained in the 
> documentation and I am puzzled by the fact that logger is duplicating 
> the log messages coming from the package Ticket.pm. Although overriding 
> the parent logger setting seems to be working, I am not able to 
> figureout why is it duplicating??
> 
> Am I using it wrong?
> 
> Thanks,
> 
> -Manju
> 
> 
> ------------------------------------------------------------------------
> 
>     log4perl.logger=FATAL, Screen
>     log4perl.logger.CCBU.Patch=ERROR, Log
>     log4perl.logger.CCBU.Patch.Ticket=WARN,Log
> 
> 
> 
>     log4perl.appender.Screen=Log::Dispatch::Screen
>     log4perl.appender.Screen.stderr=0
>     log4perl.appender.Screen.Threshold=FATAL
>     log4perl.appender.Screen.layout=Log::Log4perl::Layout::PatternLayout
>     log4perl.appender.Screen.layout.ConversionPattern=%d %p> %F{1}:%L %M - %m%n
>     
>     log4perl.appender.Log=Log::Dispatch::File
>     log4perl.appender.Log.filename=test.log
>     log4perl.appender.Log.mode=append
>     log4perl.appender.Log.layout=Log::Log4perl::Layout::PatternLayout
>     log4perl.appender.Log.layout.ConversionPattern=%d %p> %F{1}:%L %M - %m%n
>     
> 
> 
> ------------------------------------------------------------------------
> 
> #!perl-w
> 
> use 5.006001;
> use strict;
> use warnings;
> use Log::Log4perl qw(get_logger :levels);
> use CCBU::Patch::Ticket
> 
> 
> 
> Log::Log4perl->init("logsetup.conf");
> 
> my $logger = get_logger("CCBU::Patch");
> 
> 
> my $food = what_food_sub("Sushi");
> 
> $logger->info("Food in main is : $food");
> $logger->info("And the conume is ".consume($food));


-- 
Happy Trails . . .

Kevin M. Goess
(and Anne and Frank)
904 Carmel Ave.
Albany, CA  94706
(510) 525-5217

[log4perl-devel] Message Duplication

[Manju R. <mra...@ci...>]

Hello there,

I have attatched a sample configuration file and a test perl script along 
with the email.

Could you please explain me why is the logger logging the messages twice??

This example is very similar to the one you have explained in the 
documentation and I am puzzled by the fact that logger is duplicating the 
log messages coming from the package Ticket.pm. Although overriding the 
parent logger setting seems to be working, I am not able to figureout why 
is it duplicating??

Am I using it wrong?

Thanks,

-Manju

Re: [log4perl-devel] Suppressing warnings in Log::Log4perl::Layout::PatternLayout

[<msc...@ao...>]

In a message dated 1/3/2003 7:28:17 PM Eastern Standard Time, Peter Scott <Pe...@PS...> writes:

>I have an application which uses $SIG{__DIE__} to catch exceptions and 
>route them to appenders.  I often find I get a stack of messages which 
>contains:

I've got two things for you to try:

(1) Here's a new test version of Log::Log4perl, which will print "[undef]" if any expected return value of caller() is not defined, instead of the warning you're getting:
    http://log4perl.sourceforge.net/Log-Log4perl-0.28dev2.tar.gz

(2) The fact that you're getting undefined values seems to indicate that your Log::Log4perl::caller_depth variable is off. Did you modify it? If not, I would really like to track this down. Can you provide a short code sample with a signal handler and a function that reproduces the problem? Also, please make sure to read the FAQ dealing with a similar issue:

    http://log4perl.sourceforge.net/releases/Log-Log4perl/docs/html/Log/Log4perl/FAQ.html#how_can_i_make_sure_my_application_logs_a_message_when_it_dies_unexpectedly

Towards the end, it shows how to offset Log::Log4perl's caller stack to get the right subroutines/lines numbers in the pattern layout. 

Thanks for your help!

-- 
-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

[log4perl-devel] Suppressing warnings in Log::Log4perl::Layout::PatternLayout

[Peter S. <Pe...@PS...>]

I applaud the use of "use warnings" in modules but I have a problem 
with warnings emerging from Log::Log4perl::Layout::PatternLayout.  I 
have an application which uses $SIG{__DIE__} to catch exceptions and 
route them to appenders.  I often find I get a stack of messages which 
contains:

Use of uninitialized value in concatenation (.) or string at 
/opt/perl/lib/site_perl/5.6.1/Log/Log4perl/Layout/PatternLayout.pm line 194.

And since I have $SIG{__WARN__} set to do the same thing as for 
exceptions, I tend to get cascading.  (Yes, I do have to catch and 
report warnings.)

Not sure whether $^W takes precedence over the "use warnings" in 
PatternLayout.pm so maybe I could undef it around the code that's 
producing this uninit warning, but that's pretty kludgey.  In my 
version of that module the line in question is

             # For the name of the subroutine the logger was triggered,
             # we need to go one more level up
             $subroutine = (caller($caller_level+1))[3];
             $subroutine = "main::" unless $subroutine;
             $info{M} = $subroutine;
             $info{l} = "$subroutine $filename ($line)";    # <--- ***

Odd that one of those things would be undefined but this may be 
happening during global destruction.

Absent patching PatternLayout.pm to ensure there are no undef values in 
that statement, any other suggestions?
--
Peter Scott
Pacific Systems Design Technologies

[log4perl-devel] Log::Log4perl Recipe of the Week (#10)

[<log...@pe...>]

Welcome to the Log::Log4perl recipe of the week. Today:

============================================================
Log::Log4perl Recipe of the Week (#10):

What's the easiest way to turn off all logging, even with a
lengthy Log4perl configuration file?
============================================================

In addition to category-based levels and appender
thresholds, Log::Log4perl supports system-wide logging
thresholds. This is the minimum level the system will
require of any logging events in order for them to make it
through to any configured appenders.

For example, putting the line

    log4perl.threshold = ERROR

anywhere in your configuration file will limit any output to
any appender to events with priority of ERROR or higher
(ERROR or FATAL that is).

However, in order to suppress all logging entirely, you need
to use a priority that's higher than FATAL: It is simply
called "OFF", and it is never used by any logger. By
definition, it is higher than the highest defined logger
level.

Therefore, if you keep the line

    log4perl.threshold = OFF

somewhere in your Log::Log4perl configuration, the system
will be quiet as a graveyard. If you deactivate the line
(e.g. by commenting it out), the system will, upon config
reload, snap back to normal operation, providing logging
messages according to the rest of the configuration file
again.


Have fun! Until next week.

-- Mike

###################################
# Mike Schilli                    #
# log...@pe...        #
# http://perlmeister.com          #
# http://log4perl.sourceforge.net #
###################################

Re: [log4perl-devel] env-vars in log4perl config file

[Jim C. <jc...@di...>]

msc...@ao... wrote:

>In a message dated 12/27/2002 6:02:45 PM Eastern Standard Time, Kevin Goess <ke...@go...> writes:
>
>  
>
>>Ha! That's in our CVS code right now, just hasn't been released yet.
>>    
>>
>
>... and the development release (containing the new functionality) is available for download here:
> http://log4perl.sourceforge.net/releases/Log-Log4perl-0.28dev.tar.gz
>
>  
>
thanks guys - it just works!


wrt to my use of AUTOLOAD to give Auto-Categorization,
is that something for which you might accept a feature patch ?

I was thinking of a couple of options on how to do it.

use Log::Log4perl qw(:autocategorize);
use Log::Log4perl::AutoCat;

a whole new class seems overkill to add one function, but you may have 
philosophical objections to an
import-tag that causes sub AUTOLOAD {} to be compiled.

anyway, thx again

Re: [log4perl-devel] env-vars in log4perl config file

[<msc...@ao...>]

In a message dated 12/27/2002 6:02:45 PM Eastern Standard Time, Kevin Goess <ke...@go...> writes:

>Ha! That's in our CVS code right now, just hasn't been released yet.

... and the development release (containing the new functionality) is available for download here:
 http://log4perl.sourceforge.net/releases/Log-Log4perl-0.28dev.tar.gz

-- 
-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

Re: [log4perl-devel] env-vars in log4perl config file

[Kevin G. <ke...@go...>]

Jim Cromie wrote:
 > nice module - easy to use and powerful / flexible.

Thanks!


 > I tried this - hoping the layoutpattern tricks would work here, but no
 > joy -
 >        log4j.appender.LOGFILE.filename= sub {return
 > "$ENV{APP_ROOT}/logs/rootlog"}

Ha! That's in our CVS code right now, just hasn't been released yet. 
You could wait a couple weeks for the release, or you can read in the 
config file, interpolate the variable yourself, and give the resulting 
string to init() as a reference.


 > Ive written a wrapper (myLogger) which uses AUTOLOAD to delegate 
calls like
 > myLogger->debug(@args) to Log::Log4perl.

Fun stuff, that.  Someday we might implement filters, which might do 
some of what you're doing.



-- 
Happy Trails . . .

Kevin M. Goess

[log4perl-devel] env-vars in log4perl config file

[Jim C. <jc...@di...>]

Folks,

nice module - easy to use and powerful / flexible.


The only problem Ive encountered is trying to get something like this to 
work -
        log4j.appender.LOGFILE.filename=$ENV{APP_ROOT}/logs/rootlog

The code is installed into different places on several machines, and its 
a bit of a hassle to
fix the config file accordingly as I propagate code off my development box.

I tried this - hoping the layoutpattern tricks would work here, but no joy -
        log4j.appender.LOGFILE.filename= sub {return 
"$ENV{APP_ROOT}/logs/rootlog"}



Also, for what its worth, Ive disregarded your warnings (sort of): 

    However, be careful, don't go overboard: if you're devel­
       oping a system in object-oriented style, using the class
       hierarchy is usually your best choice. Think about the
       people taking over your code one day: The class hierarchy
       is probably what they know right up front, so it's easy
       for them to tune the logging to their needs.

Ive written a wrapper (myLogger) which uses AUTOLOAD to delegate calls like
myLogger->debug(@args) to Log::Log4perl.

AUTOLOAD constructs a category dynamically, then tests it before logging it,

    # use FQ name as category
    my ($pkg,$file,$ln,$sub) = caller(my $i=1);  # once had stackwalk, 
to get past '(eval)'
    my $cat = "$sub_$meth_$ln";
    my $log = Log::Log4perl->get_logger($cat);
    my $predicate = "is_$cat";
    # is it runnable ?
    eval { $runnable = $log->$predicate() };
    if ($@) {
    carp("logger: cant $meth on $cat: $@");
    return;
    } else { $seenCat{$cat}++ }
   
    if ($runnable) {
    # Dump arrays or refs
    eval {
        @p = shift @_ while @_ and not ref $_[0];
        pop @_ while @_ and not $_[$#_];
        $log->$meth(Data::Dumper->Dump([\@_],["@p"])) if @_;
        $log->$meth(@p) if ! @_;
    };
    if ($@) { carp("logger dump problem on $sub.$ln") }
    }
    return;

This gives me extraordinary control over the logging - I can turn  
logging on/off
at any level - even down to line-number; something like:

log4j.category.GwApi=DEBUG
log4j.category.GwApi._precond=INFO

# these both warn of interface violation
log4j.category.GwApi._required.notice=ERROR
log4j.category.GwApi._precond.notice=ERROR
# but..
# above suppresses a notice-level message by raising
# the logging threshold above it

# precondition check. Ive seen enough of them..
#log4j.category.GwApi._precond.info.181=WARN


Also, AUTOLOAD uuses Data::Dumper to print data-structures when @_ are refs
its a cheap (programmer-time) way to get a lot of context in the logs.

<441688> GwApi.AUTOLOAD.info: $screen yielded: = [
  bless( {
    'Buf' => [],
    'Cmd' => undef,
    'Id' => 10,
    'Vals' => {},
    'response' => bless( do{\(my $o = ' 0')}, 'TnResponse' ),
    'session' => bless( {
      '1024' => '1024',
      'host' => 'localhost',
      'resp' => bless( do{\(my $o = 'U U U C(localhost) I 2 24 80 0 0 
0x2e00028 1')}, 'TnResponse' ),
      'win' => bless( \*Symbol::GEN6, 'Expect' )
    }, 'TnSession' )
  }, 'TirksScreen' )
];


Finally -

 given that line-numbers are a clumsy way of  controlling output (better 
done by super-category)
I thought it would be good to catalog all invoations encountered in a 
given run of the sw,
which should be written out to a file written at process end.   This 
would be useful for assessing
the coverage of a test case/suite.  Its not working yet (probly should 
be in an END block),
 but you can see the direction.


sub DESTROY {
    print "# observed logging categories:\n", Dumper \%seenCat;
}

[log4perl-devel] DBI appender

[Kevin G. <ke...@go...>]

All right, I've finally come up with a good synthesis of the 
(deprecated) JDBCAppender and Log::Dispatch::DBI.

It's checked in, it's very flexible, and I'm pretty happy with the API. 
   Any suggestions are more than welcome.

I matched the JDBCAppender pretty closely, a config file for that should 
work with this, and it can use the same behavior of filling up the log 
statements in a buffer and writing them out when bufferSize is reached, 
but it can use other behaviors as well.  This also has additional 
features, like placeholders and bind values, though, which are much more 
secure and highly recommended.  I tried to make some allowance for 
performance efficiency as well.

I had to completely rewrite Log::Dispatch::DBI, creating a 
log4perl-specific version.  I put it under Log::Log4perl::Appender::DBI, 
but I'm not sure if that's right, since it's *actually* a subclass of 
Appender.pm.  Any comments on that?

Working with log4j's PatternLayout I also found that while we had 
log4perl dying if no ConversionPattern was specified, they defaulted to 
%m%n, so I changed to match that.

Already checked in, attached is a diff for your convenience (may not 
work as a patch, haven't tried).

Anybody's comments are welcome!


-- 
Happy Trails . . .

Kevin M. Goess
(and Anne and Frank)
904 Carmel Ave.
Albany, CA  94706
(510) 525-5217

[log4perl-devel] Log::Log4perl Recipe of the Week (#9)

[<log...@pe...>]

Welcome to the Log::Log4perl recipe of the week. Today:

============================================================
Log::Log4perl Recipe of the Week (#9):

How can I roll over my logfiles automatically at midnight?
============================================================

Long-running applications tend to produce ever-increasing
logfiles. For backup and cleanup purposes, however, it is
often desirable to move the current logfile to a different
location from time to time and start writing a new one.

This is a non-trivial task, because it has to happen in sync
with the logging system in order not to lose any messages in
the process.

Luckily, *Mark Pfeiffer*'s "Log::Dispatch::FileRotate"
appender works well with Log::Log4perl to rotate your
logfiles in a variety of ways. All you have to do is specify
it in your Log::Log4perl configuration file and your
logfiles will be rotated automatically.

You can choose between rolling based on a maximum size
("roll if greater than 10 MB") or based on a date pattern
("roll everyday at midnight"). In both cases,
"Log::Dispatch::FileRotate" allows you to define a number
"max" of saved files to keep around until it starts
overwriting the oldest ones. If you set the "max" parameter
to 2 and the name of your logfile is "test.log",
"Log::Dispatch::FileRotate" will move "test.log" to
"test.log.1" on the first rollover. On the second rollover,
it will move "test.log.1" to "test.log.2" and then
"test.log" to "test.log.1". On the third rollover, it will
move "test.log.1" to "test.log.2" (therefore discarding the
old "test.log.2") and "test.log" to "test.log.1". And so
forth. This way, there's always going to be a maximum of 2
saved log files around.

Here's an example of a Log::Log4perl configuration file,
defining a daily rollover at midnight (date pattern
"yyyy-MM-dd"), keeping a maximum of 5 saved logfiles around:

    log4perl.category         = WARN, Logfile
    log4perl.appender.Logfile = Log::Dispatch::FileRotate
    log4perl.appender.Logfile.filename    = test.log
    log4perl.appender.Logfile.max         = 5
    log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
    log4perl.appender.Logfile.TZ          = PST
    log4perl.appender.Logfile.layout = \
        Log::Log4perl::Layout::PatternLayout 
    log4perl.appender.Logfile.layout.ConversionPattern = %d %m %n 

Please see the "Log::Dispatch::FileRotate" documentation for
details. "Log::Dispatch::FileRotate" is available on CPAN.


Have fun! Until next week.

-- Mike

###################################
# Mike Schilli                    #
# log...@pe...        #
# http://perlmeister.com          #
# http://log4perl.sourceforge.net #
###################################

[Fwd: Re: [log4perl-devel] Patch for accepting code snippets as config values]

[Mike S. <msc...@ao...>]

Hey all,

here's an alternative way of getting dynamic values into the config file 
-- Thomas Schuler came up with the idea of simply using Text::Template, 
very elegant ...

-- 
-- Mike

Mike Schilli
log...@pe...

Re: [log4perl-devel] Forks

[<msc...@ao...>]

In a message dated 12/17/2002 2:16:01 PM Eastern Standard Time, "Wiggins d'Anconia" <wi...@da...> writes:

>We have exhibited the mixed log 
>message line problem before with syslog (rarely), 

Actually, just wanted to mention that you can certainly work around the problem by defining different log files for your children -- that's easy to do in the Log4perl config if they all belong to different categories. And, as I said, you can always fix the overlapping permanently by working with a synchronized Log::Dispatch::XXX module (which you'd have to write, but that's fairly straight-forward, just check the Log::Dispatch documentation).

>I am familar with OSS, etc. and not to ask for timelines, etc. as I know 
>they are VERY soft, but I was wondering if there is an estimated 
>timeline for a version 1.0

Funny you should ask. When we started Log::Log4perl, version 1.0 was aimed for as a complete port of Log4j. I'd say we're about 80% done for the core (on the other hand, there's still a lot of standard appenders to be written), but the remaining core parts are more or less exotic features, which hardly anyone uses, in fact no one I know of.

So, it turned out that most people (in fact all people I know of)  are perfectly happy with what we have right now. It's stable, we don't have any critical bugs and should we be informed about any, we would fix them immediately (within a couple of days usually).

If anyone wants a feature that's currently not available (like custom filters or the like), we'll go ahead and implement it right away. If not, we're gonna work on it as time allows.

So, I'm not exactly sure if we're ever going to reach 1.0 -- and it's probably not needed at all. Or, maybe we will redefine what "1.0" means, who knows.

But if you're concerned about maturity or stability, I can ascertain you that we're not going to change the API anymore (unless for a very good reason) and that we're always going to make sure that the extensive regression test suite (330 test cases) will pass flawlessly. And, yes, I'm using Log::Log4perl 0.27 on a production system.

-- 
-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

Re: [log4perl-devel] Forks

[Wiggins d'A. <wi...@da...>]

Thanks for the confidence booster.  We have exhibited the mixed log 
message line problem before with syslog (rarely), and while we would 
prefer to avoid it (obviously) that is not a reason not to use Log4perl 
(since syslog is doing it to anyways), and the benefits far outway that 
to us.

I am on the list and I am sure I will have more questions as we get 
further along in development, and will definitely keep you posted.

I am familar with OSS, etc. and not to ask for timelines, etc. as I know 
they are VERY soft, but I was wondering if there is an estimated 
timeline for a version 1.0 and what the general thoughts are on what a 
version 1.0 might look like. Or if there is a development tree/timeline, 
etc. somewhere that I should have already found :-)??  Just curious as 
our next app should launch at the end of march 2003 and I was wondering 
what the status of Log4perl hopes to be by then.....

Thanks again fro the info,

http://danconia.org

Kevin Goess wrote:
> Yeah, I agree with all that.  I would think if you don't try to reload 
> the config while the process is running, you should be mostly ok. 
> Reloading might be ok, too, we're just not saying for sure yet.  Let us 
> know how it goes for you.
> 
> 
> msc...@ao... wrote:
> 
>> In a message dated 12/16/2002 2:09:49 PM Eastern Standard Time, 
>> "Wiggins d'Anconia" <wi...@da...> writes:
>>
>>
>>> First off Log4perl rules.
>>
>>
>>
>> Thanks, we're certainly happy you like it! :)
>>
>>
>>> Our app will become a daemon, then fork once to watch a series of 
>>> directories, and then fork again to process files that fall into 
>>> those directories
>>
>>
>>
>> Log::Log4perl has been designed with multi-processing/threading in 
>> mind. In the multi-threading area we do have some issues regarding 
>> config_and_watch(), especially if you change the config file in weird 
>> ways while the program is running. In the multi-processing area I'm 
>> not aware of any problems (but please file a bug if you find any :).
>>
>> Regarding concurrency in general, please make sure to know about the 
>> following: If you have one Log4perl config file which the daemon reads 
>> and passes the Log4perl configuration on to its children, you might 
>> end up with several processes sharing a single appender instance in 
>> different process spaces. Calls to this appender are *not* 
>> synchronized. Which means that if you have a, say, out-of-the-box 
>> Log::Dispatch::File appender pointing to an open file, and all 
>> processes try to log a message concurrently, you might have mingled 
>> log lines. This is extremely rare, though, I've never seen it 
>> happening even under high concurrent load. If you can't live with 
>> that, you can still write your own appender which synchronizes calls 
>> at the OS level (with semaphores or the like), but it will be slower. 
>> Let me know how it works out for you, we're certainly gonna help 
>> should you have any questions.
>>
> 
> 

Re: [log4perl-devel] Forks

[Kevin G. <ke...@go...>]

Yeah, I agree with all that.  I would think if you don't try to reload 
the config while the process is running, you should be mostly ok. 
Reloading might be ok, too, we're just not saying for sure yet.  Let us 
know how it goes for you.


msc...@ao... wrote:
> In a message dated 12/16/2002 2:09:49 PM Eastern Standard Time, "Wiggins d'Anconia" <wi...@da...> writes:
> 
> 
>>First off Log4perl rules.
> 
> 
> Thanks, we're certainly happy you like it! :)
> 
> 
>>Our app will 
>>become a daemon, then fork once to watch a series of directories, and 
>>then fork again to process files that fall into those directories
> 
> 
> Log::Log4perl has been designed with multi-processing/threading in mind. In the multi-threading area we do have some issues regarding config_and_watch(), especially if you change the config file in weird ways while the program is running. In the multi-processing area I'm not aware of any problems (but please file a bug if you find any :).
> 
> Regarding concurrency in general, please make sure to know about the following: If you have one Log4perl config file which the daemon reads and passes the Log4perl configuration on to its children, you might end up with several processes sharing a single appender instance in different process spaces. Calls to this appender are *not* synchronized. Which means that if you have a, say, out-of-the-box Log::Dispatch::File appender pointing to an open file, and all processes try to log a message concurrently, you might have mingled log lines. This is extremely rare, though, I've never seen it happening even under high concurrent load. If you can't live with that, you can still write your own appender which synchronizes calls at the OS level (with semaphores or the like), but it will be slower. Let me know how it works out for you, we're certainly gonna help should you have any questions.
> 


-- 
Happy Trails . . .

Kevin M. Goess
(and Anne and Frank)
904 Carmel Ave.
Albany, CA  94706
(510) 525-5217

Re: [log4perl-devel] Forks

[<msc...@ao...>]

In a message dated 12/16/2002 2:09:49 PM Eastern Standard Time, "Wiggins d'Anconia" <wi...@da...> writes:

>First off Log4perl rules.

Thanks, we're certainly happy you like it! :)

>Our app will 
>become a daemon, then fork once to watch a series of directories, and 
>then fork again to process files that fall into those directories

Log::Log4perl has been designed with multi-processing/threading in mind. In the multi-threading area we do have some issues regarding config_and_watch(), especially if you change the config file in weird ways while the program is running. In the multi-processing area I'm not aware of any problems (but please file a bug if you find any :).

Regarding concurrency in general, please make sure to know about the following: If you have one Log4perl config file which the daemon reads and passes the Log4perl configuration on to its children, you might end up with several processes sharing a single appender instance in different process spaces. Calls to this appender are *not* synchronized. Which means that if you have a, say, out-of-the-box Log::Dispatch::File appender pointing to an open file, and all processes try to log a message concurrently, you might have mingled log lines. This is extremely rare, though, I've never seen it happening even under high concurrent load. If you can't live with that, you can still write your own appender which synchronizes calls at the OS level (with semaphores or the like), but it will be slower. Let me know how it works out for you, we're certainly gonna help should you have any questions.

-- 
-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

[log4perl-devel] Log::Log4perl Recipe of the Week (#8)

[<log...@pe...>]

Welcome to the Log::Log4perl recipe of the week. Today:

============================================================
Log::Log4perl Recipe of the Week (#8):

What if I need dynamic values in a static Log4perl
configuration file?
============================================================

Say, your application uses Log::Log4perl for logging and
therefore comes with a Log4perl configuration file,
specifying the logging behaviour. But, you also want it to
take command line parameters to set values like the name of
the log file. How can you have both a static Log4perl
configuration file and a dynamic command line interface?

As of Log::Log4perl 0.28, every value in the configuration
file can be specified as a *Perl hook*. So, instead of
saying

    log4perl.appender.Logfile.filename = test.log

you could just as well have a Perl subroutine deliver the
value dynamically:

    log4perl.appender.Logfile.filename = sub { logfile(); };

given that "logfile()" is a valid function in your "main"
package returning a string containing the path to the log
file.

Or, think about using the value of an environment variable:

    log4perl.appender.DBI.user = sub { $ENV{USERNAME} };

When "Log::Log4perl->init()" parses the configuration file,
it will notice the assignment above because of its "sub
{...}" pattern and treat it in a special way: It will
evaluate the subroutine (which can contain arbitrary Perl
code) and take its return value as the right side of the
assignment.

A typical application would be called like this on the
command line:

    app                # log file is "test.log"
    app -l mylog.txt   # log file is "mylog.txt"

Here's some sample code implementing the command line
interface above:

    use Log::Log4perl qw(get_logger);
    use Getopt::Std;

    getopt('l:', \our %OPTS);

    my $conf = q(
    log4perl.category.Bar.Twix         = WARN, Logfile
    log4perl.appender.Logfile          = Log::Dispatch::File
    log4perl.appender.Logfile.filename = sub { logfile(); };
    log4perl.appender.Logfile.layout   = SimpleLayout
    );

    Log::Log4perl::init(\$conf);

    my $logger = get_logger("Bar::Twix");
    $logger->error("Blah");

    ###########################################
    sub logfile {
    ###########################################
        if(exists $OPTS{l}) {
            return $OPTS{l};
        } else {
            return "test.log";
        }
    }

Every Perl hook may contain arbitrary perl code, just make
sure to fully qualify eventual variable names (e.g.
%main::OPTS instead of %OPTS).

SECURITY NOTE: this feature means arbitrary perl code can be
embedded in the config file. In the rare case where the
people who have access to your config file are different
from the people who write your code and shouldn't have
execute rights, you might want to set

    $Log::Log4perl::ALLOW_CODE_IN_CONFIG_FILE = 0;

before you call init(). This will prevent Log::Log4perl from
executing *any* Perl code in the config file (including code
for custom conversion specifiers (see "Custom cspecs" in
Log::Log4perl::Layout::PatternLayout).


Have fun! Until next week.

-- Mike

###################################
# Mike Schilli                    #
# log...@pe...        #
# http://perlmeister.com          #
# http://log4perl.sourceforge.net #
###################################

[log4perl-devel] Forks

[Wiggins d'A. <wi...@da...>]

Hello list,

First off Log4perl rules.

I am working on an application that will hopefully use the module's 
capabilities extensively, and have one early question.  Our app will 
become a daemon, then fork once to watch a series of directories, and 
then fork again to process files that fall into those directories, is 
there anything that we need to be concerned about using log4perl in this 
manner?  I read in the docs that it is moving towards or has already 
achieved thread safety, but we cannot go that route (threaded that is), 
and my concern lies with multiple forks writing to the same log at the 
same time, etc. and any issues with the multiple instances of the 
configuration, etc. in memory. (Solaris 8 for reference, perl 5.6.1 for 
the time being but planning on 5.8.0.)  Thanks for any help/suggestions,

http://danconia.org

Re: [log4perl-devel] implementing a DBI appender, RFC

[Kai P. <ka...@po...>]

Hi,
please do a commit after every execute. Otherwise, if the program crashes, the 
logs are rolled back and you cannot see whats happened. Some databases might 
be in an autocommit mode, where this is not a problem. I work with Oracle and 
the standard is to have no autocommit.

Kai

On Monday 09 December 2002 20:57, Kevin Goess wrote:
> I needed to add logging to a database, here's a proposed solution.
>
> Log4j has a JDBCAppender with the warning: "This version of JDBCAppender
> is very likely to be completely replaced in the future."  The API is
> kind of different, you call append() until the buffer is full, at which
> point the an sth is created and executed on the buffer.  So the API
> isn't interchangeable with other appenders.
>
> So I've implemented a DBI appender that's completely controllable from
> the config file, looking something like this:
>
>      log4j.category = WARN, DBAppndr
>      log4j.appender.DBAppndr             = Log::Log4perl::Appender::DBI
>      log4j.appender.DBAppndr.datasource = DBI:CSV:f_dir=t/tmp
>      log4j.appender.DBAppndr.username  = bobjones
>      log4j.appender.DBAppndr.password = 12345
>      log4j.appender.DBAppndr.sqlStatement = \
>         insert into log4perltest \
>         (level, message, shortmessage) \
>         values (?,?,?)
>      log4j.appender.DBAppndr.params.1 = %p
>      log4j.appender.DBAppndr.params.2 = %m
>      log4j.appender.DBAppndr.params.3 = %5.5m
>      ...
>
> The code is based on Log::Dispatch::DBI by Tatsuhiko Miyagawa.  His
> module gets behavior changes via subclassing, but by the time I was done
> I had overridden everything but DESTROY, so I just dropped the
> dependency on his module.
>
> Below is the only change to existing code, the module and a unit test
> are attached. Any comments or suggestions?
>
>
> ============================================================
> --- lib/Log/Log4perl/Appender.pm        18 Nov 2002 20:03:39 -0000     
> 1.17 +++ lib/Log/Log4perl/Appender.pm        9 Dec 2002 19:51:31 -0000
> @@ -120,7 +120,10 @@
>                                               $level,
>                                               3 +
> $Log::Log4perl::caller_depth,
>                                               );
> -    $self->{appender}->log(%$p);
> +    $self->{appender}->log(%$p,
> +                            #these are used by our Appender::DBI
> +                            log4p_category => $category,
> +                            log4p_level  => $level,);
>
>       return 1;
>   }

-- 
"I'd love to go out with you, but I'm taking punk totem pole carving."

Unix, WinNT and MS-DOS.  The Good, The Bad and The Ugly.
Kai Poitschke       MailTo:kai[_at_]poitschke[_dot_]de
Date/Time:  Fri Dec 13 08:52:55 MET 2002

Re: [log4perl-devel] Configuration file perl hooks

[<msc...@ao...>]

In a message dated 12/12/2002 8:01:14 PM Eastern Standard Time, Kevin Goess <ke...@go...> writes:

>But it just occurred to me, I had misgivings about 
>putting perl code into the config file for installations where different 
>people had access to the config than had access to the modules.

Definitely, good catch! Maybe we should consolidate the two features into one and control both with a single variable ALLOW_CODE_IN_CONFIG_FILE (true by default and set to 0 if restricted). 

Also, I've noticed the security notice was in there twice around the same section, now it's once in the PatternLayout section and once in the "Perl Hooks" section.

Patch attached ...


-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

Re: [log4perl-devel] Configuration file perl hooks

[Kevin G. <ke...@go...>]

That's great, Mike.  But it just occurred to me, I had misgivings about 
putting perl code into the config file for installations where different 
people had access to the config than had access to the modules.  People 
might be suprised when their config file turns out to be executable 
code.  Maybe I'm being paranoid, but should we have something like

	$Log::Log4perl::DONT_ALLOW_USER_DEFINED_CSPECS

and some security warnings?



Mike Schilli wrote:
> Just checked in the configuration file perl hooks into CVS:
> 
> =head2 Perl Hooks in the Configuration File
> 
> If some of the values used in the Log4perl configuration file
> need to be dynamically modified by the program, use Perl hooks:
> 
>    log4perl.appender.File.filename = \
>        sub { return getLogfileName(); }
> 
> Each value starting with the string C<sub {...> is interpreted as Perl 
> code to
> be executed at the time the application parses the configuration
> via C<Log::Log4perl::init()>. The return value of the subroutine
> is used by Log::Log4perl as the configuration value.
> 
> The Perl code is executed in the C<main> package, functions in
> other packages have to be called in fully-qualified notation.
> 
> Here's another example, utilizing an environment variable as a
> username for a DBI appender:
> 
>    log4perl.appender.DB.username = \
>        sub { $ENV{DB_USER_NAME } }
> 
> However, please note the difference between these code snippets and those
> used for user-defined conversion specifiers as discussed in
> L<Log::Log4perl::PatternLayout>: While the snippets above are run I<once>
> when C<Log::Log4perl::init()> is called, the conversion specifier
> snippets are executed I<each time> a message is rendered according to
> the PatternLayout.
> 


-- 
Happy Trails . . .

Kevin M. Goess
(and Anne and Frank)
904 Carmel Ave.
Albany, CA  94706
(510) 525-5217

[log4perl-devel] Configuration file perl hooks

[Mike S. <msc...@ao...>]

Just checked in the configuration file perl hooks into CVS:

=head2 Perl Hooks in the Configuration File

If some of the values used in the Log4perl configuration file
need to be dynamically modified by the program, use Perl hooks:

    log4perl.appender.File.filename = \
        sub { return getLogfileName(); }

Each value starting with the string C<sub {...> is interpreted as Perl 
code to
be executed at the time the application parses the configuration
via C<Log::Log4perl::init()>. The return value of the subroutine
is used by Log::Log4perl as the configuration value.

The Perl code is executed in the C<main> package, functions in
other packages have to be called in fully-qualified notation.

Here's another example, utilizing an environment variable as a
username for a DBI appender:

    log4perl.appender.DB.username = \
        sub { $ENV{DB_USER_NAME } }

However, please note the difference between these code snippets and those
used for user-defined conversion specifiers as discussed in
L<Log::Log4perl::PatternLayout>: While the snippets above are run I<once>
when C<Log::Log4perl::init()> is called, the conversion specifier
snippets are executed I<each time> a message is rendered according to
the PatternLayout.

-- 
-- Mike

Mike Schilli
log...@pe...

Re: [log4perl-devel] Patch for accepting code snippets as config values

[Kevin G. <ke...@go...>]

Great idea, I was going to do the same when I got around to it.


msc...@ao... wrote:
> Hey guys,
> 
> I've been playing with the idea of generally allowing Perl code snippets starting like "sub { ..." as values for Log4perl configuration file entities. Config items that are in the "cspec" hierarchy are excluded because they're evaluated at runtime, not config-parse-time. Makes things like
> 
>     log4perl.appender.Logfile.filename = \
>         sub { getlogfilename() }
> 
> easy to implement ...
> 
> 


-- 
Happy Trails . . .

Kevin M. Goess
(and Anne and Frank)
904 Carmel Ave.
Albany, CA  94706
(510) 525-5217

[log4perl-devel] Patch for accepting code snippets as config values

[<msc...@ao...>]

Hey guys,

I've been playing with the idea of generally allowing Perl code snippets starting like "sub { ..." as values for Log4perl configuration file entities. Config items that are in the "cspec" hierarchy are excluded because they're evaluated at runtime, not config-parse-time. Makes things like

    log4perl.appender.Logfile.filename = \
        sub { getlogfilename() }

easy to implement ...


-- 
-- Mike

############################
# Mike Schilli             #
# log...@pe... #
# http://perlmeister.com   #
# log4perl.sourceforge.net #
############################

Re: [log4perl-devel] implementing a DBI appender, RFC

[Mike S. <msc...@ao...>]

er...@se... wrote:

>
> I'll toss in one... it'd be nice if at least the password (if not the 
> username) could be obtained from environment variables. What we do 
> around here is we have a reasonably secure script that sets certain 
> environment variables and give permission to use said script via sudo. 
> This lets the DBAs keep their passwords under control. 

That sounds like something I'd rather not encourage ... :)

But if you insist, you could probably use the soon-to-come perl-sub 
initialization of config variables:

     username = sub { $ENV{username} }

-- 
-- Mike

Mike Schilli
log...@pe...