[log4perl-devel] UTF-8, Threads and Syswrite

[<pa...@ae...>]

Hello List,

 

New here so forgive me if this is something already solved or known.

 

Basically I have a program that is running multiple threads with
Log4perl as the logging mechanism.

 

If I simplify away everything, lets say I have a configuration file with
something like this:

 

log4perl.logger=DEBUG, Logfile

log4perl.appender.Logfile=Log::Log4perl::Appender::File

log4perl.appender.Logfile.utf8 = 1

log4perl.appender.Logfile.filename=  out.log

log4perl.appender.Logfile.layout=Log::Log4perl::Layout::PatternLayout

log4perl.appender.Logfile.layout.ConversionPattern=%d{ISO8601} %p %c -
%m%n

log4perl.appender.Logfile.mode = append

log4perl.appender.Logfile.syswrite = 1

 

Then the non-threaded perl code:

 

 

use Log::Log4perl qw(get_logger);

Log::Log4perl->init_and_watch("log.conf", 60);

 

my $char = "\x{30DE}\x{30A4}\x{30AF}";

 

foo('Somecategory');

 

sub foo {

        my $cat = shift;

        my $log = get_logger($cat);

 

        $log->info("character [$char]");

}

 

 

Everything works,  because the utf8 flag is set everything is fine.
However, as soon as we start spawning threads...

 

use threads;

use threads::shared;

use Log::Log4perl qw(get_logger);

Log::Log4perl->init_and_watch("log.conf", 60);

 

my $char = "\x{30DE}\x{30A4}\x{30AF}";

 

#my $a = threads->create(\&foo, 'Somecategory');

#$a->join();

 

foo('Somecategory');

 

sub foo {

        my $cat = shift;

        my $log = get_logger($cat);

 

        $log->info("character [$char]");

}

 

I get "Thread 1 terminated abnormally: Wide character in syswrite at
/usr/lib/perl5/site_perl/5.8.8/Log/Log4perl/Appender/File.pm line 242."

 

This seems to be a case where the file handle is losing the utf8
property, and so it must be set again on the handle.  I'm not sure why
exactly that happens and am still investigating but this seems to fix
it:

 

Adding to Log4perl/Appender/File.pm on line 242 above the syswrite:

 

 If (defined $self->{utf8}) {

        binmode($fh, ":utf8");

 }

 

Note that I have done testing on other file handles and this appears to
affect everything.  Possibly I am doing something wrong and need to make
Log4Perl more aware of threads?  The get_logger calls within the
subroutine are there to simulate the usage of my program, and are
possibly not the ideal use?

 

Please let me know, great tool otherwise!

 

-Pat