Re: [log4perl-devel] UTF-8, Threads and Syswrite

[<pa...@ae...>]

Hi Mike,

 

Thanks for the reply.

 

Sorry I should have mentioned this: My production system is perl 5.8.8.
I've had other people try the code (you can do it without log4perl just
using 

STDOUT even) and it seems to work fine in 5.10.  So this might just be
something that was improved with later versions of perl.

 

"use utf8" doesn't seem to change anything for my perl interpreter, but
regardless it won't affect the live code - the data is being sourced
from a web service, and I believe it is being correctly flagged as utf8
- though I will double check that. 

 

I will try some more up to date interpreters and see if that helps
things.

 

-Pat

 

________________________________

From: Mike Schilli [mailto:m...@pe...]
Sent: Wed 4/21/2010 12:06 AM
To: Pat Lynam
Cc: log...@li...
Subject: Re: [log4perl-devel] UTF-8, Threads and Syswrite

On Tue, 20 Apr 2010, pa...@ae... wrote:

> Everything works,  because the utf8 flag is set everything is fine.
> However, as soon as we start spawning threads...

Works for me also, only thing I changed was to add

     use utf8;

at the beginning as you're writing code in UTF8. Without it, I end up
with doubly-encoded characters in the log file (no warning, though).

Tested with perl-5.10.0 and perl-5.10.1.

-- Mike

Mike Schilli
m...@pe...

>
>
>
> use threads;
>
> use threads::shared;
>
> use Log::Log4perl qw(get_logger);
>
> Log::Log4perl->init_and_watch("log.conf", 60);
>
>
>
> my $char = "\x{30DE}\x{30A4}\x{30AF}";
>
>
>
> #my $a = threads->create(\&foo, 'Somecategory');
>
> #$a->join();
>
>
>
> foo('Somecategory');
>
>
>
> sub foo {
>
>        my $cat = shift;
>
>        my $log = get_logger($cat);
>
>
>
>        $log->info("character [$char]");
>
> }
>
>
>
> I get "Thread 1 terminated abnormally: Wide character in syswrite at
> /usr/lib/perl5/site_perl/5.8.8/Log/Log4perl/Appender/File.pm line
242."
>
>
>
> This seems to be a case where the file handle is losing the utf8
> property, and so it must be set again on the handle.  I'm not sure why
> exactly that happens and am still investigating but this seems to fix
> it:
>
>
>
> Adding to Log4perl/Appender/File.pm on line 242 above the syswrite:
>
>
>
> If (defined $self->{utf8}) {
>
>        binmode($fh, ":utf8");
>
> }
>
>
>
> Note that I have done testing on other file handles and this appears
to
> affect everything.  Possibly I am doing something wrong and need to
make
> Log4Perl more aware of threads?  The get_logger calls within the
> subroutine are there to simulate the usage of my program, and are
> possibly not the ideal use?
>
>
>
> Please let me know, great tool otherwise!
>
>
>
> -Pat
>
>
>
>
>
>