matplotlib-users

[Matplotlib-users] unicode filenames and MPL.

[Christopher B. <Chr...@no...>]

Hi all,

I'm writing a little MPL embedded in wxPython app, and just ran into this:

Traceback (most recent call last):
...
...
"..../matplotlib-0.91.2-py2.5-macosx-10.3-fat.egg/matplotlib/backends/backend_agg.py", 

line 397, in print_png
     self.get_renderer()._renderer.write_png(filename, 
self.figure.dpi.get())
TypeError: cannot return std::string from Unicode object

As soon as I saw it, I knew what happened: I'm running the unicode 
version of wxPython, so the filename I got back from a SaveFile Dialog 
is in unicode. Now, it's easy enough for me to turn that into a string 
for now, but as more and more file systems are implemented in unicode, 
it might be nice if we could use unicode file names with MPL.

Are there any plans along these lines? Or are we just going to have to 
wait for py3k?

-CHB

-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Matplotlib-users] unicode filenames and MPL.

[Michael D. <md...@st...>]

I think the real reason this wasn't done is that its tricky to do at the 
C level in a cross-platform way.  At present it uses the regular POSIX 
fopen in C, which isn't really "Unicode aware".  See the "Unicode 
filenames" section of the link below for some of the complications.  
Linux is particularly hard to get right:

http://www.amk.ca/python/howto/unicode

However, the Python runtime does take care of many of these details in a 
(mostly) reasonable way.  I think the easiest solution is to open the 
file on the Python side (rather than using POSIX fopen in C as we do 
now), and pass the file descriptor itself over to C.  There was a bug in 
extracting the file descriptor from a Python file object in earlier 
versions of mpl, so it was removed a while ago, but I think I have it 
working again.

This should now be fixed on r5025 (branch) and r5026 (trunk)

Note that if you are running 0.91.2 (and not SVN, where this was 
subsequently broken in r4874), you can do the following as a workaround 
(with a performance hit from making many Python function calls):

   savefig(open(u"CrazyUnicodeFilename.png", "w"))

On a related note -- there is still an issue where fonts with Unicode 
paths can not be loaded.  That can hopefully be addressed in a similar 
manner, but there are a few more code paths to fix up there.

Mike


Christopher Barker wrote:
> Hi all,
>
> I'm writing a little MPL embedded in wxPython app, and just ran into this:
>
> Traceback (most recent call last):
> ...
> ...
> "..../matplotlib-0.91.2-py2.5-macosx-10.3-fat.egg/matplotlib/backends/backend_agg.py", 
>
> line 397, in print_png
>      self.get_renderer()._renderer.write_png(filename, 
> self.figure.dpi.get())
> TypeError: cannot return std::string from Unicode object
>
> As soon as I saw it, I knew what happened: I'm running the unicode 
> version of wxPython, so the filename I got back from a SaveFile Dialog 
> is in unicode. Now, it's easy enough for me to turn that into a string 
> for now, but as more and more file systems are implemented in unicode, 
> it might be nice if we could use unicode file names with MPL.
>
> Are there any plans along these lines? Or are we just going to have to 
> wait for py3k?
>
> -CHB
>
>   

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Re: [Matplotlib-users] unicode filenames and MPL.

[Christopher B. <Chr...@no...>]

Michael Droettboom wrote:
> I think the real reason this wasn't done is that its tricky to do at the 
> C level in a cross-platform way.  At present it uses the regular POSIX 
> fopen in C, which isn't really "Unicode aware".

The actual error is from trying to put the filename in a std::string, 
but yes, I'm sure the fopen issue is the driver. Does C++ offer anything 
better?

   See the "Unicode
> filenames" section of the link below for some of the complications.  
> Linux is particularly hard to get right:
> 
> http://www.amk.ca/python/howto/unicode

Thanks, that's a good one.

 >  you can do the following as a workaround
> (with a performance hit from making many Python function calls):
> 
>   savefig(open(u"CrazyUnicodeFilename.png", "w"))

thanks, I'll give that a try. I'm confused, though, why the many Python 
function calls? the C++ code doesn't just grab the file pointer?

Thanks,
-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Matplotlib-users] unicode filenames and MPL.

[Michael D. <md...@st...>]

Christopher Barker wrote:
> Michael Droettboom wrote:
>   
>> I think the real reason this wasn't done is that its tricky to do at the 
>> C level in a cross-platform way.  At present it uses the regular POSIX 
>> fopen in C, which isn't really "Unicode aware".
>>     
>
> The actual error is from trying to put the filename in a std::string, 
>   
Presumably because it doesn't decode into ASCII.
> but yes, I'm sure the fopen issue is the driver. Does C++ offer anything 
> better?
>   
Not in the C++ standard library in a cross platform way, that I'm aware 
of.  All three platforms (if you include Gtk, Qt etc.) have special 
Unicode-aware APIs, and each deals with things differently.  The 
approach I took basically leverages all the trouble the Python core guys 
put into to make this work cross-platform.
>    See the "Unicode
>   
>> filenames" section of the link below for some of the complications.  
>> Linux is particularly hard to get right:
>>
>> http://www.amk.ca/python/howto/unicode
>>     
>
> Thanks, that's a good one.
>
>  >  you can do the following as a workaround
>   
>> (with a performance hit from making many Python function calls):
>>
>>   savefig(open(u"CrazyUnicodeFilename.png", "w"))
>>     
>
> thanks, I'll give that a try. I'm confused, though, why the many Python 
> function calls? the C++ code doesn't just grab the file pointer?
>   
No -- a while back, it was segfaulting when trying to do that (but I've 
since figured out why -- it was a reference counting problem).  In 
0.91.2, it calls the "write" method on the Python object for each block 
of PNG data.  SVN still supports that approach (to handle arbitrary 
Python file-like objects), but will extract the real C file handle if it 
is in fact a real file.

Mike

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA