Re: [Pythoncard-users] Unicode [slightly OT]

[bartek w. <ba...@re...>]

Citing Alex Tweedly <al...@tw...>:

> If I start up the Python interpreter (i.e. open a DOS shell box, and 
> type "python") I get my Python interpreter. I then type in the following 
> two lines, and get an error :
> 
> > C:\Documents and Settings\Eleane>python
> > Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] 
> > on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> s = u'a\u2019s'
> > >>> print s
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> >   File "C:\Python24\lib\encodings\cp850.py", line 18, in encode
> >     return codecs.charmap_encode(input,errors,encoding_map)
> > UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019' 
> > in position
> >  1: character maps to <undefined>
> > >>>
> 
> If instead I start the PythonCard codeEditor, and start a Shell (F5), 
> and type the same two lines, it works properly.
> 
> I tried the basic python interpreter adding the imports that are visible 
> within the codeEditor shell, but still get the same problem.
> 
> 
> > C:\Documents and Settings\Eleane>python
> > Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] 
> > on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import os
> > >>> import sys
> > >>> import wx
> > >>> from PythonCard import dialog, util
> > >>> s = u'a\u2019s'
> > >>> print s
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> >   File "C:\Python24\lib\encodings\cp850.py", line 18, in encode
> >     return codecs.charmap_encode(input,errors,encoding_map)
> > UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019' 
> > in position
> >  1: character maps to <undefined>
> > >>>
> 
> Anyone got any clues ?
> Does the code editor do something non-obvious that makes this all work 
> right when it opens a shell ?
> Or is there something additional I could try ?
> 
> [I don't really care about what the code editor does - just about being 
> able to get my app working, perhaps by doing the same as the codeEditor.]
> 
> (for now, I'm working around it by doing
>    s = s.encode('ascii', 'replace')
> which simply replaces all the odd characters by '?'s - ok for the short 
> term, but I do need to figure out a better answer).
> 
> 

This is caused by the fact that, according to PEP 100, when you try to "print" a
unicode string u, python implicitly calls u.encode(sys.getdefaultencoding()). 

The "recommended way" is to use always u.encode("something") when you print
unicode, but you can try to look at these posts:

http://faassen.n--tree.net/blog/view/weblog/2005/08/02/0

http://www.pycs.net/users/0000323/stories/14.html

to find out how to change the default encoding and why is it considered
harmful.


Hope that helps

Bartek