#137 Wrong unicode characters representation in Python 3.4 and py2exe-3

[Zdenko]

I have problem with Unicode strings in frozen app. I use python 3.4.1 32bit (on Windows 7 64bit Pro) and py2exe-3 from svn repository. I can demonstrate it with following code:

# -*- coding: utf-8 -*-
# file: test_py2exe.py

import sys

my_string = u"""This is a test:
ábc
End of test..."""

filename = 'test.txt'
if getattr(sys, 'frozen', False):
    filename = 'test-frozen.txt'

f = open(filename,  mode='w', encoding='utf-8')
f.write(my_string)
f.close()

If I run in standard python shell (py test_py2exe.py) the second line in test.txt is like this (correct):

ábc

If I create frozen app with

py -3.4 -m py2exe.build_exe test_py2exe.py

and run 'dist\test_py2exe.exe' I have in test-frozen.txt second line like this:

ábc

This problem is not related to storing strings to file only, but also when I use other modules (e.g. PyQt5, xlsxwriter) with unicode strings.

I found workaround for that but it is not convenient:

sys_enc = sys.getdefaultencoding()  # 'utf-8'
locale_enc = locale.getpreferredencoding()  # 'cp1250'
my_string.encode('cp1250').decode('utf-8')

[Thomas Heller]

I think I found the problem.

Can you please try if following patch works for you:

Index: py2exe/runtime.py
===================================================================
--- py2exe/runtime.py   (revision 740)
+++ py2exe/runtime.py   (working copy)
@@ -574,11 +574,9 @@

             code_objects.append(boot_code)


-            with open(target.script, "U") as script_file:
+            with open(target.script, "rb") as script_file:
                 code_objects.append(
-                    # XXX what about compiler options?
-                    # XXX what about source file encodings?
-                    compile(script_file.read() + "\n",
+                    compile(script_file.read() + b"\n",
                             os.path.basename(target.script), "exec",
                             optimize=self.options.optimize))

[Zdenko]

Thanks. This change fixed my problem.