#5663 Encoding preparations for Python 3.5

[Jonas Hahnfeld]

Encoding preparations for Python 3.5

1. Use codecs.open() to decode as utf-8

This is in preparation for Python 3.5 where the default encoding
depends on the value of the LANG environment variable. As far as
I can tell, this was changed later on and at least Python 3.7 and
version 3.8 always default to 'utf-8' on Linux. As I'm proposing to
make Python 3.5 the required minimum, we can't rely on this and need
to force 'utf-8' when reading files that could contain Unicode.

1. musicxml2ly: Correct usage of codecs module

sys.stdout is already in lilylib and codecs.open() always opens
a file in binary mode.

1. lilylib: Remove encoded_write

It's not needed anymore as sys.stdout and sys.stderr are wrapped
with encoding writers. (The argument file for show_rules in script
convert-ly is also sys.stdout.)

http://codereview.appspot.com/551340043

[Anonymous]

This 'fails' a make doc ... right at the end .. it looks like it is about to complete but it posts this

Making Documentation/out-www/web-big-page.zh.html (hard link)
Making Documentation/out-www/web/index.zh.html (hard links)
Making .htaccess 
Making out-www/offline-root/index.html 
Mirroring...
Processing HTML pages for offline target...
Traceback (most recent call last):
  File "/home/james/lilypond-git/build/scripts/build/out/www_post", line 121, in <module>
    name_filter = strip_file_name[t])
  File "/home/james/lilypond-git/python/auxiliar/postprocess_html.py", line 381, in process_html_files
    page_flavors[k][1] = page_flavors[k][1] % subst[page_flavors[k][0]]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 34: ordinal not in range(128)
/home/james/lilypond-git/GNUmakefile.in:172: recipe for target 'out-www/offline-root/index.html' failed
make[1]: *** [out-www/offline-root/index.html] Error 1
/home/james/lilypond-git/stepmake/stepmake/generic-targets.make:178: recipe for target 'doc' failed
make: *** [doc] Error 2
[1]-  Done                    gitk  (wd: ~/lilypond-git)
(wd now: ~/lilypond-git/build)

[Jonas Hahnfeld]

Thanks for testing, this process is incredibly helpful! Unfortunately I can't figure out how to make this work in Python 2 without still needing changes for Python 3 :-( so I guess I'll just put this on my growing list of incompatible changes that need to be added after after running 2to3...

[Jonas Hahnfeld]

Revert changes for www_post

http://codereview.appspot.com/551340043

[Anonymous]

Passes make, make check and a full make doc.

[Anonymous]

Patch on coutdown for Jan 21st

[Anonymous]

Patch counted down - please push

[Jonas Hahnfeld]

pushed to staging as

commit 13877f0d5eeb0f67381abb3d5e0b250b1ba2afb5
Author:     Jonas Hahnfeld <hahnjo@hahnjo.de>
AuthorDate: Wed Jan 15 22:10:35 2020 +0100
Commit:     Jonas Hahnfeld <hahnjo@hahnjo.de>
CommitDate: Tue Jan 21 20:32:43 2020 +0100

    Issue 5663/3: lilylib: Remove encoded_write

    It's not needed anymore as sys.stdout and sys.stderr are wrapped
    with encoding writers. (The argument file for show_rules in script
    convert-ly is also sys.stdout.)

commit 5ee6b58300e363762f8c1d9597f360b07a40423e
Author:     Jonas Hahnfeld <hahnjo@hahnjo.de>
AuthorDate: Wed Jan 15 16:46:29 2020 +0100
Commit:     Jonas Hahnfeld <hahnjo@hahnjo.de>
CommitDate: Tue Jan 21 20:32:43 2020 +0100

    Issue 5663/2: musicxml2ly: Correct usage of codecs module

    sys.stdout is already in lilylib and codecs.open() always opens
    a file in binary mode.

commit e0c78a4c710c51e1ea87d2b144c0ae713923a2af
Author:     Jonas Hahnfeld <hahnjo@hahnjo.de>
AuthorDate: Wed Jan 15 16:39:56 2020 +0100
Commit:     Jonas Hahnfeld <hahnjo@hahnjo.de>
CommitDate: Tue Jan 21 20:32:35 2020 +0100

    Issue 5663/1: Use codecs.open() to decode as utf-8

    This is in preparation for Python 3.5 where the default encoding
    depends on the value of the LANG environment variable. As far as
    I can tell, this was changed later on and at least Python 3.7 and
    version 3.8 always default to 'utf-8' on Linux. As I'm proposing to
    make Python 3.5 the required minimum, we can't rely on this and need
    to force 'utf-8' when reading files that could contain Unicode.