[Docutils-develop] [PATCH] Fix encoding/decoding with Python 2

[Arfrever F. T. A. <arf...@gm...>]

Users of some locales (e.g. cs_CZ.UTF-8) have been reproducing some problems with docutils and Python 2:
https://bugs.gentoo.org/show_bug.cgi?id=349101

Traceback (most recent call last):
  File "buildhtml.py", line 244, in <module>
    Builder().run()
  File "buildhtml.py", line 186, in run
    self.visit(recurse, root, dirs+files)
  File "buildhtml.py", line 211, in visit
    prune = self.process_txt(directory, name)
  File "buildhtml.py", line 237, in process_txt
    settings=settings)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/core.py", line 357, in publish_file
    enable_exit_status=enable_exit_status)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/core.py", line 641, in publish_programmatically
    output = pub.publish(enable_exit_status=enable_exit_status)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/core.py", line 203, in publish
    self.settings)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/readers/__init__.py", line 69, in read
    self.parse()
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/readers/__init__.py", line 75, in parse
    self.parser.parse(self.input, document)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/__init__.py", line 157, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 170, in run
    input_source=document['source'])
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/statemachine.py", line 233, in run
    context, state, transitions)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/statemachine.py", line 454, in check_line
    return method(match, context, next_state)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 2706, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 329, in section
    self.new_subsection(title, lineno, messages)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 398, in new_subsection
    node=section_node, match_titles=1)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 284, in nested_parse
    node=node, match_titles=match_titles)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/statemachine.py", line 233, in run
    context, state, transitions)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/statemachine.py", line 454, in check_line
    return method(match, context, next_state)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 2281, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 2293, in explicit_construct
    return method(self, expmatch)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 2035, in directive
    directive_class, match, type_name, option_presets)
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/states.py", line 2086, in run_directive
    result = directive_instance.run()
  File "/var/tmp/portage/dev-python/docutils-0.7/work/docutils-0.7/docutils/parsers/rst/directives/misc.py", line 70, in run
    (self.name, error.__class__.__name__, str(error)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not in range(128)

In case of docutils/parsers/rst/directives/misc.py, self.name was u'include' and error.message was (in Czech language)
'Adres\xc3\xa1\xc5\x99 nebo soubor neexistuje' ('No such file or directory'). String interpolation with 1 unicode object has
forced implicit decoding of str objects with wrong encoding (ascii instead of currently used encoding, in this case UTF-8).
Similar problems with usage of wrong encoding were occuring later after fixing the first problem.
I'm attaching the patch, which fixes these problems.

-- 
Arfrever Frehtes Taifersar Arahesis