#167 Python 3: test fail: UnicodeDecodeError, NameError

closed-fixed
nobody
None
6
2012-09-03
2011-08-21
U
No

With Docutils 0.8, Python 3.2 and C locale tests fail:

$ python3 --version
Python 3.2.2rc1

$ LC_ALL=C python3 alltests.py
Traceback (most recent call last):
File "/tmp/docutils-0.8/build/lib/test/test_error_reporting.py", line 233, in SafeStringTests_locale
open('\xfc')
UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 0: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/tmp/docutils-0.8/build/lib/test/test_error_reporting.py", line 238, in SafeStringTests_locale
open('\xfc'.encode(sys.getfilesystemencoding(), 'replace'))
IOError: [Errno 2] No such file or directory: '?'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "alltests.py", line 74, in <module>
suite = suite()
File "alltests.py", line 63, in suite
'test_', packages=1)
File "/tmp/docutils-0.8/build/lib/test/package_unittest.py", line 101, in loadTestModules
module = import_module(mod)
File "/tmp/docutils-0.8/build/lib/test/package_unittest.py", line 132, in import_module
mod = __import__(name)
File "/tmp/docutils-0.8/build/lib/test/test_error_reporting.py", line 216, in <module>
class SafeStringTests_locale(unittest.TestCase):
File "/tmp/docutils-0.8/build/lib/test/test_error_reporting.py", line 240, in SafeStringTests_locale
uioe = e
NameError: name 'e' is not defined

Discussion

  • U

    U - 2011-08-22
     
  • U

    U - 2011-08-22

    I applied the attached patch to my local copy. However, I still get a bunch of test failures:

    Testing Docutils 0.8 [release] with Python 3.2.2rc1 on 2011-08-22 at 00:42:12
    Working directory: /tmp/python-docutils-0.8/py3k/build/lib/test
    Docutils package: /tmp/python-docutils-0.8/py3k/build/lib/docutils
    ...............................................................................E...E.E.E..E..EEE.EEEE....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    ======================================================================
    ERROR: test_functional.py: functional/tests/math_output_mathml.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/math_output_mathml.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xcf in position 1048: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/math_output_mathjax.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/math_output_mathjax.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1566: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/math_output_latex.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/math_output_latex.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1357: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_s5_html_2.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_s5_html_2.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1474: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_html4css1.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_html4css1.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 22761: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/latex_cyrillic.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/latex_cyrillic.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 917: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_pseudoxml.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_pseudoxml.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3731: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_xetex.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_xetex.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 7584: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_latex.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_latex.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 20003: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/standalone_rst_s5_html_1.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/standalone_rst_s5_html_1.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1559: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/math_output_html.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/math_output_html.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 853: ordinal not in range(128)

    ======================================================================
    ERROR: test_functional.py: functional/tests/xetex_cyrillic.py; test (test_functional.FunctionalTestCase)
    test_functional.py: functional/tests/xetex_cyrillic.py
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/tmp/python-docutils-0.8/py3k/build/lib/test/test_functional.py", line 180, in test
    expected = '\n'.join(f.read().splitlines())
    File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 999: ordinal not in range(128)

    ----------------------------------------------------------------------
    Ran 1145 tests in 12.305s

     
  • Günter Milde

    Günter Milde - 2011-08-22

    Thanks for the report.
    The patch (and possibly more fixes) are already in the development version.
    Could you please test and ask in the docutils-devel list for a bugfix release?

     
  • Günter Milde

    Günter Milde - 2011-08-22
    • priority: 5 --> 6
    • status: open --> open-remind
     
  • Günter Milde

    Günter Milde - 2012-09-03
    • status: open-remind --> closed-fixed
     
  • Günter Milde

    Günter Milde - 2012-09-03

    This is (believed to be) fixed in Docutils 0.9 (2012-05-02).

     

Log in to post a comment.