#257 windows7 py3.4.1 testsuite encoding bug

Default
closed-fixed
None
5
2017-10-30
2014-06-08
No

running alltest.py

test_parsers\test_rst\test_directives\test_include.py::

  b'Encoding:\n\n.. include:: test_parsers/test_rst/test_directives/utf-16.csv\n   :encoding: utf-16\n'
    File "test3\alltests.py", line 40, in write
      string = string.encode('raw_unicode_escape').decode('ascii')
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xb0 in position 994: ordinal not in range(128)

fiddling with alltests.py (uncommitted) ::

  <system_message level="4" line="1" source="test data" type="SEVERE">
    <paragraph>
        Problem with "raw" directive:
        UnicodeDecodeError: \'utf-16-be\' codec can\'t decode bytes in position 90-91: illegal encoding
    <literal_block xml:space="preserve">
        .. raw:: html
           :file: test_parsers/test_rst/test_directives/utf-16.csv
           :encoding: utf-16''' != '''\

Discussion

  • Günter Milde

    Günter Milde - 2016-12-13
    • status: open --> pending-remind
     
  • Günter Milde

    Günter Milde - 2016-12-13

    Is this still a problem? (Can't test with Windows here.)
    On Debian/Linux, tests run with 2.4, 2.5, 2.6, 2.7, 3.1, 3.2 ,.3.3, 3.4, and 3.5.

     
  • Günter Milde

    Günter Milde - 2017-10-25
    • Group: repository --> Default
     
  • engelbert gruber

    • status: pending-remind --> closed-out-of-date
     
  • engelbert gruber

    no longer on windows7 python 2.7.5

     
  • engelbert gruber

    • status: closed-out-of-date --> pending-remind
    • assigned_to: engelbert gruber
     
  • engelbert gruber

    still on indows7 python 3.4.1

     
  • engelbert gruber

    2017-10-29: 0.15b.dev with Python 3.4.1 Windows 7 6.1.7601 (win32,

    test_parsers\test_rst\test_directives\test_include.py: totest['include'][12]; test_parser (DocutilsTestSupport.ParserTestCase)
    input:
    b'Encoding:\n\n.. include:: test_parsers/test_rst/test_directives/utf-16.csv\n :encoding: utf-16\n'
    -: expected
    +: output

    and similar looking

    test_parsers\test_rst\test_directives\test_include.py: totest['include'][12]; test_parser (DocutilsTestSupport.ParserTestCase)
    input:
    b'Encoding:\n\n.. include:: test_parsers/test_rst/test_directives/utf-16.csv\n :encoding: utf-16\n'
    -: expected
    +: output

     
  • engelbert gruber

    utf-16.csv has the subversion property svn:eol-style native.
    This means we get LF on unix and CRLF on windows.
    But it is utf-16 the unix end of line ist therefore: 0x00 0x0a
    Windows is: 0x00 0x0d 0x00 0x0a

    But subversion replaces 0x00 0x0a by 0x00 0x0d 0x0a and breaks the utf-16

     
  • engelbert gruber

    • status: pending-remind --> closed-fixed
     
  • engelbert gruber

    fixed by setting svn:eol-style to LF

     

Log in to post a comment.