#128 combining unicode chars count in grid tables

[Günter Milde]

Combining unicode chars in headings and simple grid tables lead to warnings and errors, as they contribute to the string lenght.

[Günter Milde]

test case (tables and headings with combining chars)

[Günter Milde]

Fixed for section headings.

The fix for tables is nontrivial, as both stringlist.get_2D_block() in statemachine.py and
SimpleTableParser.check_columns() in tableparser.py must be made to account for
zero-width combining chars.

[Günter Milde]

Fixed for headings and simple tables.
Fixing grid tables is an open task. Tools are in docutils.utils
but the grid table parser is rather complex.

[Günter Milde]

In http://permalink.gmane.org/gmane.text.docutils.devel/7679
Edvard d'auvergne pointed to https://pypi.python.org/pypi/wcwidth, a Python implementation of the wcswidth C function.

We already similar functions in docutils.utils, used for section headings and simple tables.
However, currently these do not treat zero-width characters.

TODO:

• Fix grid tables (difficult, see attachment for a try).

• Consider additional zero-width characters (ZWSP, WJ, ZWNBSP
  ...) in table parsing and section heading recognition.

According to David, this "Seems like more trouble than it's worth."
http://permalink.gmane.org/gmane.text.docutils.devel/7707

[Edward d'Auvergne]

+1

[Edward d'Auvergne]

That was me adding weight, as the SF voting buttons are not visible for docutils tickets for non-members.

[Adam Turner]

Cannot reproduce on latest master, I believe Python 3's native unicode handling makes this a non-issue.

A

[Günter Milde]

This is fixed for headings and simple tables but still fails for grid tables (not tested in the original test sample). The new sample fails here.

[Günter Milde]

Even when adjusting the grid, the parser goes wrong. See [bugs:#512].

[Günter Milde]

Fixed, now also for grid tables, in [r10251].

[Günter Milde]

Fixed in Docutils 0.22.3.