#474 IndexError while parsing specially crafted option arguments

[Andrew Pan]

When rendering arbitrary input with the RST parser, I expect docutils to either return successfully or raise a MarkupError. However, the following input causes docutils to raise an IndexError:

-a<, , >

docutils tries to render this as an option group. It splits the group into individual options:

docutils/parsers/rst/states.py:1555:

optionstrings = match.group().rstrip().split(', ')

Which yields the following array (note the empty string):

(Pdb) optionstrings
['-a<', '', ' >']

Which is then tokenized with split(). When processing an empty "option", split() yields an empty array, causing this statement to raise an IndexError:

docutils/parsers/rst/states.py:1559:

firstopt = tokens[0].split('=', 1)

I checked the documentation and it appears that angle brackets are usually used to denote option arguments. I'm not sure if docutils should parse and render this kind of input. If it shouldn't, it would be nice to get a MarkupError.

Here's the output from docutils -V:
docutils (Docutils 0.20.1, Python 3.11.3, on darwin)

I've reproduced this on macOS 13.5.2 (22G91) and Ubuntu 22.04 with the attached reproducer.

[Andrew Pan]

Apologies, I attached the wrong reproducer. Please use this one.

[Günter Milde]

The underlying problem (splitting the "option marker" inside < >) is fixed in [r9473].
This also solves the IndexError.
Thank you for reporting.