#270 `python setup.py check --restructuredtext --strict --metadata` fails with `warning: check: Could not finish the parsing.` if the RST document uses `code` or `code-block` directives.
python setup.py check --restructuredtext --strict --metadata fails with:
warning: check: Could not finish the parsing.
if the RST document uses code or code-block directives.
This is annoying because the document is valid, but it appears to be invalid
and confuses people. For example, see
https://github.com/ionelmc/pytest-benchmark/pull/4#issuecomment-66940307
How to reproduce this bug
Clone a repo that has a README.rst with code-block directives in it. E.g.:
$ git clone git@github.com:ionelmc/pytest-benchmark.git
$ cd pytest-benchmark
$ git checkout ab0b08f6fccb06a7909905a8409f8faa8b01e0d8
Observe that it has "code-blocks" in it:
$ grep 'code-block' README.rst .. code-block:: python .. code-block:: python
Observe that RST document is valid, according to rst2html.py:
$ rst2html.py --halt=1 README.rst > README.html && echo "RST was OK." RST was OK. $ head -n 3 README.html <?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
Observe that python setup.py check --restructuredtext --strict --metadata fails:
$ python setup.py check --restructuredtext --strict --metadata running check warning: check: Could not finish the parsing. error: Please correct your package. $ echo $? 1
What was expected: python setup.py check --restructuredtext --strict
--metadata should succeed with no warnings, just as rst2html.py did, because
README.rst is a valid RST document.
What actually happened: python setup.py check --restructuredtext --strict
--metadata prints a warning and an error and fails, unlike rst2html.py
The error is coming from here:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L142
It's happening because of this line:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L125
settings = frontend.OptionParser().get_default_values()
If this is changed to:
settings = frontend.OptionParser(components=(Parser,)).get_default_values()
then things work much better (this is how tools/quicktest.py does it for example)
so this might actually be a bug in distutils, but changing CPython (and maybe PyPy) is a more laborious and slow process, so I'm thinking it might be nice to work around this in docutils, if isn't too much of a burden. Docutils is on a much shorter release cycle than CPython, so changing docutils is a way to get this fix out to users much faster.
Here's a patch to docutils that solves the problem for me:
diff --git a/docutils/frontend.py.2014-12-15-070721 b/docutils/frontend.py index f837c62..7427e15 100644 --- a/docutils/frontend.py.2014-12-15-070721 +++ b/docutils/frontend.py @@ -41,6 +41,7 @@ import docutils import docutils.utils import docutils.nodes from docutils.utils.error_reporting import locale_encoding, ErrorOutput, ErrorString +from docutils.parsers import Parser def store_multiple(option, opt, value, parser, *args, **kwargs): @@ -561,7 +562,20 @@ class OptionParser(optparse.OptionParser, docutils.SettingsSpec): self.version = self.version_template # Make an instance copy (it will be modified): self.relative_path_settings = list(self.relative_path_settings) + from docutils.parsers.rst import Parser self.components = (self,) + tuple(components) + + # distutils.command.check does: + # settings = frontend.OptionParser().get_default_values() + # and as a result is missing important defaults like + # `syntax_highlight`, causing + # `python setup.py check --restructuredtext --strict --metadata` to + # fail with `warning: check: Could not finish the parsing.` if the RST + # document uses `code` or `code-block` directives. + if len(self.components) == 1: + from docutils.parsers.rst import Parser + self.components += (Parser,) + self.populate_from_components(self.components) self.set_defaults_from_dict(defaults or {}) if read_config_files and not self.defaults['_disable_config']:
Attaching
bug_270.patch, a patch to docutils that fixes this issue ...Last edit: marcman 2014-12-15
After some discussion on the mailing list with David Goodger, I am revising this a bit. He wanted it to check that it was being called by docutils before messing with anything.
I also created a Python issue (and patch) for this:
http://bugs.python.org/issue23063
but it would be nice still to add a workaround to docutils, so that folks don't need Python 3.5 for this to work.
I also created a branch and pull request on GitHub for this, in case folks find that easier to read and pull from:
https://github.com/msabramo/docutils/pull/1
It seems like this is fixed upstream now (no need to wait for 3.5):
http://bugs.python.org/msg234051
Thank you for the report(s), analysis, patch, and patience.