Menu
▾
▴
#508 qemu build problem after docutils update to 0.22
After updating docutils in pkgsrc to 0.22 (with sphinx 8.2.3), qemu 10.0.3 (https://gitlab.com/qemu-project/qemu) stopped building. The output is not very clear:
[5596/6204] Generating docs/QEMU manual with a custom command
FAILED: [code=2] docs/docs.stamp
/scratch/emulators/qemu/work/.tools/bin/env CONFDIR=/usr/pkg/etc/qemu/qemu /scratch/emulators/qemu/work/qemu-10.0.3/build/pyvenv/bin/sphinx-build -q -j auto -Dversion=10.0.3 -Drelease= -Ddepfile=docs/docs.d -Ddepfile_sta
mp=docs/docs.stamp -b html -d /scratch/emulators/qemu/work/qemu-10.0.3/build/docs/manual.p /scratch/emulators/qemu/work/qemu-10.0.3/docs /scratch/emulators/qemu/work/qemu-10.0.3/build/docs/manual
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
Possible precedence problem between ! and pattern match (m//) at /scratch/emulators/qemu/work/qemu-10.0.3/docs/../scripts/kernel-doc line 1597.
/scratch/emulators/qemu/work/qemu-10.0.3/docs/../system/qtest.c:70: CRITICAL: Inconsistent title style: skip from level 1 to 3.
Valid requests
^^^^^^^^^^^^^^
Established title styles: =/= - ^ [docutils]
/scratch/emulators/qemu/work/qemu-10.0.3/docs/../system/qtest.c:73: CRITICAL: Inconsistent title style: skip from level 1 to 4.
10:17:35 [41/1829]
Clock management:
"""""""""""""""""
Established title styles: =/= - ^ " [docutils]
/scratch/emulators/qemu/work/qemu-10.0.3/docs/../system/qtest.c:107: CRITICAL: Inconsistent title style: skip from level 1 to 4.
PIO and memory access:
""""""""""""""""""""""
Established title styles: =/= - ^ " [docutils]
/scratch/emulators/qemu/work/qemu-10.0.3/docs/../system/qtest.c:215: CRITICAL: Inconsistent title style: skip from level 1 to 4.
IRQ management:
"""""""""""""""
Established title styles: =/= - ^ " [docutils]
/scratch/emulators/qemu/work/qemu-10.0.3/docs/../system/qtest.c:239: CRITICAL: Inconsistent title style: skip from level 1 to 4.
Setting interrupt level:
""""""""""""""""""""""""
Established title styles: =/= - ^ " [docutils]
Sphinx parallel build error!
Versions
========
* Platform: netbsd11; (NetBSD-11.99.1-amd64-x86_64-64bit-ELF)
* Python version: 3.13.5 (CPython)
* Sphinx version: 8.2.3
* Docutils version: 0.22
* Jinja2 version: 3.1.6
* Pygments version: 2.19.2
Last Messages
=============
reading sources... [ 91%]
system/s390x/protvirt .. system/target-loongarch
reading sources... [ 94%]
system/target-m68k .. system/target-sparc64
reading sources... [ 97%]
system/target-xtensa .. tools/qemu-nbd
Loaded Extensions
=================
* sphinx.ext.mathjax (8.2.3)
* alabaster (1.0.0)
* sphinxcontrib.applehelp (2.0.0)
* sphinxcontrib.devhelp (2.0.0)
* sphinxcontrib.htmlhelp (2.1.0)
* sphinxcontrib.serializinghtml (2.0.0)
* sphinxcontrib.qthelp (2.0.0)
* depfile (1.0)
* hxtool (1.0)
* kerneldoc (1.0)
* qapi_domain (1.0)
* qapidoc (2.0)
* qmp_lexer (unknown version)
* dbusdoc (1.0)
* sphinxcontrib.jquery (4.1)
* sphinx_rtd_theme (unknown version)
Traceback
=========
File "/usr/pkg/lib/python3.13/site-packages/sphinx/util/parallel.py", line 137, in _join_one
raise SphinxParallelError(*result)
sphinx.errors.SphinxParallelError: TypeError: unsupported operand type(s) for +: 'NoneType' and 'list'
The full traceback has been saved in:
/tmp/sphinx-err-gj6pt4nj.log
To report this error to the developers, please open an issue at <https://github.com/sphinx-doc/sphinx/issues/>. Thanks!
Please also report this if it was a user error, so that a better error message can be provided next time.
The full traceback from the log lists docutils code as the origin of the breakage:
Traceback
=========
(Error in parallel process)
Traceback (most recent call last):
File "/usr/pkg/lib/python3.13/site-packages/sphinx/util/parallel.py", line 83, in _process
ret = func(arg) # type: ignore[call-arg]
File "/usr/pkg/lib/python3.13/site-packages/sphinx/builders/__init__.py", line 603, in read_process
self.read_doc(docname, _cache=False)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/sphinx/builders/__init__.py", line 648, in read_doc
publisher.publish()
~~~~~~~~~~~~~~~~~^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/core.py", line 269, in publish
self.document = self.reader.read(self.source, self.parser,
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
self.settings)
^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/sphinx/io.py", line 103, in read
self.parse()
~~~~~~~~~~^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/readers/__init__.py", line 101, in parse
self.parser.parse(self.input, document)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/sphinx/parsers.py", line 86, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 165, in run
results = StateMachineWS.run(self, input_lines, input_offset,
input_source=document['source'])
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 235, in run
context, next_state, result = self.check_line(
~~~~~~~~~~~~~~~^
context, state, transitions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 447, in check_line
return method(match, context, next_state)
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2356, in explicit_markup
self.explicit_list(blank_finish)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2381, in explicit_list
newline_offset, blank_finish = self.nested_list_parse(
~~~~~~~~~~~~~~~~~~~~~~^
self.state_machine.input_lines[offset:],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
blank_finish=blank_finish,
^^^^^^^^^^^^^^^^^^^^^^^^^^
match_titles=self.state_machine.match_titles)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 312, in nested_list_parse
state_machine.run(block, input_offset, memo=self.memo,
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
node=node, match_titles=match_titles)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 191, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 235, in run
context, next_state, result = self.check_line(
~~~~~~~~~~~~~~~^
context, state, transitions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 447, in check_line
return method(match, context, next_state)
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2659, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2366, in explicit_construct
return method(self, expmatch)
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2103, in directive
return self.run_directive(
~~~~~~~~~~~~~~~~~~^
directive_class, match, type_name, option_presets)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 2153, in run_directive
result = directive_instance.run()
File "/scratch/emulators/qemu/work/qemu-10.0.3/docs/sphinx/qapidoc.py", line 620, in run
contentnode = self.transmogrify(schema)
File "/scratch/emulators/qemu/work/qemu-10.0.3/docs/sphinx/qapidoc.py", line 549, in transmogrify
nested_parse_with_titles(self.state, content, contentnode)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/sphinx/util/nodes.py", line 378, in nested_parse_with_titles
ret = state.nested_parse(content, content_offset, node, match_titles=True)
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 275, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
node=node, match_titles=match_titles)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 191, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 235, in run
context, next_state, result = self.check_line(
~~~~~~~~~~~~~~~^
context, state, transitions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/statemachine.py", line 447, in check_line
return method(match, context, next_state)
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 3024, in text
self.section(title.lstrip(), source, style, lineno + 1, messages)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 321, in section
self.new_subsection(title, lineno, messages)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/pkg/lib/python3.13/site-packages/docutils/parsers/rst/states.py", line 370, in new_subsection
self.parent += section_node
File "/usr/pkg/lib/python3.13/site-packages/docutils/nodes.py", line 736, in __radd__
return other + self.children
~~~~~~^~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'NoneType' and 'list'
Even with the "CRITICAL: Inconsistent title style" problems fixed, the build stops.
Any ideas what the problem is or how to fix it?
Thanks.
The reason is an incompatibility of Sphinx or one of the Sphinx extensions with the new section parsing algorithm introduced in commit [r10093] with updates in [r10129], [r10131], [r10131].
The unhelpful "TypeError" is displayed because Sphinx carries on after a CRITICAL problem which leads to follow-up problems (here,
self.parentbeeing None instead of an Element instance).The error reported by Docutils gives some hints. Let's dissect the first message:
It provides the source file and line (system/qtest.c, line 70) and a reason (
Inconsistent title style: skip from level 1 to 3.).How does it come to this conclusion:
The parser keeps a record of section adornment styles in the
memoattribute of the
documentobject. These are the "Established titlestyles" (under- and overline with
=for level 1, underlinewith
-for level 2, and underline with^for level 3.The new section header uses an underline with
^.Hence, the new section should be at level 3.
The current section level is determined scanning the parent elements
of the current element for
<section>elements.Level 1 means that one parent section was found.
As the level (depth) of a section element is determined from its physical nesting level, it is impossible to represent a level 3 section inside a level 1 section in the doctree.
Why does the Docutils update trigger this problem?
The source is a docstring in a C file. Some extension must have been used to extract it and include it (together with other docstrings) in a wrapper document.
When including a source, there are two strategies to handle section adornment styles:
1. enforce a consistent style hierarchy across all included parts,
2. establish a new style hierarchy for the included parts.
It seems as if with previous Docutisl versions there was strategy 2 and now its stratagy 1.
This is most likely not intended but a side effect of the changes to the inner working of the parser.
To fix it, we would need to know which extension is used to extract the docstrings and cooperate with its developers.
Related
Commit: [r10093]
Commit: [r10129]
Commit: [r10131]
Last edit: Günter Milde 2025-08-19
Thank you for the analysis!
When building qemu, I have the following sphinx packages installed:
sphinxcontrib-applehelp-2.0.0
sphinxcontrib-devhelp-2.0.0
sphinxcontrib-htmlhelp-2.1.0
sphinxcontrib-jsmath-1.0.1
sphinxcontrib-qthelp-2.0.0
sphinxcontrib-serializinghtml-2.0.0
sphinx-8.2.3
sphinxcontrib-jquery-4.1
sphinx-rtd-theme-3.0.2
I think there might be second problem though. When I remove the whole comment that causes the CRITICAL errors, i.e. use the following diff:
then the CRITICAL warnings are gone, but the build still fails.
The traceback from the log file:
looks to me the same as before. So it looks like the CRITICAL errors are not the root cause of this.
You are right. As also seen in [#509], the CRITICAL errors are just on top of the problem (and helped to find the reasong beeing some "docstring reading" extension (plus docstrings that include section headings) beeing incompatible with Docutils 0.22.
(Unfortunately, the new section algorithm was only tested with Sphinx, not with Sphinx extensions and docstrings containing section headings.)
Related
Bugs:
#509What is strange, though is that Sphinx' autodoc extension does not have a problem with the new section parsing algorithm. Which extension does the extraction of docstrings rsp. the combination of the docstrings into a document?
Could you re-try with [r10197] or the attached patch if this gets a more telling log and maybe some output?
Related
Commit: [r10197]
Last edit: Günter Milde 2025-08-19
I've tried the attached patch, and with it, the qemu build succeeds, even without the patch I included above.
There are still a lot of warnings:
but they don't break the build.
Thank you!
Yes, Docutils now checks, whether the new "self.parent" after switching the section level is valid and reports these CRITICAL system messages (downgraded to ERROR in [r10198].
However, there is still a serious problem: an incompatibility with "qapidoc" rsp
sphinx.util.nodes.nested_parse_with_titles().This should be fixed in [r10200].
Can you re-try?
If it still fails, please run with a "docutils.conf" file in the project directory that sets
halt_level to
3?For added robustness, I recommend that "qapidoc" switches from using
sphinx.util.nodes.nested_parse_with_titles()tosphinx.util.parsing.nested_parse_to_nodes()as recommended in the docstring of nested_parse_with_titles();( The difference is, that
nested_parse_to_nodes()does not have anodeargument but instead adds the new nodes to a "dummy node" and finally just returnes its children.)Related
Commit: [r10198]
Commit: [r10200]
Last edit: Günter Milde 2025-08-11
I added the second patch (r10200) as well, and qemu still builds fine. The warnings are now:
Did you check the generated HTML for missing section headings?
While the build runs without crashing, there are still serious errors.
Unfortunately, I cannot replicate or trace the reason.
Here is what I found so far:
The error is in docstrings that are collected by a Sphinx extension (qapidoc?).
The error message tells, that the parser expects a consistent hierarchy of title adornment styles (i.e. sections in docstrings under/overlined with
*, subsections underlined with-). Docstrings using this style (e.g. /scratch/emulators/qemu/work/qemu-10.0.3/qapi/pci.json) don't report an error any more.The mystery is, that Sphinx "autodoc" and (as far as I can see in the source) "qapidoc" set up a "fresh" title style hierarchy for docstrings (with the auxiliary functions
nested_parse_with_titles()rsp.nested_parse_to_nodes()).Nevertheless, the docstrings in
include/hw/qdev-core.handsystem/qtest.care parsed with a non-empty section title style hierarchy.The traceback of a run with a "docutils.conf" file in the project directory that sets
halt_level to 3 may give some hints.
As a workaround, you may consider "normalizing" the section title adornments in
include/hw/qdev-core.handsystem/qtest.c.The docstrings in this case are not from the qapidoc sphinx extension, but from docs/sphinx/kerneldoc.py (which we have borrowed from the Linux kernel's sphinx documentation).
Looking at
docs/sphinx/kerneldoc.py, I see that it uses Docutil'sRSTState.nested_parse()which (up to now) forces a consistent title style hierarchy.OTOH, quapidoc uses Sphinx's
nested_parse_with_titles()that starts a new title style hierarchy.Could you check, if the remaining CRITICAL errors also show up with Docutils 0.21?
I've now reported this to the qemu project at
https://gitlab.com/qemu-project/qemu/-/issues/3077
I am working on a fix...
The precedence errors are there with docutils 0.21 as well:
but none of the CRITICAL ones.
Last edit: Thomas Klausner 2025-08-15
Those "possible precedence problem" warnings are from the kernel-doc perl script -- I guess that you're using a new version of Perl which emits the warnings. They shouldn't be relevant to this docutils issue, I think.
(We're going to upgrade our kernel-doc script to the kernel's current one, which is a complete rewrite in Python; that will fix these warnings as a side effect.)
Could you re-try with a checkout or snapshot of [r10204] that introduces independent section title style hierarchies for nested parsing?
Related
Commit: [r10204]
I applied all changes since the 0.22 release to my copy (except the version bump in init.py) There are no CRITICAL errors, the section of the build looks like this:
Good news.
The remaining warnings are from a different issue (see the comment by Peter Maydell).
I will do some more testing and cleanups. Then we can release a new Docutils version.
Thank you for report and testing.
This should be fixed in [r10227]. Could you re-try?
Related
Commit: [r10227]
Just to confirm, with the code changes between 0.22 and r10233 applied to 0.22, qemu still builds.
Thank you for testing. so it should be fine with Docutils 0.22.1.rc1.
Fixed in Docutils 0.22.1.
Please reopen if there are still problems.
Thank you for reporting and tests.