[Docutils-develop] [docutils:bugs] #511 Problems with nested parsing and sections.

[Günter M. <mi...@us...>]

Element after a section from nested parsing may be invalid.
    
`parsers.rst.RSTSTate.nested_parse()` with `match_titles=True` (i.e. support for sections) leads to an invalid document tree, if the nested block contains a section but the element following the nested block is not a section.
    
The [structure model](https://docutils.sourceforge.io/docs/ref/doctree.html#structure-model) ony allows a `<section>` as sibling after a `<section>`.

An invalid doctree can be prevented if the following content is appended to the last nested section instead of its parent. The "nested" directive attempts this but fails if it is called from another nested state machine: In a nested state_machine we cannot access/change the `node` attribute ("insertion point") of the calling state_machine (cf. [r10222]).
A  fix using a new attribute to store the parent state machine of nested state machines is attached.


Attachments:

- [0001-New-attribute-to-store-the-parent-state-machine-of-n.patch](https://sourceforge.net/p/docutils/bugs/_discuss/thread/4184882201/239b/attachment/0001-New-attribute-to-store-the-parent-state-machine-of-n.patch) (6.0 kB; text/x-patch)


---

**[bugs:#511] Problems with nested parsing and sections.**

**Status:** pending-remind
**Created:** Tue Aug 26, 2025 10:13 AM UTC by Günter Milde
**Last Updated:** Tue Aug 26, 2025 10:25 AM UTC
**Owner:** nobody
**Attachments:**

- [old_nested_parsing.py](https://sourceforge.net/p/docutils/bugs/511/attachment/old_nested_parsing.py) (7.5 kB; text/x-python)


In Docutils <= 0.22, parsing a nested content block with `docutils.parsers.rst.state.RSTState.nested_parse()` uses the document-wide "title style hierarchy".

With Docutils <=0.21, this method can **loose complete sections** without warning when the `match_titles` argument is True and the nested content block contains sections with a title style that matches a lower section level than the current section level (try the attached test with a Docutils version below 0.22).

Docutils itself does not use `nested_parse()` with `match_titles` True and did not test.
However, the feature is used by Sphinx and several Sphinx extensions (cf. [bugs:#508], [bugs#509], and  https://github.com/sphinx-doc/sphinx/issues/13845).

The new section parsing algorithm introduced in Docutils 0.22 fixes the data loss, but  sections with a title style that matches a lower section level than the current section level are attached in wrong order: After the nested parsing is complete, the calling parser continues where it left of, messing the order of elements in the doctree (try the attached test with [r10204]).

This was fixed in [r10206] at the expense of dropping support for document-wide title styles in nested parsing. 
As a result, the Sphinx "only" directive (which uses a new section style hierarchy with later re-attachment of the first section) now fails: https://github.com/sphinx-doc/sphinx/issues/13845#issuecomment-3209397215.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.