#410 The 'document' attribute is None for many nodes

[Brecht Machiels]

While this doesn't seem to be documented, it seemed safe to assume that the document attribute on docutils nodes would hold a reference to the document node at the top of the document tree. However, this doesn't seem to be the case for several nodes, e.g. the figure and table nodes. Child nodes of these neither have their document attribute set.

It is possible to reach the document node by recursively following the parent attribute up to the top of the tree. However, it would be better if the document attribute could be depended on.

[Günter Milde]

While it might be sensible to assume the existence of some undocumented features, this is never safe -- especially in parts that are explicitely specified like the Docutils Document Tree. There is no "document" attribute in the doctree specification (docs/ref/doctree.txt, docs/ref/docutils.dtd). OTOH, most node-processing classes (like Writer and Transform and their descendents) as well as states.Inliner store a reference to the document beeing processed in a "document" attribute.

IMO, traveling up to find the top node is not so costly that it merits a permanent link to the top of the tree from every node.
If there is a convincing use case, please reopen as an enhancement request.

[Brecht Machiels]

Thanks for your quick reply, Günther.

I haven't been able to find explicit documentation on this. The Docutils Document Tree document is labeled as incomplete and it doesn't mention the parent attribute either. So should that attribute also be assumed unsafe, even though it's extensively used in the docutils code itself?

The only documentation I could find on these attributes was in the docutils/nodes.py source file, in the docstrings of the Node class definition. These do not make any distinction between the safeness of the document and parent attributes:

class Node(object):

    """Abstract base class of nodes in a document tree."""

    parent = None
    """Back-reference to the Node immediately containing this Node."""

    document = None
    """The `document` node at the root of the tree containing this Node."""

    source = None
    """Path or description of the input source which generated this Node."""

    line = None
    """The line number (1-based) of the beginning of this Node in `source`."""

Do you know of any other documentation resource I could reference?

[Günter Milde]

Thank you for the follow up. Now I understand the reason for the mixup.

docutils.nodes.Node class instances are used to represent Doctree elements. Both have attributes but these are completely different categories:

Node instances have class attributes.¹ They are documented in docstrings in the Python source. Your report was about the class attribute docutils.Node.document, right?

Doctree elements have element attributes that are described in doctree.html and docutils.dtd² . These element attributes are stored in Element.attributes and accessed as via class methods like Element.attlist() or as element[att].

|¹ Instances of the "Node" subclass "Element" have, amongst others, the class attribute "attributes" containing the element attributes.

|² While descriptions in "doctree.html" are incomplete, all elements and attributes at least have a stub entry, "docutils.dtd" should be complete.

With the cited docstring in docutils/nodes.py, the "document" class attribute becomes part of the API.
The "document" class attribute in any nodes.figure or nodes.table instance in a Docutils-generated document tree should be set to the document node. If this is not the case, this is a bug.

Unfortunately, the "document" class attribute is only auto-set for instances of nodes.document(). It is passed on for nodes generated with the "correct" helpers like nodes.Node.setup_child() or nodes.Node.copy().
This inheritance chain is easily broken.

In order to fix the bug we would need a "minimal working example",
because observations of <node_instance>.document == None do not depend on the node type but rather on how/where this instance is generated.
(It is also worth checking whether the error is with a 3rd-party extension.)

OTOH, the Node.document attribute seems not to be used in Docutils and maintaining its proper state adds complexity and is error prone.
I wonder whether it may be an improvement to either

• drop the attribute (after a deprecation warning for 3rd-party code),
• change the description adding a note that it is vital to set this for any new elements added to a doctree,
• change the description to tell that it may hold a reference to the document node,
• or to implement Node.document as a @property-decorated method that recurses up self.parent to fetch a nodes.document instance.

[Brecht Machiels]

Yes, my report is about the Node class (well, instance) attribute.

Since the parent instance attribute seems to be consistently set on all nodes, perhaps it's a good idea to go for the @property-based solution. This is easy to implement and adds useful functionality.

This issue reminds me fact that the source and line instance attributes that are missing (= None) in many nodes. I remember that that would require a lot of work to fix. Thankfully, that doesn't seem to be the case here.

[Günter Milde]

Fixed in r8589. Please test and close this bug if it solves the issue.

This issue reminds me fact that the source and line instance attributes that are missing (= None) in many nodes. I remember that that would require a lot of work to fix.

This is an ongoing effort.
Feel free to report concrete cases (with MWE, preferabely to the docutils-devel list).

[Brecht Machiels]

Yes, this seems to work as intended! Thanks, Günter.

[Günter Milde]

Fixed in [r8589]

[Günter Milde]

Fixed in Docutils 0.17.
Thanks again for the report.