#41 Not all Node instances have the source and line attributes set

[Brecht Machiels]

The source and line attributes of Node elements are not set for all types of elements. For example, these attributes never get set on bulleted or enumerated list nodes. Other elements (such as literal block) don't have source set depending on the context.

[Günter Milde]

Ticket moved from /p/docutils/bugs/245/

[Günter Milde]

Source and line attributes of list items are not set, but coincide with the source and line attributes of the first child.

When do literal blocks not have the internal source and line arguments?

[Brecht Machiels]

I have had another look at adding these for all elements. I think I managed to set the source and line for all inline elements with a small addition to Inliner.parse().

However, due to the way docutils works, this doesn't seem to be possible for body elements; they need to be set in each method that corresponds to a body element. That would introduce a lot of duplication, so it would be better to refactor this, but I'm not sure how to go about this. Thoughts and suggestions welcome!

I'm not yet sure about directives. They might also allow for a more centralized approach.

[Günter Milde]

Could you attach the patch for inline elements, please?

Also, do these set the correct source line or the start line of the containing paragraph?
(See #145 error report with paragraph start line instead of error-line.
https://sourceforge.net/p/docutils/bugs/145/)

[Brecht Machiels]

The patch for inline elements is basically the following code inserted before the return statement of Inliner.parse():

        for node in processed:
            node.source = memo.document['source']
            node.line = lineno

I think lineno here is indeed the start line of the containing paragraph.

[Günter Milde]

I think lineno here is indeed the start line of the containing paragraph.

In this case, I would rather keep inline nodes as-is. Inline nodes cannot be from a different source to their parent block-level node, as the "include" directive is a block-level syntax construct without inline alternative.
Source/line info can be easily retrieved from <inline_node_instance>.parent (or .parent.parent... in case we get nested inline some day). See the definition of the nodes.Node.document property for a simple way to do this.

[Brecht Machiels]

For directives, the best I can come up with is to rename all methods that override Directive.run() to "_run" and calling this from Directive.run() which takes care of setting the source and line attributes on the nodes returned by _run().

For body elements, there is no function that returns the nodes. The code directly appends nodes to the parent object, so there is no easy way to set source/line attributes for them. How would you feel about refactoring this to make this possible? Or would you rather set the source/line separately for each body element?

David, Günther, any comments?

[Günter Milde]

Unfortunately, any major refactoring has the potential of breaks for downstream users (Sphinx, Sphinx extensions, alternative wrappers) that tend to take the current implementation as granted.
The reaction to the 0.18 release showed that we still have
insufficient awareness of announced changes and testing of beta versions.
This means any backwards-incompatible refactoring will better wait until
Docutils 2.0 :-(

[Brecht Machiels]

My conclusion is that implementing this properly will require significant refactoring, and hence will take a significant amount of work.

[Günter Milde]

On 2021-10-24, Mickey Endito sent a comprehensive patch-set to docutils-develop.

[Günter Milde]

See also [bugs:#438] "The source info of topic node is not filled when topic directive only contains a bullet list" and [bugs:#449] "Image has no line number with an :alt: attribute".

[Adam Turner]

Also potentially [bugs:#145]

[Günter Milde]

Also potentially [bugs:#145]

No, "#145 error report with paragraph start line instead of error-line" can/must be solved independently.

[Adam Turner]

Should we close 438 and 449 as duplicates of this?

A

[Günter Milde]

[r9475] adds source/line info to <admonition> and <definition_list> nodes.
See https://docutils.sourceforge.io/test/test_parsers/test_rst/test_source_line.py for the state of the art.