[Docutils-develop] Custom node types (was Re: pdf writer)

[David G. <go...@us...>]

Tony J Ibbs (Tibs) wrote:
> Let us imagine that we have a product-specific tool (for instance, to
> pick a random example(!) pysource, or perhaps a Wiki tool), which
> generates a DPS tree, and wishes to use a Writer to output
> the results.
>
> The tree-construction part of the tool has two basic choices:
>
> 1. work entirely with the existing DPS nodes, to render what
>    it wants to do (hint: I think this is the right approach)
>
> 2. possibly create new node types specific to its application
>    area (which I believe David has advocated for pysource, at
>    least, in the past).
>
> *If* (2) is the "proper" approach,

What I've advocated in the past (or meant to, anyhow) is a blend of #1
and #2.  I'll describe the implementation.  Introducing custom node
types is OK, as long as:

(a) a transform resolves them to standard Docutils nodes before they
    reach the Writer proper, or

(b) the custom node is explicitly supported by certain Writers, and is
    wrapped in a filtered ``pending`` node.

In the case of PySource, an example of (a), I would expect to have a
transform reduce any custom nodes to standard node structures; tables
may suit the current code.  Then a standard HTML writer, perhaps in
conjunction with a specialized stylesheet, could produce HTML very
different from that produced from standalone reStructuredText.  (Not
having worked though the code yet, I wouldn't be surprised if this
isn't enough.  That's OK; we'll fix it in time.)

The HTML <meta> tag is an example of (b); the *only* example,
currently.  The ``.. meta::`` directive creates a ``pending`` node,
which contains knowledge that the embedded ``meta`` node can only be
handled by HTML-compatible writers.  The ``pending`` node is resolved
by the ``transforms.components.Filter`` transform, which checks that
the calling writer supports HTML; if it doesn't, the ``meta`` node is
removed from the document.

The Writer itself works entirely with existing Docutils nodes, and any
nodes specific to its format.  Readers know about input contexts
(PySource, PEP, standalone file, etc.), but Writers are intentionally
ignorant of context.

> then we automatically have the issues that Engelbert Gruber is
> concerned about - what does a Writer do when it encounters nodes it
> does not recognise?

It raises a "NotImplementedError" exception.  It is an error for a
Writer to encounter an unknown node.  It might not be the Writer's
fault though.

> It seems to me that the counter-argument that anyone who invents
> such nodes must amend any Writers they "care about" is not a
> sufficient answer

"Care about" doesn't enter into it.  The requirements are simple: all
Writers must handle all standard Docutils nodes, and any non-standard
nodes not explicitly supported by certain Writers must be transformed
into standard nodes or removed.  Whenever new standard nodes are
introduced *all* Writers *must* be updated.

> I have two "obvious" counter-examples:
>
> i.  the author of the Reader phase may not have the time or
>     ability (or permission, even) to alter the Writer.

That's why the API has to be well-defined and components have to be
decoupled.  We want the Writers to be as independent of the Readers as
possible.

> ii. given how simple it is to write XML out from docutils (in
>     fact, the capability is already provided), and also, to
>     read it back in (not provided, but trivial to do), there
>     is no particular need for Reader and Writer to be in the
>     same tool.

Except that there's a lot of internal data that doesn't get stored
with the XML, and will need to be recreated by the Writer-equivalent.
The ``nodes.document`` object (the root of a Docutils document tree)
stores a lot of details.  The consumer of the XML would have to be
quite sophisticated (like a web browser, which can resolve links).
It's quite possible that there would be some data loss; I couldn't say
without auditing the code (it's a tad hairy).

> On the other hand, since all of David's current nodes "declare" (by
> inheritance) what sort of entity they are,

This is meant for transforms to use to identify nodes.  The transforms
in ``transforms.frontmatter`` skip nodes descended from
``nodes.PreBibliographic`` (title, comment, system_message, etc.).
The ``transforms.parts.Contents`` transform searches for nodes that
are instances of ``nodes.section``, including *subclasses* (which
opens the door for custom sections).

> it should be possible (note my hands waving vaguely in the air) to
> make sensible "fall back" code for any future nodes, whether they
> are added "officially" or not.

Except for interim, under development code, I don't think this is a
good idea.

> (Briefly, my reason for liking option (1) is that with the single
> addition of a new "style" attribute to all nodes, I can get 80% of
> what I want ... and "style" translates, in HTML, to "class"

There's already a "class" attribute on all nodes, which remains on the
HTML generated from the node.  For example, see how the ``topic`` node
is handled in ``transforms.parts.Contents``.

> and with a new node called "group", I get 100% - where "group"
> translates, in HTML, to <DIV> or <SPAN> as appropriate

I think there's a danger in an overly generic node like "group", which
is why I'm resisting.  If you look (once again) at
http://docutils.sf.net/spec/pysource.dtd, you'll see my first cut at a
structure for representing the custom Python-source-related nodes
needed (probably out of date).  Look at the "Additional Structural
Elements" section.  Each of the ``..._section`` elements have a
different structure, composed of custom child elements.  How are you
going to represent all of those with a single "group"?  Especially
when different views of the data (different styles) will probably be
required?

But since I haven't gone through the pysource code, my arguments may
not hold water.  My gut says "group" is an evil generalization.
Perhaps my head just needs to see it in action to override my gut.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/