Re: [Docutils-users] Docutils-users Digest, Vol 143, Issue 1

[Guenter M. <mi...@us...>]

On 2020-09-24, Chris Sewell wrote:

> Indeed, the Docutils maintainers may be able to offer a better
> explanation, but IMO there are a few distinct aspects of Docutils to
> bare in mind:

> - The Docutils Abstract Syntax Tree (AST, aka doctree): this is the
>   structure that holds the document in an input/output agnostic format.
>   It has (in principal) nothing to do with the input format
>   reStructuredText/Markdown/etc, or the output format HTML/LaTeX/etc

The specification https://www.python.org/dev/peps/pep-0258/#document-tree
describes the `Document Tree` as a data structure is similar to a DOM tree,
but with specific node names (classes) instead of DOM's generic nodes.
The schema is documented in an XML DTD (eXtensible Markup Language
Document Type Definition) docutils.dtd__

__ http://docutils.sourceforge.net/docs/ref/docutils.dtd

> - The input parser, which converts a (text) file to the AST, of which
>   is the reStructuredText parser is the primary example. MyST does the
>   same for Markdown, and also “shares" reStructuredText’s
>   role/directive syntax extension system

Actually, the `parser` parses an input string presented to it by the
`reader` (it may come from file or other sources).

> - The output renderer, which converts the AST to the output format

In Docutils, this component is called a `writer`.

> What sphinx does is, in essence, extend the Docutils render process to
> better deal with multiple, inter-connected, documents (e.g.  a full
> website/book). For example, adding cross-document referencing and AST
> caching.

Spinx also provides a framework for several extensions, including 
at least two Markdown parsers (recommonmark and myst-parser).

These parsers should in principle also work with core Docutils. However,
I don't know of any implementation or documentation for this combination.

It would be a nice advancement, to have frontends similar to rst2...
that work with markdown input.

Another idea would be to add a ``:format:`` option to the "include"
directive that allows things like::

  .. include:: README.md
     :format: markdown
     
This could be an optional feature depending on an external markdown
parser that can be imported from Docutils. (Somewhat similar to
syntax-highlight with Pygments or some HTML-Math-Output options.)


Yet another idea is a Docutils-XML parser that would parse the native
Docutils XML format - allowing to store parsed documents as XML and
offers a gateway to the XML world (e.g. converting any input to Docutils
XML via XSLT).


Günter