docutils-develop

[Docutils-develop] Writing directives

[Aahz <aa...@py...>]

Here's some sample reST text::

    Overall, my primary source for determining what to include in this book
    was the netnews group ``comp.lang.python``.  I've been reading that
    group for several years, and the same questions seem to show up over and
    over again.

    .. index::
        comp.lang.python
        Usenet
        netnews

What am I doing wrong with this directive::

    from docutils import nodes
    from docutils.parsers.rst import directives

    registry = directives._directive_registry
    registry['index'] = ('OOdirectives', 'index_directive')

    class index_entry(nodes.General, nodes.Element): pass

    def index_directive(name, arguments, options, content, lineno,
            content_offset, block_text, state, state_machine):
        text = '\n'.join(content)
        text_nodes, messages = state.inline_text(text, lineno)
        node = index_entry(text, '', *text_nodes)
        return [node] + messages

    index_directive.content = 1

If I replace the code of index_directive() with "return []", I don't get
an exception.  With the above code (ripped directly from
docutils.parsers.rst.directives.body.line_block()), I get the following
traceback::

    AttributeError: 'str' object has no attribute 'parent'
    input line 9
    module /home/aahz/src/Lib-Python/docutils/nodes.py, line 72, function setup_child
    Traceback (most recent call last):
      File "publish.py", line 27, in ?
        publish(description=description)
      File "/home/aahz/src/Lib-Python/docutils/core.py", line 161, in publish
        pub.publish(argv, usage, description, option_spec)
      File "/home/aahz/src/Lib-Python/docutils/core.py", line 135, in publish
        document = self.reader.read(self.source, self.parser, self.options)
      File "/home/aahz/src/Lib-Python/docutils/readers/__init__.py", line 69, in read
        self.parse()
      File "/home/aahz/src/Lib-Python/docutils/readers/__init__.py", line 77, in parse
        self.parser.parse(self.input, document)
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/__init__.py", line 89, in parse
        self.statemachine.run(inputlines, document, inliner=self.inliner)
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/states.py", line 163, in run
        results = StateMachineWS.run(self, input_lines, input_offset)
      File "/home/aahz/src/Lib-Python/docutils/statemachine.py", line 218, in run
        context, next_state, result = self.check_line(
      File "/home/aahz/src/Lib-Python/docutils/statemachine.py", line 400, in check_line
        return method(match, context, next_state)
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/states.py", line 1885, in explicit_markup
        nodelist, blank_finish = self.explicit_construct(match)
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/states.py", line 1897, in explicit_construct
        return method(self, expmatch)
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/states.py", line 1669, in directive
        return self.parse_directive(
      File "/home/aahz/src/Lib-Python/docutils/parsers/rst/states.py", line 1751, in parse_directive
        block_text, self, self.state_machine)
      File "OOdirectives.py", line 17, in index_directive
        node = index_entry(text, '', *text_nodes)
      File "/home/aahz/src/Lib-Python/docutils/nodes.py", line 252, in __init__
        self.extend(children)           # maintain parent info
      File "/home/aahz/src/Lib-Python/docutils/nodes.py", line 418, in extend
        self.setup_child(node)
      File "/home/aahz/src/Lib-Python/docutils/nodes.py", line 72, in setup_child
        child.parent = self
    AttributeError: 'str' object has no attribute 'parent'

Re: [Docutils-develop] Writing directives

[David G. <go...@us...>]

Aahz wrote:
> Here's some sample reST text::
> 
>     Overall, my primary source for determining what to include in
>     this book was the netnews group ``comp.lang.python``.  I've been
>     reading that group for several years, and the same questions
>     seem to show up over and over again.
> 
>     .. index::
>         comp.lang.python
>         Usenet
>         netnews
> 
> What am I doing wrong with this directive::
> 
>     from docutils import nodes
>     from docutils.parsers.rst import directives
> 
>     registry = directives._directive_registry
>     registry['index'] = ('OOdirectives', 'index_directive')

You must be importing this module explicitly somewhere, right?
Otherwise, how does the registration code run before the directive is
registered? ;->

>     class index_entry(nodes.General, nodes.Element): pass

This is the problem.  Replace "nodes.Element" with
"nodes.TextElement".  Their __init__ signatures are different.

>     def index_directive(name, arguments, options, content, lineno,
>             content_offset, block_text, state, state_machine):
>         text = '\n'.join(content)
>         text_nodes, messages = state.inline_text(text, lineno)
>         node = index_entry(text, '', *text_nodes)
                                   ^^
Right here, the code is sending a string ('') as a child node.

>         return [node] + messages
> 
>     index_directive.content = 1

I'm guessing that this is a "first draft" of the directive, yes?
You'll probably want to create an index_entry for each of the terms
listed in the directive, and insert them at the beginning of the
paragraph (therefore "index_entry" should be an inline element).  In
fact, the class declaration should probably be::

    class index_entry(nodes.Inline, nodes.TextElement): pass

> If I replace the code of index_directive() with "return []", I don't
> get an exception.

That's odd.  Are you sure you made no other changes?  This is where
the error originates:

>       File "OOdirectives.py", line 17, in index_directive
>         node = index_entry(text, '', *text_nodes)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Writing directives

[Aahz <aa...@py...>]

On Thu, Oct 10, 2002, David Goodger wrote:
> Aahz wrote:
>> 
>> What am I doing wrong with this directive::
>> 
>>     from docutils import nodes
>>     from docutils.parsers.rst import directives
>> 
>>     registry = directives._directive_registry
>>     registry['index'] = ('OOdirectives', 'index_directive')
> 
> You must be importing this module explicitly somewhere, right?
> Otherwise, how does the registration code run before the directive is
> registered? ;->

Of course.  I'm running a hacked publish.py currently to validate the
structure.  Next I'll be following your directions to Michael Hudson to
generate the actual OpenOffice.org XML (needs to go into a string so
that it can all get shoved into a ZIP file with the metadata).

>>     class index_entry(nodes.General, nodes.Element): pass
> 
> This is the problem.  Replace "nodes.Element" with
> "nodes.TextElement".  Their __init__ signatures are different.

Got it.  I was getting confused by the inheritance tree in nodes.py.
I'm still not sure that's the route I want to take, but it'll do for
now.

>>         return [node] + messages
>> 
>>     index_directive.content = 1
> 
> I'm guessing that this is a "first draft" of the directive, yes?
> You'll probably want to create an index_entry for each of the terms
> listed in the directive, and insert them at the beginning of the
> paragraph (therefore "index_entry" should be an inline element).  In
> fact, the class declaration should probably be::
> 
>     class index_entry(nodes.Inline, nodes.TextElement): pass

It appears that simply declaring something Inline doesn't do anything;
I'm assuming I need to run a transform in the writer?  Or should the
directive find its parent/predecessor-sibling during parsing?

>> If I replace the code of index_directive() with "return []", I don't
>> get an exception.
> 
> That's odd.  Are you sure you made no other changes?  

Yes, that line replaces all four lines of code, so the node wasn't
getting created.  ;-)  I was actually starting to think that the node
creation might be the problem before you responded, but I was at the
point where staring at the code gave me brain-burn.
-- 
Aahz (aa...@py...)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/

Re: [Docutils-develop] Writing directives

[David G. <go...@us...>]

Aahz wrote:
> It appears that simply declaring something Inline doesn't do
> anything;

Correct.  It's just for classification purposes, so that later
transforms can query a node with isinstance().

> I'm assuming I need to run a transform in the writer?

No.  By the time the document gets to the Writer, it should be in
final form.  The Writer's job is simply (and only) to translate from
the Docutils doctree structure to the target format.  Some small
transforms may be required, but they should be local and
format-specific.  The "index_entry" node class is not format-specific
and should become a standard Docutils node/element (i.e. part of
nodes.py and docutils.dtd).  We can figure out its syntax (attributes
& content model) later.

> Or should the directive find its parent/predecessor-sibling during
> parsing?

I think so.  Given::

    Here's a paragraph.

    .. index:: paragraph

The directive should produce something like this XML::

    <paragraph>
    <index_entry text="paragraph"/>
    Here's a paragraph.
    </paragraph>

Thus the "nodes.Inline" base class; it's an inline element.  This kind
of content model would also allow true inline index-entries::

    Here's a `paragraph`:index:.

If the "index" role were the default for the application, it could be
dropped::

    Here's a `paragraph`.

Both of these would result in this XML::

    <paragraph>
    Here's a <index_entry>paragraph</index_entry>.
    </paragraph>

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Writing directives

[Aahz <aa...@py...>]

On Fri, Oct 11, 2002, David Goodger wrote:
> Aahz wrote:
>>
>> I'm assuming I need to run a transform in the writer?
> 
> No.  By the time the document gets to the Writer, it should be in
> final form.  The Writer's job is simply (and only) to translate from
> the Docutils doctree structure to the target format.  Some small
> transforms may be required, but they should be local and
> format-specific.  The "index_entry" node class is not format-specific
> and should become a standard Docutils node/element (i.e. part of
> nodes.py and docutils.dtd).  We can figure out its syntax (attributes
> & content model) later.
> 
>> Or should the directive find its parent/predecessor-sibling during
>> parsing?
> 
> I think so.  Given::
> 
>     Here's a paragraph.
> 
>     .. index:: paragraph
> 
> The directive should produce something like this XML::
> 
>     <paragraph>
>     <index_entry text="paragraph"/>
>     Here's a paragraph.
>     </paragraph>

Given my lack of knowledge about XML and reST internals, where should I
look for information about writing this part of the directive?  That is,
how do I go about correctly inserting a node into the tree during
directive processing?  (Instead of returning a list of nodes.)

BTW, I was able to successfully write my ``include-output`` directive
quite easily (this is the one that includes the output of a Python
script).  Aside from my difficulties with interacting with reST
internals, I'm pretty happy with the current system for writing
directives.  The HOW-TO has been quite helpful.
-- 
Aahz (aa...@py...)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/

Re: [Docutils-develop] Writing directives

[David G. <go...@us...>]

[David]
>> Given::
>> 
>>     Here's a paragraph.
>> 
>>     .. index:: paragraph
>> 
>> The directive should produce something like this XML::
>> 
>>     <paragraph>
>>     <index_entry text="paragraph"/>
>>     Here's a paragraph.
>>     </paragraph>

[Aahz]
> Given my lack of knowledge about XML and reST internals, where should I
> look for information about writing this part of the directive?

Just the source for now.  This is the kind of thing that transforms
usually do.  For example, docutils.transforms.peps.Contents inserts a table
of contents placeholder as the second element in the document (after the PEP
field list)::

    self.document.insert(1, pending)

The parts.SectNum tranform inserts section numbers (in "generated" elements)
into section titles.  The frontmatter.DocTitle transform is another good
example.

> That is, how do I go about correctly inserting a node into the tree
> during directive processing?  (Instead of returning a list of nodes.)

Here's some quick & dirty code, based on what's in OOdirectives.py::

    def index_directive(name, arguments, options, content, lineno,
                        content_offset, block_text, state,
                        state_machine):
        nodes = []
        for entry in content:
            nodes.append(index_entry(entry, entry))
        parent = state_machine.node
        last_node = parent[-1]  # add check for empty parent later
        assert isinstance(last_node, nodes.paragraph), \
            ('The "index" directive is only valid after a paragraph, '
             'for now.') # must generalize it later
        last_node.insert(0, nodes)
        return []

It doesn't handle the general case yet.  Left as an exercise...

> BTW, I was able to successfully write my ``include-output`` directive
> quite easily (this is the one that includes the output of a Python
> script). 

Please check in the code!

> Aside from my difficulties with interacting with reST
> internals, I'm pretty happy with the current system for writing
> directives.  The HOW-TO has been quite helpful.

Glad to hear it.  Thanks go to Dethe Elza for getting the ball rolling.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/