Re: [Docutils-develop] Circular dependencies in rst.directives

[David G. <go...@py...>]

I was waiting to see what pearls would spring forth from
Docutils-develop's collective wisdom, but alas... no such luck.
So here goes ...

Edward Loper wrote:
 >   Second cycle:
 >
 >     - rst.states needs "directives" from rst.directives
 >     - rst.directives needs the individual directives in its
 >       submodules (to populate the directive registry)
 >     - The individual directives need several classes & functions
 >       from rst.states (eg SpecializedBody, SubstitutionDef, &
 >       escape2null)

Except for one case, all of the classes & functions accessed from
docutils.parsers.rst.states are within functions, so don't contribute
to the cycle.  I think the only case of a module-level access is in
directives/html.py:

     class MetaBody(states.SpecializedBody):

That whole class definition could be moved into another module and
imported from within the "meta" function.  That ought to break the
second cycle, right?

On the other hand, a single delayed import (option 2 below) seems
more elegant.

 > The directives package currently deals with these cycles by
 > initially populating the directives registry with *names* of the
 > individual directives, and delaying their import until they're
 > needed.  By that time, both the rst.states and the rst.directives
 > have been fully loaded.

I can't remember if this effect was an intentional design decision or
was accidental.  But it works.  :-)

 > But I'd rather refactor the package to get rid of the cycles (to the
 > extent possible).  I'd also like to use a system like rst.roles,
 > where a registration function (register_canonical_directive) is used
 > to register each directive at the place where it's defined, instead
 > of defining a single centralized list in rst.directives.

Why is it that you prefer that a registration function be called for
each directive?

 > The first cycle is fairly easy to get rid of:
 >
 >   - Create a new submodule rst.directives._directives (or
 >     .framework) that contains the basic framework for defining
 >     directives.

Call it rst.directives._core or ._infrastructure.  I like _core
better; it's shorter and easier.  Do we

 >   - Move the current contents of rst.directives.__init__ into
 >     rst.directives._directives
 >   - Change the directive subpackages to import the functions they
 >     need from rst.directives._directives (instead of from
 >     rst.directives)
 >   - Change rst.directives.__init__ to do:
 >
 >        # Import the basic framework.
 >        from docutils.parsers.rst.directives._directives import *

Why bother?  Everything will be importing directly from _core.
Is this for backwards compatibility?  Or to hide the fact that _core
is an implementation detail?

 > The second cycle is a little harder to break.  The easiest way is to
 > just accept the dependency cycle, and use a delayed import for
 > rst.directives.directive (the function used to look up the directive
 > for a given name).  I.e., change docutils.parsers.rst.states to
 > wait to import rst.directives.directive until it needs it:
 >
 >     +        # Late import, because of cyclic dependency:
 >     +        from docutils.parsers.rst.directives import directive
 >     -        directive_function, messages = directives.directive(
 >     +        directive_function, messages = directive(
 >                  type_name, self.memo.language, self.document)

That seems painless.

 > Another option is to actually break the cycle.  I don't know enough
 > about how the state machine works to know if this is reasonable to
 > do, but a brief look at the dependencies suggests that we would at
 > least need to divide rst.states into two modules, with Body in one
 > module and SpecializedBody, SubstitutionDef, and escape2null in
 > another module.
...
 > But from my brief look at the states package, that seems like an
 > unnatural place to divide up the module.

I'd rather not break up states.py.  If you look at the end of the
module, you'll see that there's a module global attribute
"state_classes", which several subclasses of SpecializedBody.  So
they'd have to be imported anyhow.

 > So...  Which option do people prefer?
 >
 >   1. Keep the directive package as it is, with delayed imports for
 >      individual directives.

If it ain't broke, don't fix it?  That's a fallback if there's no
obvious winner.

 >   2. Refactor the directive package to get rid of the first cycle,
 >      and use a delayed import to get rid of the second cycle.

This sounds like the right solution.

 >   3. Refactor both the directive package and the states module
 >      to get rid of both cycles.

I don't want to break up states.py.

 >   4. import the directive subpackages from rst, rather than from
 >      rst.directives, thus breaking both cycles

This feels unnatural to me; docutils.parsers.rst doesn't and shouldn't
care about the directive package.

-- 
David Goodger                               http://python.net/~goodger
For hire: http://python.net/~goodger/cv