Re: [Docutils-users] Migration docutils 0.12 to 0.13

[David G. <go...@py...>]

On Mon, Jan 2, 2017 at 5:03 PM, Jean Baptiste Favre
<web...@jb...> wrote:
> Still have the same error.
> The full traceback is attached, sorry to have forgotten it before.
>
> My subclass is as follow:
>
> from docutils import nodes
> from docutils.parsers.rst import states
> from docutils.utils import unescape
>
> # Customize parser.inliner in the only way that Sphinx supports.
> # docutils.parsers.rst.Parser takes an instance of states.Inliner or a
> # subclass but Sphinx initializes it from
> # SphinxStandaloneReader.set_parser('restructuredtext') which is called
> # from Publisher.set_components() and initializes the parser without
> # arguments.
>
> BaseInliner = states.Inliner
> class Inliner(BaseInliner):
>
>   def init_customizations(self, settings):
>     BaseInliner.init_customizations(self, settings)
>
>     issue_pattern = re.compile(u'''
>       {start_string_prefix}
>       TS-\d+
>       {end_string_suffix}'''.format(
>         start_string_prefix=self.start_string_prefix,
>         end_string_suffix=self.end_string_suffix),
>       re.VERBOSE | re.UNICODE)
>
>     self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>
>   def issue_reference(self, match, lineno):
>     text = match.group(0)
>
>     rawsource = unescape(text, True)
>     text = unescape(text, False)
>
>     refuri = 'https://issues.apache.org/jira/browse/' + text
>
>     return [nodes.reference(rawsource, text, refuri=refuri)]
>
> states.Inliner = Inliner

That last line is the culprit. You're monkey-patching Docutils. Don't
do that. If something in Docutils breaks as a result, you can't
reasonably complain. If it worked before, it was just accidental.

I distilled the issue down, see the attached x.py & y.py. Put them on
your PYTHONPATH so y.py can import x.py. Run y.py. Note the output.
Now edit y.py and uncomment the commented line (the monkey patch). Now
run y.py again, and note the difference in the output. The monkey
patch messes up the namespace hierarchy, so x.A.run doesn't have
access to x.attr_x anymore.

Wasn't there another version of this code that worked in a different way?

Maybe you can monkey patch at the method level instead of the class
level. I.e. inject your methods into the existing Docutils Inliner
class, instead of replacing the class with your subclass. See z.py.
Caveat: monkey patching is dangerous. Even if this works now, I can't
guarantee that it will always work.

You're trying to do this through Sphinx, which isn't configurable, as
the comment in https://github.com/apache/trafficserver/blob/master/doc/conf.py
describes:

# Customize parser.inliner in the only way that Sphinx supports.
# docutils.parsers.rst.Parser takes an instance of states.Inliner or a
# subclass but Sphinx initializes it from
# SphinxStandaloneReader.set_parser('restructuredtext') which is called
# from Publisher.set_components() and initializes the parser without
# arguments.

Perhaps you should complain to Sphinx, and ask them to enable
configurability. If your code was using Docutils directly this
wouldn't be an issue (or would be less of an issue).

David Goodger
<http://python.net/~goodger>


> On 02/01/2017 22:53, David Goodger wrote:
>> On Mon, Jan 2, 2017 at 3:26 PM, Jean Baptiste Favre
>> <web...@jb...> wrote:
>>> Hello David,
>>>
>>> Thanks for your answer.
>>> Unfortunatly, the patch doesn't help.
>>>
>>> I still have a "KeyError: 'non_unescaped_whitespace_escape_before'" in
>>> docutils/parsers/rst/states.py at line 533.
>>
>> In future, please send a full traceback. We need to know the context
>> (what called what).
>>
>>> I guess it's the same kind of problem and I should redefine all local
>>> attributes (from lines 655 to 687) as class attributes as you did in the
>>> patch.
>>
>> I think I see what the problem is now. docutils/parsers/rst/states.py,
>> Inliner.init_customizations contains the following line::
>>
>>     args.update(vars(self.__class__))
>>
>> The "vars" function gets the __dict__ from the object's class. But
>> you're using a subclass, so vars returns the __dict__ of the subclass,
>> without the the superclass's attributes, which is what is needed. It's
>> a bit of metaprogramming gone wrong. It's not safe for subclassing,
>> which is a bug. It's already a bit of a kludgey shortcut. We could
>> replace it with::
>>
>>     args.update(vars(Inliner))
>>
>> That's a bit smelly, but I see it's done elsewhere in the code. Try
>> making that change (patch attached).
>>
>> We may have to explicitly refer to all the class attributes. I'll think on it.
>>
>> David Goodger
>>
>>> I'll test it asap and report,
>>> Cheers,
>>> Jean Baptiste
>>>
>>>