docutils-users — General discussions, questions, help requests.

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[David G. <go...@py...>]

On Mon, Jan 2, 2017 at 3:26 PM, Jean Baptiste Favre
<web...@jb...> wrote:
> Hello David,
>
> Thanks for your answer.
> Unfortunatly, the patch doesn't help.
>
> I still have a "KeyError: 'non_unescaped_whitespace_escape_before'" in
> docutils/parsers/rst/states.py at line 533.

In future, please send a full traceback. We need to know the context
(what called what).

> I guess it's the same kind of problem and I should redefine all local
> attributes (from lines 655 to 687) as class attributes as you did in the
> patch.

I think I see what the problem is now. docutils/parsers/rst/states.py,
Inliner.init_customizations contains the following line::

    args.update(vars(self.__class__))

The "vars" function gets the __dict__ from the object's class. But
you're using a subclass, so vars returns the __dict__ of the subclass,
without the the superclass's attributes, which is what is needed. It's
a bit of metaprogramming gone wrong. It's not safe for subclassing,
which is a bug. It's already a bit of a kludgey shortcut. We could
replace it with::

    args.update(vars(Inliner))

That's a bit smelly, but I see it's done elsewhere in the code. Try
making that change (patch attached).

We may have to explicitly refer to all the class attributes. I'll think on it.

David Goodger

> I'll test it asap and report,
> Cheers,
> Jean Baptiste
>
>
> On 02/01/2017 21:24, David Goodger wrote:
>> On Sun, Jan 1, 2017 at 4:17 PM, Jean Baptiste Favre
>> <web...@jb...> wrote:
>>> Hello,
>>> I tried another workaround.
>>> Instead of using a custom class, I overrided init_cutomizations method
>>> using:
>>>
>>>   class Inliner(BaseInliner):
>>>     def __init(self):
>>>       BaseInliner.__init__(self)
>>>
>>>     def init_customizations(self, settings):
>>>       BaseInliner.init_customizations(self, settings)
>>>
>>>       issue_pattern = re.compile(u'''
>>>         {start_string_prefix}
>>>         TS-\d+
>>>         {end_string_suffix}'''.format(
>>>           start_string_prefix=self.start_string_prefix,
>>>           end_string_suffix=self.end_string_suffix),
>>>         re.VERBOSE | re.UNICODE)
>>>
>>>       self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>>>
>>> I don't call explicitly init_customizations, but it's called: during
>>> run, I still get the error:
>>>
>>>   Exception occurred:
>>>     File
>>> "/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
>>> 530, in init_customizations
>>>         """ % args, re.VERBOSE | re.UNICODE),
>>>   KeyError: 'non_unescaped_whitespace_escape_before'
>>>   The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
>>> you want to report the issue to the developers.
>>>   Please also report this if it was a user error, so that a better error
>>> message can be provided next time.
>>>   A bug report can be filed in the tracker at
>>> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>   make[6]: *** [man] Error 1
>>>
>>> What did I do wrong ?
>>
>> You didn't do anything wrong that I can see. There was an internal
>> change to Docutils in revision 7942 that refactored some code, and a
>> side effect was to remove some attributes from the Inliner class (they
>> became locals instead). Your code depends on these class attributes.
>>
>> Try applying the attached patch and let us know how it works. The
>> missing class attributes will be present as instance attributes, which
>> should be close enough.
>>
>> Günter, I think the attached patch should also be rolled into a 0.13.2
>> bugfix release.
>>
>> David Goodger
>> <http://python.net/~goodger>
>>
>>
>>> On 01/01/2017 20:59, Guenter Milde wrote:
>>>> On 2016-12-31, Jean Baptiste Favre wrote:
>>>>> Hello,
>>>>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>>>>> build with docutils 0.13.1
>>>>
>>>>> Error comes from a custom Inliner class which is defined in doc/conf.py
>>>>> of Trafficserver project(starting line 163).
>>>>> This allow to transform text like "(TS-XXXX)" in a link to
>>>>> trafficserver's Jira
>>>>
>>>>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>>>>> with following error:
>>>>
>>>>>   Exception occurred:
>>>>>     File "conf.py", line 185, in __init__
>>>>>       start_string_prefix=self.start_string_prefix,
>>>>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>>>>
>>>>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>>>>> have to call init_customiations for that.
>>>>
>>>> Yes, this changed with the implementation of the long expected feature
>>>>
>>>>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>>>>
>>>> and the new configuration setting "character_level_inline_markup".
>>>>
>>>>
>>>>> First problem: one  must pass settings param when calling
>>>>> init_customizations, but I can't find any format or structure for it.
>>>>
>>>>> Second problem: I tried a workaround, creating a InlinerSettings class
>>>>> with minimal properties so that init_customizations could pass:
>>>>
>>>>>   class InlinerSettings:
>>>>>     character_level_inline_markup=None
>>>>>     pep_references=None
>>>>>     rfc_references=None
>>>>
>>>>> But, then, I faced another error:
>>>>
>>>> ...
>>>>
>>>> "settings" is a an object returned from the option parser (optparse module).
>>>>
>>>> There are others facing similar problems, e.g. Python distutils.
>>>> There, I learned the trick is to use
>>>>
>>>>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>>>>
>>>>   -- https://hg.python.org/cpython/rev/db09d760b965
>>>>
>>>>
>>>>> Third problem: since the above tries didn't worked, I'd a look on custom
>>>>> directives and roles.
>>>>> But, it does not seems to be allowed to define a role with a custom regex.
>>>>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>>>>> XXXX".
>>>>
>>>> This is the "minimal-invasive" approach, it would be work now but might save
>>>> issues later, as it depends less on Docutils internals.
>>>>
>>>>> Is there any way to achieve the migration in a compatible way with
>>>>> docutils 0.12 *and* without rewriting the docuemntation ?
>>>>
>>>> If the above example does not lead to a solution, you could also consider to
>>>> copy the definition of start_string_prefix ...
>>>> from states.py (importing utils.punctuation_chars first)
>>>>
>>>> Günter
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[Jean B. F. <web...@jb...>]

Hello David,

Thanks for your answer.
Unfortunatly, the patch doesn't help.

I still have a "KeyError: 'non_unescaped_whitespace_escape_before'" in
docutils/parsers/rst/states.py at line 533.
I guess it's the same kind of problem and I should redefine all local
attributes (from lines 655 to 687) as class attributes as you did in the
patch.

I'll test it asap and report,
Cheers,
Jean Baptiste


On 02/01/2017 21:24, David Goodger wrote:
> On Sun, Jan 1, 2017 at 4:17 PM, Jean Baptiste Favre
> <web...@jb...> wrote:
>> Hello,
>> I tried another workaround.
>> Instead of using a custom class, I overrided init_cutomizations method
>> using:
>>
>>   class Inliner(BaseInliner):
>>     def __init(self):
>>       BaseInliner.__init__(self)
>>
>>     def init_customizations(self, settings):
>>       BaseInliner.init_customizations(self, settings)
>>
>>       issue_pattern = re.compile(u'''
>>         {start_string_prefix}
>>         TS-\d+
>>         {end_string_suffix}'''.format(
>>           start_string_prefix=self.start_string_prefix,
>>           end_string_suffix=self.end_string_suffix),
>>         re.VERBOSE | re.UNICODE)
>>
>>       self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>>
>> I don't call explicitly init_customizations, but it's called: during
>> run, I still get the error:
>>
>>   Exception occurred:
>>     File
>> "/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
>> 530, in init_customizations
>>         """ % args, re.VERBOSE | re.UNICODE),
>>   KeyError: 'non_unescaped_whitespace_escape_before'
>>   The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
>> you want to report the issue to the developers.
>>   Please also report this if it was a user error, so that a better error
>> message can be provided next time.
>>   A bug report can be filed in the tracker at
>> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>   make[6]: *** [man] Error 1
>>
>> What did I do wrong ?
> 
> You didn't do anything wrong that I can see. There was an internal
> change to Docutils in revision 7942 that refactored some code, and a
> side effect was to remove some attributes from the Inliner class (they
> became locals instead). Your code depends on these class attributes.
> 
> Try applying the attached patch and let us know how it works. The
> missing class attributes will be present as instance attributes, which
> should be close enough.
> 
> Günter, I think the attached patch should also be rolled into a 0.13.2
> bugfix release.
> 
> David Goodger
> <http://python.net/~goodger>
> 
> 
>> On 01/01/2017 20:59, Guenter Milde wrote:
>>> On 2016-12-31, Jean Baptiste Favre wrote:
>>>> Hello,
>>>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>>>> build with docutils 0.13.1
>>>
>>>> Error comes from a custom Inliner class which is defined in doc/conf.py
>>>> of Trafficserver project(starting line 163).
>>>> This allow to transform text like "(TS-XXXX)" in a link to
>>>> trafficserver's Jira
>>>
>>>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>>>> with following error:
>>>
>>>>   Exception occurred:
>>>>     File "conf.py", line 185, in __init__
>>>>       start_string_prefix=self.start_string_prefix,
>>>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>>>
>>>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>>>> have to call init_customiations for that.
>>>
>>> Yes, this changed with the implementation of the long expected feature
>>>
>>>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>>>
>>> and the new configuration setting "character_level_inline_markup".
>>>
>>>
>>>> First problem: one  must pass settings param when calling
>>>> init_customizations, but I can't find any format or structure for it.
>>>
>>>> Second problem: I tried a workaround, creating a InlinerSettings class
>>>> with minimal properties so that init_customizations could pass:
>>>
>>>>   class InlinerSettings:
>>>>     character_level_inline_markup=None
>>>>     pep_references=None
>>>>     rfc_references=None
>>>
>>>> But, then, I faced another error:
>>>
>>> ...
>>>
>>> "settings" is a an object returned from the option parser (optparse module).
>>>
>>> There are others facing similar problems, e.g. Python distutils.
>>> There, I learned the trick is to use
>>>
>>>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>>>
>>>   -- https://hg.python.org/cpython/rev/db09d760b965
>>>
>>>
>>>> Third problem: since the above tries didn't worked, I'd a look on custom
>>>> directives and roles.
>>>> But, it does not seems to be allowed to define a role with a custom regex.
>>>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>>>> XXXX".
>>>
>>> This is the "minimal-invasive" approach, it would be work now but might save
>>> issues later, as it depends less on Docutils internals.
>>>
>>>> Is there any way to achieve the migration in a compatible way with
>>>> docutils 0.12 *and* without rewriting the docuemntation ?
>>>
>>> If the above example does not lead to a solution, you could also consider to
>>> copy the definition of start_string_prefix ...
>>> from states.py (importing utils.punctuation_chars first)
>>>
>>> Günter

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[David G. <go...@py...>]

Patch attached.

David Goodger
<http://python.net/~goodger>


On Mon, Jan 2, 2017 at 2:24 PM, David Goodger <go...@py...> wrote:
> On Sun, Jan 1, 2017 at 4:17 PM, Jean Baptiste Favre
> <web...@jb...> wrote:
>> Hello,
>> I tried another workaround.
>> Instead of using a custom class, I overrided init_cutomizations method
>> using:
>>
>>   class Inliner(BaseInliner):
>>     def __init(self):
>>       BaseInliner.__init__(self)
>>
>>     def init_customizations(self, settings):
>>       BaseInliner.init_customizations(self, settings)
>>
>>       issue_pattern = re.compile(u'''
>>         {start_string_prefix}
>>         TS-\d+
>>         {end_string_suffix}'''.format(
>>           start_string_prefix=self.start_string_prefix,
>>           end_string_suffix=self.end_string_suffix),
>>         re.VERBOSE | re.UNICODE)
>>
>>       self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>>
>> I don't call explicitly init_customizations, but it's called: during
>> run, I still get the error:
>>
>>   Exception occurred:
>>     File
>> "/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
>> 530, in init_customizations
>>         """ % args, re.VERBOSE | re.UNICODE),
>>   KeyError: 'non_unescaped_whitespace_escape_before'
>>   The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
>> you want to report the issue to the developers.
>>   Please also report this if it was a user error, so that a better error
>> message can be provided next time.
>>   A bug report can be filed in the tracker at
>> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>   make[6]: *** [man] Error 1
>>
>> What did I do wrong ?
>
> You didn't do anything wrong that I can see. There was an internal
> change to Docutils in revision 7942 that refactored some code, and a
> side effect was to remove some attributes from the Inliner class (they
> became locals instead). Your code depends on these class attributes.
>
> Try applying the attached patch and let us know how it works. The
> missing class attributes will be present as instance attributes, which
> should be close enough.
>
> Günter, I think the attached patch should also be rolled into a 0.13.2
> bugfix release.
>
> David Goodger
> <http://python.net/~goodger>
>
>
>> On 01/01/2017 20:59, Guenter Milde wrote:
>>> On 2016-12-31, Jean Baptiste Favre wrote:
>>>> Hello,
>>>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>>>> build with docutils 0.13.1
>>>
>>>> Error comes from a custom Inliner class which is defined in doc/conf.py
>>>> of Trafficserver project(starting line 163).
>>>> This allow to transform text like "(TS-XXXX)" in a link to
>>>> trafficserver's Jira
>>>
>>>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>>>> with following error:
>>>
>>>>   Exception occurred:
>>>>     File "conf.py", line 185, in __init__
>>>>       start_string_prefix=self.start_string_prefix,
>>>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>>>
>>>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>>>> have to call init_customiations for that.
>>>
>>> Yes, this changed with the implementation of the long expected feature
>>>
>>>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>>>
>>> and the new configuration setting "character_level_inline_markup".
>>>
>>>
>>>> First problem: one  must pass settings param when calling
>>>> init_customizations, but I can't find any format or structure for it.
>>>
>>>> Second problem: I tried a workaround, creating a InlinerSettings class
>>>> with minimal properties so that init_customizations could pass:
>>>
>>>>   class InlinerSettings:
>>>>     character_level_inline_markup=None
>>>>     pep_references=None
>>>>     rfc_references=None
>>>
>>>> But, then, I faced another error:
>>>
>>> ...
>>>
>>> "settings" is a an object returned from the option parser (optparse module).
>>>
>>> There are others facing similar problems, e.g. Python distutils.
>>> There, I learned the trick is to use
>>>
>>>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>>>
>>>   -- https://hg.python.org/cpython/rev/db09d760b965
>>>
>>>
>>>> Third problem: since the above tries didn't worked, I'd a look on custom
>>>> directives and roles.
>>>> But, it does not seems to be allowed to define a role with a custom regex.
>>>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>>>> XXXX".
>>>
>>> This is the "minimal-invasive" approach, it would be work now but might save
>>> issues later, as it depends less on Docutils internals.
>>>
>>>> Is there any way to achieve the migration in a compatible way with
>>>> docutils 0.12 *and* without rewriting the docuemntation ?
>>>
>>> If the above example does not lead to a solution, you could also consider to
>>> copy the definition of start_string_prefix ...
>>> from states.py (importing utils.punctuation_chars first)
>>>
>>> Günter

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[David G. <go...@py...>]

On Sun, Jan 1, 2017 at 1:59 PM, Guenter Milde <mi...@us...> wrote:
> On 2016-12-31, Jean Baptiste Favre wrote:
>> Hello,
>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>> build with docutils 0.13.1
>
>> Error comes from a custom Inliner class which is defined in doc/conf.py
>> of Trafficserver project(starting line 163).
>> This allow to transform text like "(TS-XXXX)" in a link to
>> trafficserver's Jira
>
>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>> with following error:
>
>>   Exception occurred:
>>     File "conf.py", line 185, in __init__
>>       start_string_prefix=self.start_string_prefix,
>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>
>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>> have to call init_customiations for that.
>
> Yes, this changed with the implementation of the long expected feature
>
>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>
> and the new configuration setting "character_level_inline_markup".

This is another example of an API change coming back to bite us.

>> First problem: one  must pass settings param when calling
>> init_customizations, but I can't find any format or structure for it.
>
>> Second problem: I tried a workaround, creating a InlinerSettings class
>> with minimal properties so that init_customizations could pass:
>
>>   class InlinerSettings:
>>     character_level_inline_markup=None
>>     pep_references=None
>>     rfc_references=None
>
>> But, then, I faced another error:
>
> ...
>
> "settings" is a an object returned from the option parser (optparse module).
>
> There are others facing similar problems, e.g. Python distutils.
> There, I learned the trick is to use
>
>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>
>   -- https://hg.python.org/cpython/rev/db09d760b965

I think in this case, this is a red herring (a false issue). It's not
the cause of the problem.

>> Third problem: since the above tries didn't worked, I'd a look on custom
>> directives and roles.
>> But, it does not seems to be allowed to define a role with a custom regex.
>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>> XXXX".
>
> This is the "minimal-invasive" approach, it would be work now but might save
> issues later, as it depends less on Docutils internals.
>
>> Is there any way to achieve the migration in a compatible way with
>> docutils 0.12 *and* without rewriting the docuemntation ?
>
> If the above example does not lead to a solution, you could also consider to
> copy the definition of start_string_prefix ...
> from states.py (importing utils.punctuation_chars first)

Or we could fix our mistake. I think that's better.

David Goodger
<http://python.net/~goodger>

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[David G. <go...@py...>]

On Sun, Jan 1, 2017 at 4:17 PM, Jean Baptiste Favre
<web...@jb...> wrote:
> Hello,
> I tried another workaround.
> Instead of using a custom class, I overrided init_cutomizations method
> using:
>
>   class Inliner(BaseInliner):
>     def __init(self):
>       BaseInliner.__init__(self)
>
>     def init_customizations(self, settings):
>       BaseInliner.init_customizations(self, settings)
>
>       issue_pattern = re.compile(u'''
>         {start_string_prefix}
>         TS-\d+
>         {end_string_suffix}'''.format(
>           start_string_prefix=self.start_string_prefix,
>           end_string_suffix=self.end_string_suffix),
>         re.VERBOSE | re.UNICODE)
>
>       self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>
> I don't call explicitly init_customizations, but it's called: during
> run, I still get the error:
>
>   Exception occurred:
>     File
> "/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
> 530, in init_customizations
>         """ % args, re.VERBOSE | re.UNICODE),
>   KeyError: 'non_unescaped_whitespace_escape_before'
>   The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
> you want to report the issue to the developers.
>   Please also report this if it was a user error, so that a better error
> message can be provided next time.
>   A bug report can be filed in the tracker at
> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>   make[6]: *** [man] Error 1
>
> What did I do wrong ?

You didn't do anything wrong that I can see. There was an internal
change to Docutils in revision 7942 that refactored some code, and a
side effect was to remove some attributes from the Inliner class (they
became locals instead). Your code depends on these class attributes.

Try applying the attached patch and let us know how it works. The
missing class attributes will be present as instance attributes, which
should be close enough.

Günter, I think the attached patch should also be rolled into a 0.13.2
bugfix release.

David Goodger
<http://python.net/~goodger>


> On 01/01/2017 20:59, Guenter Milde wrote:
>> On 2016-12-31, Jean Baptiste Favre wrote:
>>> Hello,
>>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>>> build with docutils 0.13.1
>>
>>> Error comes from a custom Inliner class which is defined in doc/conf.py
>>> of Trafficserver project(starting line 163).
>>> This allow to transform text like "(TS-XXXX)" in a link to
>>> trafficserver's Jira
>>
>>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>>> with following error:
>>
>>>   Exception occurred:
>>>     File "conf.py", line 185, in __init__
>>>       start_string_prefix=self.start_string_prefix,
>>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>>
>>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>>> have to call init_customiations for that.
>>
>> Yes, this changed with the implementation of the long expected feature
>>
>>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>>
>> and the new configuration setting "character_level_inline_markup".
>>
>>
>>> First problem: one  must pass settings param when calling
>>> init_customizations, but I can't find any format or structure for it.
>>
>>> Second problem: I tried a workaround, creating a InlinerSettings class
>>> with minimal properties so that init_customizations could pass:
>>
>>>   class InlinerSettings:
>>>     character_level_inline_markup=None
>>>     pep_references=None
>>>     rfc_references=None
>>
>>> But, then, I faced another error:
>>
>> ...
>>
>> "settings" is a an object returned from the option parser (optparse module).
>>
>> There are others facing similar problems, e.g. Python distutils.
>> There, I learned the trick is to use
>>
>>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>>
>>   -- https://hg.python.org/cpython/rev/db09d760b965
>>
>>
>>> Third problem: since the above tries didn't worked, I'd a look on custom
>>> directives and roles.
>>> But, it does not seems to be allowed to define a role with a custom regex.
>>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>>> XXXX".
>>
>> This is the "minimal-invasive" approach, it would be work now but might save
>> issues later, as it depends less on Docutils internals.
>>
>>> Is there any way to achieve the migration in a compatible way with
>>> docutils 0.12 *and* without rewriting the docuemntation ?
>>
>> If the above example does not lead to a solution, you could also consider to
>> copy the definition of start_string_prefix ...
>> from states.py (importing utils.punctuation_chars first)
>>
>> Günter

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[Jean B. F. <web...@jb...>]

Hello,
I tried another workaround.
Instead of using a custom class, I overrided init_cutomizations method
using:

  class Inliner(BaseInliner):
    def __init(self):
      BaseInliner.__init__(self)

    def init_customizations(self, settings):
      BaseInliner.init_customizations(self, settings)

      issue_pattern = re.compile(u'''
        {start_string_prefix}
        TS-\d+
        {end_string_suffix}'''.format(
          start_string_prefix=self.start_string_prefix,
          end_string_suffix=self.end_string_suffix),
        re.VERBOSE | re.UNICODE)

      self.implicit_dispatch.append((issue_pattern, self.issue_reference))

I don't call explicitly init_customizations, but it's called: during
run, I still get the error:

  Exception occurred:
    File
"/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
530, in init_customizations
        """ % args, re.VERBOSE | re.UNICODE),
  KeyError: 'non_unescaped_whitespace_escape_before'
  The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
you want to report the issue to the developers.
  Please also report this if it was a user error, so that a better error
message can be provided next time.
  A bug report can be filed in the tracker at
<https://github.com/sphinx-doc/sphinx/issues>. Thanks!
  make[6]: *** [man] Error 1

What did I do wrong ?

Cheers,
Jean Baptiste Favre

On 01/01/2017 20:59, Guenter Milde wrote:
> On 2016-12-31, Jean Baptiste Favre wrote:
>> Hello,
>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>> build with docutils 0.13.1
> 
>> Error comes from a custom Inliner class which is defined in doc/conf.py
>> of Trafficserver project(starting line 163).
>> This allow to transform text like "(TS-XXXX)" in a link to
>> trafficserver's Jira
> 
>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>> with following error:
> 
>>   Exception occurred:
>>     File "conf.py", line 185, in __init__
>>       start_string_prefix=self.start_string_prefix,
>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
> 
>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>> have to call init_customiations for that.
> 
> Yes, this changed with the implementation of the long expected feature
> 
>   - Apply [ 103 ] Recognize inline markups without word boundaries.
> 
> and the new configuration setting "character_level_inline_markup".
> 
> 
>> First problem: one  must pass settings param when calling
>> init_customizations, but I can't find any format or structure for it.
> 
>> Second problem: I tried a workaround, creating a InlinerSettings class
>> with minimal properties so that init_customizations could pass:
> 
>>   class InlinerSettings:
>>     character_level_inline_markup=None
>>     pep_references=None
>>     rfc_references=None
> 
>> But, then, I faced another error:
> 
> ...
> 
> "settings" is a an object returned from the option parser (optparse module).
> 
> There are others facing similar problems, e.g. Python distutils. 
> There, I learned the trick is to use
> 
>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>   
>   -- https://hg.python.org/cpython/rev/db09d760b965
> 
> 
>> Third problem: since the above tries didn't worked, I'd a look on custom
>> directives and roles.
>> But, it does not seems to be allowed to define a role with a custom regex.
>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>> XXXX".
> 
> This is the "minimal-invasive" approach, it would be work now but might save
> issues later, as it depends less on Docutils internals.
> 
>> Is there any way to achieve the migration in a compatible way with
>> docutils 0.12 *and* without rewriting the docuemntation ?
> 
> If the above example does not lead to a solution, you could also consider to
> copy the definition of start_string_prefix ...
> from states.py (importing utils.punctuation_chars first)
> 
> Günter

Re: [Docutils-users] Migration docutils 0.12 to 0.13

[Guenter M. <mi...@us...>]

On 2016-12-31, Jean Baptiste Favre wrote:
> Hello,
> I'm facing an issue with Trafficserver documentation [1] which doesn't
> build with docutils 0.13.1

> Error comes from a custom Inliner class which is defined in doc/conf.py
> of Trafficserver project(starting line 163).
> This allow to transform text like "(TS-XXXX)" in a link to
> trafficserver's Jira

> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
> with following error:

>   Exception occurred:
>     File "conf.py", line 185, in __init__
>       start_string_prefix=self.start_string_prefix,
>   AttributeError: Inliner instance has no attribute 'start_string_prefix'

> Since docutils 0.13, start_string_prefix isn't statically defined. We
> have to call init_customiations for that.

Yes, this changed with the implementation of the long expected feature

  - Apply [ 103 ] Recognize inline markups without word boundaries.

and the new configuration setting "character_level_inline_markup".


> First problem: one  must pass settings param when calling
> init_customizations, but I can't find any format or structure for it.

> Second problem: I tried a workaround, creating a InlinerSettings class
> with minimal properties so that init_customizations could pass:

>   class InlinerSettings:
>     character_level_inline_markup=None
>     pep_references=None
>     rfc_references=None

> But, then, I faced another error:

...

"settings" is a an object returned from the option parser (optparse module).

There are others facing similar problems, e.g. Python distutils. 
There, I learned the trick is to use

  settings = frontend.OptionParser(components=(Parser,)).get_default_values()
  
  -- https://hg.python.org/cpython/rev/db09d760b965


> Third problem: since the above tries didn't worked, I'd a look on custom
> directives and roles.
> But, it does not seems to be allowed to define a role with a custom regex.
> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
> XXXX".

This is the "minimal-invasive" approach, it would be work now but might save
issues later, as it depends less on Docutils internals.

> Is there any way to achieve the migration in a compatible way with
> docutils 0.12 *and* without rewriting the docuemntation ?

If the above example does not lead to a solution, you could also consider to
copy the definition of start_string_prefix ...
from states.py (importing utils.punctuation_chars first)

Günter

[Docutils-users] Migration docutils 0.12 to 0.13

[Jean B. F. <web...@jb...>]

Hello,
I'm facing an issue with Trafficserver documentation [1] which doesn't
build with docutils 0.13.1

Error comes from a custom Inliner class which is defined in doc/conf.py
of Trafficserver project(starting line 163).
This allow to transform text like "(TS-XXXX)" in a link to
trafficserver's Jira

This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
with following error:

  Exception occurred:
    File "conf.py", line 185, in __init__
      start_string_prefix=self.start_string_prefix,
  AttributeError: Inliner instance has no attribute 'start_string_prefix'

Since docutils 0.13, start_string_prefix isn't statically defined. We
have to call init_customiations for that.

First problem: one  must pass settings param when calling
init_customizations, but I can't find any format or structure for it.

Second problem: I tried a workaround, creating a InlinerSettings class
with minimal properties so that init_customizations could pass:

  class InlinerSettings:
    character_level_inline_markup=None
    pep_references=None
    rfc_references=None

But, then, I faced another error:

  Exception occurred:
    File
"/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
530, in init_customizations
      """ % args, re.VERBOSE | re.UNICODE),
  KeyError: 'non_unescaped_whitespace_escape_before'

It doesn't seem to be the right way to achieve it.

Third problem: since the above tries didn't worked, I'd a look on custom
directives and roles.
But, it does not seems to be allowed to define a role with a custom regex.
This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
XXXX".

Is there any way to achieve the migration in a compatible way with
docutils 0.12 *and* without rewriting the docuemntation ?

Cheers,
Jean Baptiste Favre

[1]: https://github.com/apache/trafficserver/tree/master/doc

Re: [Docutils-users] strikeout and hyperlinks

[Saša J. <go...@at...>]

On Fri, 23 Dec 2016 11:53:15 -0600
David Goodger <go...@py...> wrote:

> Note that you'd have to break up your sentence into 3 pieces.

That looks too obtrusive to me and it seems better to just change gear
into markdown when such things are required until there is proper
support for nested markup in rst.


Sincerely,
Gour

-- 
The humble sages, by virtue of true knowledge, see with equal
vision a learned and gentle brāhmana, a cow, an elephant, a dog
and a dog-eater.

Re: [Docutils-users] strikeout and hyperlinks

[Gour <go...@at...>]

On Fri, 23 Dec 2016 17:18:06 +0000 (UTC)
Guenter Milde <mi...@us...> wrote:

> You may consider giving the strikeout class to the list item:

I played with it a bit, but the problem is that it works as long as one
needs to apply class to the *whole* list item, but, afaict, it fails
when one needs to be applied only to the part of the sentence, right?


Sincerely,
Gour

-- 
Therefore, without being attached to the fruits of activities,
one should act as a matter of duty, for by working without
attachment one attains the Supreme.

Re: [Docutils-users] strikeout and hyperlinks

[Gour <go...@at...>]

On Fri, 23 Dec 2016 17:18:06 +0000 (UTC)
Guenter Milde <mi...@us...> wrote:

> BTW: There is no need for the :class: argument in this case (the class
> defaults to the role name if not explicitely given).

Thank you - learnt something new. ;)

> This is an unfortunate, long known limitation and TODO item. It holds
> for both, pre-defined and custom roles as well as substitutions and
> links.

Is there any chance that this feature is going to be added in the
foreseeable future?

> 
> > Do you have any suggestion in rst for something simple as:   
> ...
> 
> You may consider giving the strikeout class to the list item:
> 
> Things to consider:
> 
> - .. class:: strike
> 
>   point which is already done in our `archive
>   <http://www.example.com/archive/>`_ which contains mp3 files

That one looks pretty good. Thanks a lot!


Sincerely,
Gour

-- 
He who is satisfied with gain which comes of its own accord, who
is free from duality and does not envy, who is steady in both
success and failure, is never entangled, although performing actions.

Re: [Docutils-users] strikeout and hyperlinks

[David G. <go...@py...>]

On Fri, Dec 23, 2016 at 3:30 AM, Gour <go...@at...> wrote:
> I've a need for strikeout text in some of my blog posts that I migrate to
> Nikola static-site-generator using rst markup...
>
> I've the following in my post:
>
> +---------------------+
> .. role:: strike
>     :class: strike
>
> Here are few things to do:
>
> - :strike:`point which is already done in our `archive
>   <http://www.example.com/archive/>` which contains mp3 files`

There is a workaround to reST limitation regarding nested inline
markup that allows you to style a hyperlink reference, described here:
http://docutils.sourceforge.net/FAQ.html#is-nested-inline-markup-possible.

This is how you'd do the example above:

:strike:`point which is already done in our` |archive|_ :strike:`which
contains mp3 files`

.. |archive| replace:: :strike:`archive`
.. _archive: http://www.example.com/archive/

Note that you'd have to break up your sentence into 3 pieces.

Also, if you use your strikeout role a lot, you can set it as the
default interpreted text role and use `this syntax`:
http://docutils.sourceforge.net/docs/ref/rst/directives.html#setting-the-default-interpreted-text-role

David Goodger
<http://python.net/~goodger>

Re: [Docutils-users] Book using rst2html: From Python to Numpy

[Nicolas P. R. <Nic...@in...>]

> On 23 Dec 2016, at 18:25, Guenter Milde <mi...@us...> wrote:
> 
> Dear Nicolas,
> 
> On 2016-12-22, Nicolas P. Rougier wrote:
> 
>> I've put online a book (From Python to Numpy) that has been written in
>> rst and transformed using rst2html.py. I find the result quite nice but
>> of course, it's a matter of personal taste...
> 
>> The book can be read at: http://www.labri.fr/perso/nrougier/from-python-to-numpy/
>> Sources are available at: https://github.com/rougier/from-python-to-numpy
> 
> 
> Thank you for sharing the work.
> 
> Did you also try to get a LaTeXed version? I would be interested to see
> the mileage of such a larger work with the latex writer. It should be
> possible to use the tufte class (or any other suitable one) with
> rst2latex.py or rst2xetex.py (the latter can also be used together with
> lualatex, despite the name).

I did not try the latex writer yet because I introduced some html-tricks into my rst.
I wanted to have figure legends on the side at the level of the image and the syntax
I came up is:

.. admonition:: **Figure**

   Figure caption

.. image:: some-image.png
   :width:100%

I did not yet managed to do the same with the figure environment. This would make a mess
in latex.


> 
> Also, it would be interesting to see whether the html5 writer results in
> improvements. However, this would require some work to get the custom CSS
> adapted.

Currently the output of rst2html5.py is quite broken but I need to investigate.
Most probably some of my css needs to be rewritten.


If you want to try, you can fork the github repo and compare:


rst2html.py --link-stylesheet --toc-top-backlinks --cloak-email-addresses  --stylesheet=book.css book.rst book-1.html

vs
rst2html5.py --link-stylesheet --toc-top-backlinks --cloak-email-addresses  --stylesheet=book.css book.rst book-2.html


Merry Christmas,
Nicolas

> 
> Joyeux Noël !
> 
> Günter
> 
> 
> ------------------------------------------------------------------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
> 
> Please use "Reply All" to reply to the list.

Re: [Docutils-users] Book using rst2html: From Python to Numpy

[Guenter M. <mi...@us...>]

Dear Nicolas,

On 2016-12-22, Nicolas P. Rougier wrote:

> I've put online a book (From Python to Numpy) that has been written in
> rst and transformed using rst2html.py. I find the result quite nice but
> of course, it's a matter of personal taste...

> The book can be read at: http://www.labri.fr/perso/nrougier/from-python-to-numpy/
> Sources are available at: https://github.com/rougier/from-python-to-numpy


Thank you for sharing the work.

Did you also try to get a LaTeXed version? I would be interested to see
the mileage of such a larger work with the latex writer. It should be
possible to use the tufte class (or any other suitable one) with
rst2latex.py or rst2xetex.py (the latter can also be used together with
lualatex, despite the name).

Also, it would be interesting to see whether the html5 writer results in
improvements. However, this would require some work to get the custom CSS
adapted.

Joyeux Noël !

Günter

Re: [Docutils-users] strikeout and hyperlinks

[Guenter M. <mi...@us...>]

On 2016-12-23, Gour wrote:
> Hello,

> I've a need for strikeout text in some of my blog posts that I migrate to
> Nikola static-site-generator using rst markup...

> I've the following in my post:

> .. role:: strike
>     :class: strike

BTW: There is no need for the :class: argument in this case (the class
defaults to the role name if not explicitely given).

> but that does not work.

You missed the trailing _ in the link, but most importantly:

.. note:: Docutils does not support nested inline markup!
 
This is an unfortunate, long known limitation and TODO item. It holds for
both, pre-defined and custom roles as well as substitutions and links.

> Do you have any suggestion in rst for something simple as: 
...

You may consider giving the strikeout class to the list item:

Things to consider:

- .. class:: strike

  point which is already done in our `archive
  <http://www.example.com/archive/>`_ which contains mp3 files


Alternatively, you could strike out with Unicode:

- point which is a̶l̶r̶e̶a̶d̶y̶ ̶d̶o̶n̶e̶ ̶i̶n̶ ̶o̶u̶r̶ `a̶r̶c̶h̶i̶v̶e̶
  <http://www.example.com/archive/>`_ which contains mp3 files


Günter

[Docutils-users] strikeout and hyperlinks

[Gour <go...@at...>]

Hello,

I've a need for strikeout text in some of my blog posts that I migrate to
Nikola static-site-generator using rst markup...

I've the following in my post:

+---------------------+
.. role:: strike
    :class: strike

Here are few things to do:

- :strike:`point which is already done in our `archive
  <http://www.example.com/archive/>` which contains mp3 files`

- another point

+-----------------+
but that does not work. If I replace the above hyperlink with:

.. archive: httpi://www.example.com/archive 

and then write item list as:

- :strike:`point which is already done in our archive_ which contains
  mp3 files`

then the whole item's text is striked-out, but I lose the hyperlink for
the 'archive'.

 
Do you have any suggestion in rst for something simple as: 

+---------------------------+
Here are few things to do:

- ~~point which is already done in our [archive]( <http://www.example.com/archive/) 
which contains mp3 files~~

- another point
 
+------------------+

when written in Markdown and rendered as:

<p>Here are few things to do:</p>
<ul>
<li><p><del>point which is already done in our 
<a href="%3Chttp://www.example.com/archive/">archive</a> which contains 
mp3 files</del></p></li>
<li><p>another point</p></li>
</ul>


Sincerely,
Gour


-- 
When your intelligence has passed out of the dense forest
of delusion, you shall become indifferent to all that has
been heard and all that is to be heard.

Re: [Docutils-users] Book using rst2html: From Python to Numpy

[Paolo C. <cav...@fa...>]

Il 22/12/2016 20:13, Nicolas P. Rougier ha scritto:

> I've put online a book (From Python to Numpy) that has been written in rst and transformed using rst2html.py.
> I find the result quite nice but of course, it's a matter of personal taste...
> 
> The book can be read at: http://www.labri.fr/perso/nrougier/from-python-to-numpy/
> Sources are available at: https://github.com/rougier/from-python-to-numpy

Thanks Nicolas, very inspiring.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis

[Docutils-users] Book using rst2html: From Python to Numpy

[Nicolas P. R. <Nic...@in...>]

Hi all,

I've put online a book (From Python to Numpy) that has been written in rst and transformed using rst2html.py.
I find the result quite nice but of course, it's a matter of personal taste...

The book can be read at: http://www.labri.fr/perso/nrougier/from-python-to-numpy/
Sources are available at: https://github.com/rougier/from-python-to-numpy


Nicolas

Re: [Docutils-users] rst2html5: why is table data wrapped in a paragraph

[Guenter M. <mi...@us...>]

On 2016-12-21, Alan Isaac wrote:
> When producing a csv-table with the new writer,
> I discovered the cell content is wrapped not
> just in a TD element but additionally in a P element.
> Is this intentional? I can imagine a table cell with
> multiple paragraphs, but I cannot imagine how to
> produce that with a CSV table.

It is intentional. The aim is consistency.

For a table object in the document tree, it should not matter whether it was
created with a grid-table, a csv-table or a list-table. 

The Docutils doctree and the native Docutils XML always have paragraphs
in table cells.
The html4css1 writer stripped paragraph elements in some cases in lists and
table cells, because "styling away" the leading and trailing vertical space
is impossible/complicated with CSS1. However, with CSS2 this is no longer a
problem.


Günter

[Docutils-users] rst2html5: why is table data wrapped in a paragraph

[Alan I. <ala...@gm...>]

When producing a csv-table with the new writer,
I discovered the cell content is wrapped not
just in a TD element but additionally in a P element.
Is this intentional? I can imagine a table cell with
multiple paragraphs, but I cannot imagine how to
produce that with a CSV table.

It took me a while to figure out why my table was not styling correctly...

Thanks,
Alan Isaac

[Docutils-users] Fwd: SourceForge Project of the Week December 19th, 2016

[David G. <go...@py...>]

FYI. Go ahead and promote this if you like.

David Goodger
<http://python.net/~goodger>

---------- Forwarded message ----------
From: Joan Uy Ang <joa...@sl...>
Date: Wed, Dec 14, 2016 at 12:34 AM
Subject: SourceForge Project of the Week December 19th, 2016
To:


Hi,

If you’re receiving this email it is because you are listed as an
admin on our next Projects of the Week Issue for December 19th, 2016
on the SourceForge Blog.

Here are the projects selected:
...
docutils
...

You will see them reflected live on the SourceForge homepage and blog next week.

Thanks,

Joan Nadene
SourceForge Community Coordinator

Re: [Docutils-users] why is UNICODE space after literal markup (``) invalid, but after emphasis(*) or strong(**) ok?

[Guenter M. <mi...@us...>]

Dear log4yes,

thank you for the bug report and analysis.


On 2016-12-19, log4yes wrote:
> Hi, does anyone know that? For example,  r"**a**\u2002b" is valid but 
> r"``a``\u2002b" will generate a warnning message. \u2002 is a whitespace.

> I've checked the source code. Docutils uses regular expression to search 
> the end of inline markups. It compiles STRONG regex with re.UNICODE 
> option, so it matches the UNICODE space. But why doesn't it use the same 
> option with LITERAL regex?

> Is here a mistake of implementation? Is there something specially different?

It is an omission that hithero slipped our attention and tests.

I prepared a patch that will be at the repository soon.

Thanks,
Günter

[Docutils-users] why is UNICODE space after literal markup (``) invalid, but after emphasis(*) or strong(**) ok?

[log4yes <lo...@gm...>]

Hi, does anyone know that? For example,  r"**a**\u2002b" is valid but 
r"``a``\u2002b" will generate a warnning message. \u2002 is a whitespace.

I've checked the source code. Docutils uses regular expression to search 
the end of inline markups. It compiles STRONG regex with re.UNICODE 
option, so it matches the UNICODE space. But why doesn't it use the same 
option with LITERAL regex?

Is here a mistake of implementation? Is there something specially different?

[Docutils-users] [Possibly OT] rst Type Tool For Spreadsheets

[Tim D. <tu...@tu...>]

This may officialy be a Dumb Question (tm) but ...

Is there an rst-ish tool - let's call it rst2xls - that would allow
a plain text markdown file to be transformed into a spreadsheet with
formatting, titles, formulae, etc.?

I regularly have to extract data from client host machines, using command line
tools and then go through a very painful process of converting to csv,
formatting, titling, etc.  

This can be done programmatically using the 'xlswriter' module for Python.
But, this tends to encourage one-off solutions for the moment that are
very task specific.   I'm just wondering if anyone has run into a more
generic markdown approach to handling this problem.

Thanks,
-- 
----------------------------------------------------------------------------
Tim Daneliuk     tu...@tu...
PGP Key:         http://www.tundraware.com/PGP/

[Docutils-users] Release 0.13.1

[engelbert g. <gr...@us...>]

Release 0.13.1 (2016-12-09)
===========================

* docutils/writers/html_plain

  - New HTML writer generating `HTML 5`_.

* tools/

  - New front-end ``rst2html5.py``.

.. _HTML 5: http://www.w3.org/TR/html5/

* languages: persian/farsi and latvian/la mappings.

* change default base url for :rfc: to http://tools.ietf.org/html/

* tables accept widths, a list and align

* latex2e: Fix admonition width, remove deprecated options, better
tablewidth auto, ...

* rst.el : The problem with ``electric-indent-mode`` has been fixed.