Re: [Docutils-develop] inline markup without word boundaries

[David G. <go...@py...>]

I think you forgot to hit [Reply-All]. I added Docutils-Develop back in.

On Wed, May 25, 2016 at 3:22 PM, Guenter Milde <mi...@us...> wrote:
> Dear David,
>
> thank you for the fast reply.
>
> On 25.05.16, David Goodger wrote:
>> On Tue, May 24, 2016 at 7:30 AM, Guenter Milde <mi...@us...> wrote:
>> >
>> > about three years ago, Atsuo Ishimoto provided a patch with a setting to
>> > relax the `docutils inline markup recognition rules`__.
> ...
>> > What should the new setting be called?
>> > --------------------------------------
>> >
>> > Suggestion:
>> >
>> > setting: simple_inline_markup,
>
>> -1. "Simple" is a very vague, misleading term. I can imagine users
>> turning this on and the result will be anything but simple.
>
> I see.
>
> (Still, understanding the inline markup recognition rules becomes really
> much simpler when leaving out the two clauses about outside characters.)

It's the *result* that isn't simple. As
http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#inline-markup-recognition-rules
states,

"""
The inline markup recognition rules were devised to allow 90% of
non-markup uses of "*", "`", "_", and "|" without escaping.
"""

This is a very important *feature*. Any setting that negates this
feature should appear at least a little bit scary to users. "Simple"
looks inviting.

ReStructuredText was never designed for East Asian languages, and
adapting it will involve compromises. Change the inline markup
recognition rules, and suddenly a "4*5=20" in somebody's input text
triggers an error.

>> > command line options:
>> >
>> >   --conservative-inline-markup
>> >                           Ignore inline markup unless surrounded by whitespace
>> >                           or punctuation. Enabled by default.
>> >   --simple-inline-markup  No restrictions on characters around inline markup.
>
>> "Conservative" is not great but OK here.
>
> I actually choose this, because it matches in two of its 5 senses:
>
>   1. (11) conservative -- (resistant to change)
>   3. (1) cautious, conservative -- (avoiding excess; "a conservative estimate")
>
>   -- `wordnet conservative --over`

Sense 1 only applies relative to the current rules. If we change them,
it no longer makes sense.

Sense 3 does apply, but like "simple", it's too vague.

>> The opposite of "conservative" is "liberal".
>
> At least in some of its senses. However, "liberal inline markup" does not
> work well and "immoderate" is in not better.
>
> The command line option does relax the inline markup recognition rules, but
> "--relaxed-inline-markup" may be a too informal description.
>
>
>> Perhaps --delimited-inline-markup (default) vs
>> --arbitrary-inline-markup or --unbounded-inline-markup.
>
> I understand that this relates to "arbitrary position in the text flow" or
> "not restricted to word boundaries", however both terms have far reaching
> and potentially misleading conotations. (We keep 5 out of 7 rules limiting
> inline markup recognition.)

We will have to keep looking for the best term then. Because "simple"
is unacceptable.

Better to have an explicit (even if unwieldy) name, like
--punctuation-delimited-inline-markup.

>> > How about third alternative "force"?
>> > ------------------------------------
>> >
>> > A recent post to docutils-users asked for a way to check for empty custom
>> > roles and warn the user.
>> >
>> > Currently, empty roles (:math:``, say) are not recognized as rST markup.
>> >
>> > A "force inline markup" setting could lift all restrictions except
>> > explicitely escaped characters.
>
>> I don't understand what you're suggesting. What are the alternatives
>> and implications?
>
> Currently, it is not possible to issue a warning for empty interpreted
> text like :sub:`` or :var:`` (with custom role "var").
>
> The tokens are not recognized as inline markup due to the rule
>
>     The inline markup end-string must be separated by at least one
>     character from the start-string.

That rule didn't anticipate empty (no content) inline text. Put
another way, inline markup without content was considered an error. If
we want to allow inline markup without content (perhaps as a marker to
say "here!" for code to act upon) then we can override that rule in
the case of interpreted text with an explicit role.

>> Why can't we just fix the parser to recognize :math:`` as proper reST?
>
> «:math:``» is valid rST just as «||» or «hello world» is -- simple unadorned
> text.
>
> Parsing it as inline markup requires a change of the specification. For
> backwards compatibility we should only do this if there is a setting to
> keep the current behaviour.

If we keep the change small such that there's no adverse effect (no
existing correct reST would suddenly become incorrect), there's no
need for strict backwards compatibility. It's acceptable for some
currently-incorrect (parses with an error) reST to suddenly become
correct (parses without error).

> I agree that this should be handled independent from the character-leve inline
> markup, though.

Yes, they're two independent features, and should not be conflated.
They should be discussed separately.

David