python-markdown-discuss

[Python-markdown-discuss] The WikiLinks Conundrum

[Waylan L. <wa...@gm...>]

As many of you know, the wikilink extension is not currently working.
Initially we had some problems with the way inline placeholders were
implemented but that has been resolved.

The problem is that CamelCase words that correctly appear in other
parts of the markup are mistakenly being identified as wikilinks.
Consider the following cases:

        <a href="http://example.com/">RealLink</a>
	
	<http://example.com/AutoLink>
	
 	[MarkdownLink](http://example.com/)

In all three cases, the label is identified as a wikilink and we get
nested links i.e.:

         <a href="http://example.com/"><a href="/RealLink/"
class="wikilink">RealLink</a></a>

In the past this was not a problem because any markup in the labels
was ignored. Now that that bug is squished, I need a way to tell
markdown to not run the WikiLink pattern against the label. There are
currently two mechanisms to do that. One it to set each label as an
AtomicString. However, then we would again be back were we were with
nested markup being ignored in labels.

The other solution is to reorder the InlinePatterns. The problem is,
we don't want the wikilink pattern to match against camelcase URLs
(i.e. "AutoLink" of the above autolink) so the wikilink pattern must
run after all the  various link patterns. Additionally, for various
other reasons (such as allowing asterisk in URLs) the link patterns
must be run before most other inline patterns - so we can't set the
labels as AtomicString *and* reorder either.

What this means is that, unless I'm missing something obvious, there
is no way to reliably match CamelCase wikilinks without getting nested
links when a label contains a CamelCase word. Interestingly, it
occurred to me that if InlinePattern was passed the parent element, we
could test ``parent_elem.tag != 'a'``, but even then, if that
CamelCase word is nested in strong or em tags, that test would fail.
Besides, I can't think of any other use for doing that.

Therefore, I'm thinking we will need to throw out the wikilink
extension as it currently exists. Alternatively, I could replace the
regex with a pattern that matched bracketed links (i.e. [[somelink]])
regardless of case as someone recently suggested, but then anyone
using the extension would need to update their documents to the new
syntax.

Any thoughts?

PS: If I change to [[bracketed links]], I will also change the name of
the extension to avoid any confusion. Although, probably not a drastic
change - say from "wikilink" to "wikilinks" (what it should have been
called to begin with). That way, the extension users will be forced to
note the difference and be alerted that something changed.
-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Blake W. <bw...@la...>]

Waylan Limberg wrote:
> The problem is that CamelCase words that correctly appear in other
> parts of the markup are mistakenly being identified as wikilinks.
> 
> In the past this was not a problem because any markup in the labels
> was ignored. Now that that bug is squished, I need a way to tell
> markdown to not run the WikiLink pattern against the label. There are
> currently two mechanisms to do that. One it to set each label as an
> AtomicString. However, then we would again be back were we were with
> nested markup being ignored in labels.

Ooh, ooh!  SemiAtomicString?  ;)

> The other solution is to reorder the InlinePatterns.

I think that having patterns be order dependant isn't a great idea, in 
general.

Could the WikiLink pattern be made more context-sensitive somehow,
so that it wouldn't match if there it was inside a "Label"?

> Interestingly, it
> occurred to me that if InlinePattern was passed the parent element, we
> could test ``parent_elem.tag != 'a'``, but even then, if that
> CamelCase word is nested in strong or em tags, that test would fail.
> Besides, I can't think of any other use for doing that.

Well, if you could walk up your parents, you could check all of them for 
an 'a' (and check whether you were an attribute).  It seems like it 
might be worth defining a couple of custom walkers, to handle cases 
where we want to expand stuff, but only if it occurs in 
non-link/non-attribute places.

> Any thoughts?

It's a tough problem, to be sure.

Later,
Blake.

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Waylan L. <wa...@gm...>]

On Fri, Sep 5, 2008 at 4:52 PM, Blake Winton <bw...@la...> wrote:
> Waylan Limberg wrote:
>> The problem is that CamelCase words that correctly appear in other
>> parts of the markup are mistakenly being identified as wikilinks.
>>
>> In the past this was not a problem because any markup in the labels
>> was ignored. Now that that bug is squished, I need a way to tell
>> markdown to not run the WikiLink pattern against the label. There are
>> currently two mechanisms to do that. One it to set each label as an
>> AtomicString. However, then we would again be back were we were with
>> nested markup being ignored in labels.
>
> Ooh, ooh!  SemiAtomicString?  ;)
>
>> The other solution is to reorder the InlinePatterns.
>
> I think that having patterns be order dependant isn't a great idea, in
> general.
>
> Could the WikiLink pattern be made more context-sensitive somehow,
> so that it wouldn't match if there it was inside a "Label"?
>
>> Interestingly, it
>> occurred to me that if InlinePattern was passed the parent element, we
>> could test ``parent_elem.tag != 'a'``, but even then, if that
>> CamelCase word is nested in strong or em tags, that test would fail.
>> Besides, I can't think of any other use for doing that.
>
> Well, if you could walk up your parents, you could check all of them for
> an 'a' (and check whether you were an attribute).  It seems like it
> might be worth defining a couple of custom walkers, to handle cases
> where we want to expand stuff, but only if it occurs in
> non-link/non-attribute places.
>
This was my initial thought as well. I guess I left out the fact that
ElementTree doesn't track parents of an Element - only children. Grrr.
Although, the project website does offer some sample code [1] that can
be used to walk the entire tree from root and generate a list of all
parents of an element. Seemed like an awful lot of extra work just to
check a parent - I'm sure we'd take a performance (and memory?) hit
with it.

[1]: http://effbot.org/zone/element.htm#accessing-parents

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Neale P. <ne...@wo...>]

I'm currently working on a gigantic patch that completely eliminates
placeholders and stashes.

I don't know if this is going to even work yet, so don't hold your
breath.  But if it does work it should eliminate all these kludges to
prevent patterns from matching placeholders and allow a WikiLink
extension to work with no problems.

Neale

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Waylan L. <wa...@gm...>]

Neale, I'd be interested in seeing your progress on this. Feel free to
create a clone on Gitorious (anyone can) and push your changes. That
way, other's can track what your doing and maybe even offer input. I
look forward to hearing more.

On Fri, Sep 5, 2008 at 11:56 PM, Neale Pickett <ne...@wo...> wrote:
> I'm currently working on a gigantic patch that completely eliminates
> placeholders and stashes.
>
> I don't know if this is going to even work yet, so don't hold your
> breath.  But if it does work it should eliminate all these kludges to
> prevent patterns from matching placeholders and allow a WikiLink
> extension to work with no problems.
>
> Neale
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Waylan L. <wa...@gm...>]

Oh, I should also point out that I have already resolved the problems
with the placeholders. That's not at all the issue I'm talking about
here. Of course, perhaps if your reimplementing some of the way inline
patterns work, that might make a difference.

On Fri, Sep 5, 2008 at 11:56 PM, Neale Pickett <ne...@wo...> wrote:
> I'm currently working on a gigantic patch that completely eliminates
> placeholders and stashes.
>
> I don't know if this is going to even work yet, so don't hold your
> breath.  But if it does work it should eliminate all these kludges to
> prevent patterns from matching placeholders and allow a WikiLink
> extension to work with no problems.
>
> Neale
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] The WikiLinks Conundrum

[Neale P. <ne...@wo...>]

"Waylan Limberg" <wa...@gm...> writes:

> Oh, I should also point out that I have already resolved the problems
> with the placeholders. That's not at all the issue I'm talking about
> here. Of course, perhaps if your reimplementing some of the way inline
> patterns work, that might make a difference.

Actually I don't think my changes would help with this particular
problem.  I was just using the thread as an "in" to discuss what I was
working on.  ;)

Neale