Re: [Python-markdown-discuss] Escape character and backtick bug.

[Waylan L. <wa...@gm...>]

On Feb 16, 2008 11:05 PM, Blake Winton <bw...@la...> wrote:
> Waylan Limberg wrote:
> > Blake, this is an interesting approach I hadn't thought of and I like it .
>
> Thanks!
>
> > When we run through the unescape code,
> > how do we know that it was an escaped character, not something
> > intentionally typed by the author that should be left alone?
> > Or is that why you also escape the `&`?
>
> Yup, that's exactly why I escape the &.
>
> > Either way, I was never particularly fond of the way markdown
> > currently handles escaping. I think your solution is much more elegant
> > and I'd prefer to use something very much like it.
> >
> > However, I'm thinking that one of two things needs to happen:
> >
> > 1. The escaping needs to happen at the inline pattern level (or at
> > least right before/after them, perhaps in _handleInline), rather than
> > on the entire document. That eliminates the need to worry about
> > messing with code blocks and raw html.
>
> Well, I think that escapes can happen (almost) anywhere, not just in
> inline patterns.  (Unless inline patterns handle everything except code
> blocks...)  Reading the documentation again, I'm not entirely sure why I
> believe that.  Perhaps confirmation from Mr. Gruber is in order.

Something to keep in mind here. Python-markdown is not coded against
markdown.pl, but rather Mr. Gruber's syntax rules. The internal
implementation can be whatever we want it to be as long as the end
result gets us the expected output. Mr. Gruber himself has said that
there are some situations where markdown.pl is actually wrong and
should not be looked to as an example.

Anyway, yes in Python-Markdown inlinePatterns never see code blocks.

>
> Either way, though, I think that the other factor that kind of renders
> this moot is that any characters that get escaped will get unescaped on
> the way out, so it's as if we didn't do anything to them at all.  We're
> just hiding them from the internal processors.  And the internal
> processors can even pay attention to them if they want, since our
> mapping is consistent.  (If you see "&#x[0-9a-zA-Z]+;", you know it's an
> escaped character.  If you see "&#x26;", you know it was an &.)

Yeah, thats clear to me now. if the author typed "&#x26" it'd escape
to "&#x26#x26" and so on. I don't see any reason why we can't go with
option 2 then.

>
> > 2. Assuming the above concern is moot, the escaping and unescaping
> > should be pre and post processors respectively. Check the doc strings
> > on the latest in svn (see below) for TextPreprocessor (line 413) and
> > TextPostprocessor (line 945) which run on the entire document as a
> > single string rather than split lines.
>
> Yeah, I could see that.  Were the pre and post processors in 1.6 and I
> just missed them?

That depends on whether you have 1.6, 1.6a or 1.6b. Things kind of got
messed up with that release. Some code in that release never made it
into svn and some code in svn never made it into that release. 1.6b is
the only one I recommend people use. Unfortunately, I'm not sure it's
the one available with easy_install.

Regardless, the TextPostprocessor is brand new. In fact, the code that
removes and restores raw html are now text pre/post processors, so we
can easily avoid escaping on raw html if we use them.

> As a side note, I'm running against 1.6, because that
> was what got installed when I typed "easy_install markdown".  :)

Either tonight to tomorrow I should be releasing 1.7 (currently at a
release candidate - so I don't expect your patch to make it in --
sorry)

>
> > I appreciate all your hard work on this. However, it would be helpful
> > if your diffs were against the latest in svn. There's been quite a few
> > changes since 1.6.
>
> Fair enough.  I'll see when I next have some free time, and give that a try.
>
> > You can checkout from here:
> >      svn co https://python-markdown.svn.sourceforge.net/svnroot/python-markdown
> > python-markdown
> > Which includes the test suite so you can check whether your breaking
> > anything else.
>
> Ah, I was wondering where that was...  Okay, I've got a copy in my
> python 2.5 installation, and all the tests pass, so I should be good to
> go with my modifications, I believe.
>
> Thanks,
> Blake.
>
>



-- 
----
Waylan Limberg
wa...@gm...