python-markdown-discuss

[Python-markdown-discuss] Not liking Mako-like syntax

[Aaron G. <fl...@sh...>]

Hi,

I was looking to use this markdown module (the 1.7RC1 at the moment)  
on some strings that also have mako template tags in them. They  
basically look like this:

<%def .... />
<%page .... />
% for x in foo:
...
% endfor

Anyways, I figured I might need to do something special for the  
strange '%' flow control structures, but I assumed that like the other  
markdown implementations it would ignore the other stuff like HTML  
tags, but it's totally choking. Here's a "test case":

<%foo />

Heading
=======


Simply returns the entire thing as-is. I'm guessing it never thought  
whatever kind of tag that was there ended up closing.

Can I fix this through the API somehow?

Aaron Gyes

Re: [Python-markdown-discuss] Not liking Mako-like syntax

[Waylan L. <wa...@gm...>]

Thanks for the report Aaron. I see you reported this in the bug
tracker too - which will make sure we don't forget it. I'll look into
this when I get a chance.

On Dec 30, 2007 6:04 AM, Aaron Gyes <fl...@sh...> wrote:
> Hi,
>
> I was looking to use this markdown module (the 1.7RC1 at the moment)
> on some strings that also have mako template tags in them. They
> basically look like this:
>
> <%def .... />
> <%page .... />
> % for x in foo:
> ...
> % endfor
>
> Anyways, I figured I might need to do something special for the
> strange '%' flow control structures, but I assumed that like the other
> markdown implementations it would ignore the other stuff like HTML
> tags, but it's totally choking. Here's a "test case":
>
> <%foo />
>
> Heading
> =======
>
>
> Simply returns the entire thing as-is. I'm guessing it never thought
> whatever kind of tag that was there ended up closing.
>
> Can I fix this through the API somehow?
>
> Aaron Gyes
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Not liking Mako-like syntax

[Aaron G. <fl...@sh...>]

Thanks!

If anyone has any ideas for a quick-and-dirty hack to make in  
markdown.py to make it act normal I would really appreciate it. I  
played around for 20 minutes never got anywhere! I think maybe it's  
not exactly in the _get_right_tag, it seems that something before that  
part is acting different. `block` contains stuff I don't see if  it  
were to be going over something like <foo />.

Aaron

On Dec 30, 2007, at 7:45 PM, Waylan Limberg wrote:

> Thanks for the report Aaron. I see you reported this in the bug
> tracker too - which will make sure we don't forget it. I'll look into
> this when I get a chance.
>
> On Dec 30, 2007 6:04 AM, Aaron Gyes <fl...@sh...> wrote:
>> Hi,
>>
>> I was looking to use this markdown module (the 1.7RC1 at the moment)
>> on some strings that also have mako template tags in them. They
>> basically look like this:
>>
>> <%def .... />
>> <%page .... />
>> % for x in foo:
>> ...
>> % endfor
>>
>> Anyways, I figured I might need to do something special for the
>> strange '%' flow control structures, but I assumed that like the  
>> other
>> markdown implementations it would ignore the other stuff like HTML
>> tags, but it's totally choking. Here's a "test case":
>>
>> <%foo />
>>
>> Heading
>> =======
>>
>>
>> Simply returns the entire thing as-is. I'm guessing it never thought
>> whatever kind of tag that was there ended up closing.
>>
>> Can I fix this through the API somehow?
>>
>> Aaron Gyes
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Python-markdown-discuss mailing list
>> Pyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>>
>
>
>
> -- 
> ----
> Waylan Limberg
> wa...@gm...

Re: [Python-markdown-discuss] Not liking Mako-like syntax

[Waylan L. <wa...@gm...>]

Well,  the proper way would be to write your own pre-processor that
strips out all the mako markup and adds it to the htmlstash. That is,
after all, how html is currently ignored (in HtmlBlockPreprocessor -
line 486).

With only a quick look, my guess is that the problem stems from the
fact that python-markdown has a very limited understanding of xml.
Therefore, the `%`s are throwing it off. In fact, if you try your
given test case without the `%` it works fine:

    <foo />

    Heading
    ======

For a quick and dirty hack you could just add a special condition to
check for `%` (similar to the check for html comments) right in
HtmlBlockPreprocessor. However, I doubt that would ever make it into
the distribution [^1]. After all, there are various non-xml based
template languages which would still not be supported. That said, it
wouldn't hurt to at least explore the possibility of better xml
parsing. That, however, is another issue for another time. And that is
why the proper approach is to write your own extension with a
preprocessor that strips the markup for your preferred template
language.

[^1]: I realize that other markdown implementation may currently work
with mako syntax. However, I believe it likely that that is an
unexpected side affect of the way their respective html/xml parsers
are implemented rather than a specific feature that was intentionally
built in. That being the case, any code specifically to address one
single template language's syntax belongs in an extension rather than
markdown proper.

On Dec 31, 2007 1:16 AM, Aaron Gyes <fl...@sh...> wrote:
> Thanks!
>
> If anyone has any ideas for a quick-and-dirty hack to make in
> markdown.py to make it act normal I would really appreciate it. I
> played around for 20 minutes never got anywhere! I think maybe it's
> not exactly in the _get_right_tag, it seems that something before that
> part is acting different. `block` contains stuff I don't see if  it
> were to be going over something like <foo />.
>
> Aaron
>
>
> On Dec 30, 2007, at 7:45 PM, Waylan Limberg wrote:
>
> > Thanks for the report Aaron. I see you reported this in the bug
> > tracker too - which will make sure we don't forget it. I'll look into
> > this when I get a chance.
> >
> > On Dec 30, 2007 6:04 AM, Aaron Gyes <fl...@sh...> wrote:
> >> Hi,
> >>
> >> I was looking to use this markdown module (the 1.7RC1 at the moment)
> >> on some strings that also have mako template tags in them. They
> >> basically look like this:
> >>
> >> <%def .... />
> >> <%page .... />
> >> % for x in foo:
> >> ...
> >> % endfor
> >>
> >> Anyways, I figured I might need to do something special for the
> >> strange '%' flow control structures, but I assumed that like the
> >> other
> >> markdown implementations it would ignore the other stuff like HTML
> >> tags, but it's totally choking. Here's a "test case":
> >>
> >> <%foo />
> >>
> >> Heading
> >> =======
> >>
> >>
> >> Simply returns the entire thing as-is. I'm guessing it never thought
> >> whatever kind of tag that was there ended up closing.
> >>
> >> Can I fix this through the API somehow?
> >>
> >> Aaron Gyes
> >>
> >> -------------------------------------------------------------------------
> >> This SF.net email is sponsored by: Microsoft
> >> Defy all challenges. Microsoft(R) Visual Studio 2005.
> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> >> _______________________________________________
> >> Python-markdown-discuss mailing list
> >> Pyt...@li...
> >> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> >>
> >
> >
> >
> > --
> > ----
> > Waylan Limberg
> > wa...@gm...
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Not liking Mako-like syntax

[Yuri T. <yu...@cs...>]

On Dec 31, 2007 10:49 AM, Waylan Limberg <wa...@gm...> wrote:
> Well,  the proper way would be to write your own pre-processor that
> strips out all the mako markup and adds it to the htmlstash. That is,
> after all, how html is currently ignored (in HtmlBlockPreprocessor -
> line 486).
>
> With only a quick look, my guess is that the problem stems from the
> fact that python-markdown has a very limited understanding of xml.
> Therefore, the `%`s are throwing it off. In fact, if you try your
> given test case without the `%` it works fine:

First, for the record, <%foo /> doesn't count as XML.  Second, yes,
the _right_ way to do this is to write a custom preprocessor.

However, my intension was to support this, so I suppose this is a bug.
 Let me explain how it is _supposed_ to work:

When the script encounters an HTML tag in the beginning of the block,
it classifies the tag into three types:

1. Tags that open an HTML block, such as <table> or <pre>.  In this
case, there will be no markdown processing until the block is closed
with the same tag.  There will also be no "<p>..</p>" around this
chunk of html.
2. Tags like <div> or <?php>.  In this case we don't look for the
closing tag, but we also avoid putting <p>...</p> around the block.
3. Inline tags.  We don't look for where the tag closes and we put
<p>...</p> around the block.

#2 is supposed to include "div", as well as any tags starting with
"%", "?".  The rationale is that for something like <?php> or <%foo>
we don't really know what that tag means and we don't want to make any
assumption as to whether it needs to be closed _or_ whether it needs
<p> around it.  If you want "<p>" around your <%foo/>, you can put it
manually, after all, but you won't be able to remove it easily if we
add it.

So, "%foo" and "foo" are supposed to be treated differently.
"<foo>**bar**" should become "<p><foo><b>bar</b><p>" while
"<%foo>**bar**" is supposed to become as "<%foo><b>bar</b>"

It obviously doesn't work that way.  I think that the bug is either
_get_right_tag() or _equal_tag().  The quick fix is to change line 499
to

        if left_tag in ['div'] or left_tag[0] in ["?", "@", "%"]: #
handle PHP, etc.

Though, at some point, the whole HTML_Block_Preprocessor logic needs
to be rethought.  It seems more convoluted than it needs to be.

I hope this helps.

- yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Not liking Mako-like syntax

[Aaron G. <fl...@sh...>]

Thanks Yuri and Waylan for the explanations. Especially thanks for the  
code change that's got me able to resume work on my project!

Aaron Gyes