python-markdown-discuss — A general purpose discussion list

Re: [Python-markdown-discuss] Limit parser to a subset of the syntax

[Waylan L. <wa...@gm...>]

On Tue, May 4, 2010 at 8:08 PM, Tom Ritter <to...@ri...> wrote:
> This isn't the best solution, but I disable images by using this extension:
>
> import markdown
> from markdown import etree
>
> class DisableImagesExtension(markdown.Extension):
>    def extendMarkdown(self, md, md_globals):
>        md.treeprocessors.add('disableImages', DisableImages(md), '_end')
>
> class DisableImages(markdown.treeprocessors.Treeprocessor):
>    def descendRemove(self, element):
>        for i in element:
>            if i.tag == 'img':
>                element.remove(i)
>            else:
>                self.descendRemove(i)
>    def run(self, root):
>        self.descendRemove(root)
>        return root
>

How about this:

    import markdown

    class DisableImagesExtension(markdown.Extension):
        def extendMarkdown(self, md, md_globals):
            del md.inlinePatterns["link"]
            del md.inlinePatterns["image_reference"]

That's it. The image syntax will not never be parsed - which is more
of a markdown type behavior.  Any unrecognized syntax should go
through unaltered. The names of all the patterns and processors can be
found where they are added in the` Markdown.__init__` method of the
`markdown/__init__.py` file.

-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Limit parser to a subset of the syntax

[Waylan L. <wa...@gm...>]

See the documentation for writing extensions[1]. Using that API, you
can alter any part of the parser. Simply remove the processors and
patterns that you do not want. See the section on OrderedDict for how
to do that.

[1]: http://www.freewisdom.org/projects/python-markdown/Writing_Extensions

2010/5/4 Hernâni Cerqueira <he...@dz...>:
> Hello,
>
> Is there anyway to limit the markdown to html conversion to a small
> subset of the markdown syntax? Let's say I want to allow only italics,
> bolds and links, is that possible?
>
> Thanks in advance
> Hernani
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Limit parser to a subset of the syntax

[Tom R. <to...@ri...>]

This isn't the best solution, but I disable images by using this extension:

import markdown
from markdown import etree

class DisableImagesExtension(markdown.Extension):
    def extendMarkdown(self, md, md_globals):
        md.treeprocessors.add('disableImages', DisableImages(md), '_end')

class DisableImages(markdown.treeprocessors.Treeprocessor):
    def descendRemove(self, element):
        for i in element:
            if i.tag == 'img':
                element.remove(i)
            else:
                self.descendRemove(i)
    def run(self, root):
        self.descendRemove(root)
        return root


-tom

[Python-markdown-discuss] Limit parser to a subset of the syntax

[Hernâni C. <he...@dz...>]

Hello,

Is there anyway to limit the markdown to html conversion to a small 
subset of the markdown syntax? Let's say I want to allow only italics, 
bolds and links, is that possible?

Thanks in advance
Hernani

Re: [Python-markdown-discuss] Markdown syntax in block level tags

[Waylan L. <wa...@gm...>]

That is a new work in progress which was only started within the last
few months. It actually is enabled with the extra extension as it
exists in the repo. But, as you noted, it is a little buggy. Once the
bugs are worked out, documentation will be added, and after the next
release, that documentation will be copied to the official docs.

Of course, patches are welcome.

On Tue, May 4, 2010 at 12:05 PM, Kolya Ay <kol...@gm...> wrote:
> Hi,
>
> I have traversed the source of git master version of Markdown and
> noticed that there is an option to use markdown syntax in block level
> tags.
> So i'm setting attribute `markdown_in_raw` of `HtmlBlockPreprocessor`
> to `True` and trying to make html-table:
>
>    This *table* is not too complicated:
>
>    <table markdown>
>       <tr><td>Hello, **world**!</td></tr>
>    <table>
>
> This produces the following result:
>
>    <p>This <em>table</em> is not too complicated:</p>
>    <table>
>
>     <p><tr><td>Hello, <strong>world</strong>!</td></tr>
>      <table></p>
>
> Note extra <p> elements in theresult.
> Is this behavior a bug that I must submit? Why markdown_in_raw option
> is not documented?
>
> Regards,
> Kolya
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] Markdown syntax in block level tags

[Kolya Ay <kol...@gm...>]

Hi,

I have traversed the source of git master version of Markdown and
noticed that there is an option to use markdown syntax in block level
tags.
So i'm setting attribute `markdown_in_raw` of `HtmlBlockPreprocessor`
to `True` and trying to make html-table:

    This *table* is not too complicated:

    <table markdown>
       <tr><td>Hello, **world**!</td></tr>
    <table>

This produces the following result:

    <p>This <em>table</em> is not too complicated:</p>
    <table>

     <p><tr><td>Hello, <strong>world</strong>!</td></tr>
      <table></p>

Note extra <p> elements in theresult.
Is this behavior a bug that I must submit? Why markdown_in_raw option
is not documented?

Regards,
Kolya

Re: [Python-markdown-discuss] blockquotes

[Gerry L. <gjl...@al...>]

> In other words, follow the syntax rules, not his code when the two
> appear to be in conflict. I believe this may fit that type of
> situation.
> 
> I would call this a bug in markdown.pl (and any other implementation
> that follows it), not a bug in python-markdown.

The notion of following the syntax rules is the only potential gotcha
here.  I looked high and low for an explicit statement regarding usage
of the blockquote.  I came up empty.

The implication seems to be that blockquote processing is triggered if
the '>' character is the first non-whitespace character on the line.
However, if a blank space is added between list items (i.e. create a
list with paragraphs) with blockquotes, then the blockquote is processed
in most implementations, including markdown.pl.

So, it seems it comes back to what the reasonable output expectation
would be for text formatted like that.  Certainly, python-markdown's
handling of this (if not edge case, then low usage) format can be
considered reasonable.  So count me in agreement.

Re: [Python-markdown-discuss] blockquotes

[Waylan L. <wa...@gm...>]

On Tue, Mar 23, 2010 at 1:04 PM, Gerry LaMontagne
<gjl...@al...> wrote:
>
> Sorry if this is getting into pain-in-the-a** territory, but I've got
> something that I think is less a bug and more a matter of
> interpretation.  The following example:
>
> 1. > Quote1
> 2. > Quote2
>
> is turned into a list with blockquotes in the list items by
> python-markdown.  However, markdown.pl does not create blockquotes.
> Instead, it encodes the '>' character as html entities and treats the
> list item as text:
>
> <li>&gt; Quote1</li>
>
> I'm aware that this is out in the weeds as far as usage is concerned.
> But, seeing as I found it, I thought I'd do diligence and ask if it's
> something that you want dealt with.
>

At some point in the past, John Gruber had stated that there may be as
yet undiscovered bugs in his implementation and that just because his
implementation acts a certain way, doesn't mean it is always correct.
In other words, follow the syntax rules, not his code when the two
appear to be in conflict. I believe this may fit that type of
situation.

I would call this a bug in markdown.pl (and any other implementation
that follows it), not a bug in python-markdown.

If anyone feels differently, please speak up.

--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] blockquotes

[Gerry L. <gjl...@al...>]

Sorry if this is getting into pain-in-the-a** territory, but I've got
something that I think is less a bug and more a matter of
interpretation.  The following example:

1. > Quote1
2. > Quote2

is turned into a list with blockquotes in the list items by
python-markdown.  However, markdown.pl does not create blockquotes.
Instead, it encodes the '>' character as html entities and treats the
list item as text:

<li>&gt; Quote1</li>

I'm aware that this is out in the weeds as far as usage is concerned.
But, seeing as I found it, I thought I'd do diligence and ask if it's
something that you want dealt with.

Regards-
Gerry LaMontagne

Re: [Python-markdown-discuss] Possible blockquote bug?

[Waylan L. <wa...@gm...>]

On Mon, Mar 22, 2010 at 3:01 PM, Gerry LaMontagne <
gjl...@al...> wrote:

> In the context of considering a fix for ticket 58, I tried the following:
>
> * > something meaningless
>
>    * more gibberish
>
> * and so forth
>    * and so on
>
> * I think
>    * you've got the idea
>
> I was expecting the blockquoted section to be put in p-tags, but they
> weren't.  So I visited babelmark and sure enough, they do place the quoted
> text in p tags.  Then I started playing- it appears the blockquoted text
> is **always** supposed to be placed in p tags?  Further, p-m never puts
> blockquoted text in p tags when part of a list item.
>
> Shall I open another ticket here? or just include a fix as part of the
> current
> ticket?
>

It seems closely related enough to just put it all together in one fix if
you want.

-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] Possible blockquote bug?

[Gerry L. <gjl...@al...>]

In the context of considering a fix for ticket 58, I tried the following:

* > something meaningless

    * more gibberish

* and so forth
    * and so on

* I think
    * you've got the idea

I was expecting the blockquoted section to be put in p-tags, but they
weren't.  So I visited babelmark and sure enough, they do place the quoted
text in p tags.  Then I started playing- it appears the blockquoted text
is **always** supposed to be placed in p tags?  Further, p-m never puts 
blockquoted text in p tags when part of a list item.

Shall I open another ticket here? or just include a fix as part of the current
ticket?  

Regards-
Gerry LaMontagne

Re: [Python-markdown-discuss] Tickets 57 and 53

[Waylan L. <wa...@gm...>]

On Sat, Mar 20, 2010 at 9:42 AM, Gerry LaMontagne
<gjl...@al...> wrote:
> Waylan-
>
> Only because you mentioned copying prior discussions in the last email-
> I've still got our original correspondence pertaining to the previous
> tickets.  Should I copy the mailing list for posterity's sake with those
> emails?

No, that won't be necessary. We have Tickets which record the problems
and I never delete my email, so I can always find them if needed. Now
that the porblems are resolved, it would just be noise for everyone.


-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] Tickets 57 and 53

[Gerry L. <gjl...@al...>]

Waylan-

Only because you mentioned copying prior discussions in the last email-
I've still got our original correspondence pertaining to the previous
tickets.  Should I copy the mailing list for posterity's sake with those
emails?

Regards-
Gerry LaMontagne

Re: [Python-markdown-discuss] Ticket #000057

[Waylan L. <wa...@gm...>]

On Thu, Mar 18, 2010 at 11:00 PM, Gerry LaMontagne
<gjl...@al...> wrote:
>
> Alright, I've got 2 different possible fixes for you to evaluate.

Thanks. I've committed your patch2.diff as it more closely follows the
existing pattern in the parser.

> I haven't had a chance to dig into the other item I noted previous, but
> it's on my todo list.

Cool. A few things :

I've created Ticket 58 [1] to track that issue.

If you create an account and clone [2] at gitourious.org, you can push
your commits up, and make merge requests [3].  Then I can merge your
commits and you will get full credit in the log as the commiter (not
just via my comments). Of course, if you'd rather not, that's fine
too.

Could you continue any further discussions on the mailing list [4] so
others can pitch in if they want and so that we have an archive of how
the solution was reached (I'v copied the list with this response.
Probably should have done this earlier).

Your time and energy on this is **very much** appreciated. I've barely
had the time to review and commit your patches.  There's no way I
would have had the time to debug the problems and hack the patches
together myself.

[1]: http://www.freewisdom.org/projects/python-markdown/Tickets/000058
[2]: http://gitorious.org/python-markdown/mainline/clone
[3]: http://gitorious.org/python-markdown/mainline/merge_requests
[4]: https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss

--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Interesting bug resulting in INLINE_PLACEHOLDER left in output

[Waylan L. <wa...@gm...>]

On Mon, Feb 22, 2010 at 1:04 PM, Waylan Limberg <wa...@gm...> wrote:
[snip]
>
> (2) Have the inline raw html  pattern check for any placeholders and
> swap them out.
>
[snip]

FYI, I just pushed a patch up that fixes this. See it here:

http://gitorious.org/python-markdown/mainline/commit/996bcdd332551b82791ab0851e2f7f826bc0a4ed



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Looking for advices to write a markdown plugin

[Alexandre L. <new...@al...>]

Thanks Yuri, much appreciate your help

On 02/18/2010 06:19 PM, Yuri Takhteyev wrote:
> Anyway, if I were you, I would focus on identifying and re-using
> existing document structure. E.g., allowing people to attach comments
> to sections/headings, paragraphs, list items, sentences, quotations,
> etc.

I agree with you I have to work on the HTML structure. For instance I 
recently uploaded an interview formatted as a big <dl/> list. In its 
current development our software considered that definition list to be a 
single element to comment since it was enclosed by a single <dl></dl> 
tag whereas it would make much more sense to break it on each <dt/> 
</dd> pair.

> If this isn't enough, then you can think of two separate markups
> - one for identifying sub-paragraph sections, and another one for
> grouping paragraphs.
>
>    - yuri
>    

The previous solution doesn't solve the arbitrary slicing issue, and we 
will need to define a syntax for that. It is still unclear to me what 
those markups could be. Do you have anything in mind?

Best,

Alexandre

Re: [Python-markdown-discuss] Looking for advices to write a markdown plugin

[Alexandre L. <new...@al...>]

Thanks Yuri, much appreciate your help

On 02/18/2010 06:19 PM, Yuri Takhteyev wrote:
> Anyway, if I were you, I would focus on identifying and re-using
> existing document structure. E.g., allowing people to attach comments
> to sections/headings, paragraphs, list items, sentences, quotations,
> etc.

I agree with you I have to work on the HTML structure. For instance I 
recently uploaded an interview formatted as a big <dl/> list. In its 
current development our software considered that definition list to be a 
single element to comment since it was enclosed by a single <dl></dl> 
tag whereas it would make much more sense to break it on each <dt/> 
</dd> pair.

> If this isn't enough, then you can think of two separate markups
> - one for identifying sub-paragraph sections, and another one for
> grouping paragraphs.
>
>    - yuri
>    

The previous solution doesn't solve the arbitrary slicing issue, and we 
will need to define a syntax for that. It is still unclear to me what 
those markups could be. Do you have anything in mind?

Best,

Alexandre

Re: [Python-markdown-discuss] Interesting bug resulting in INLINE_PLACEHOLDER left in output

[Waylan L. <wa...@gm...>]

On Mon, Feb 22, 2010 at 11:31 AM, Yuri Takhteyev <qar...@gm...> wrote:
> What happens is that \] is turned into placeholder, which gives us
> "<x\x02klzzwxh:0000\x03>". Then <x\x02klzzwxh:0000\x03> is picked up
> as an HTML pattern and stashed away, which prevents the placeholder
> from being replaced back with "]".
>
> I am guessing we should make HTML_RE more restrictive.

Actually it is a little more complicated that that. We are removing
raw html in two places. In a preprocessor we find and remove block
level html (div, p, etc.). Then in the inlinepatterns we remove inline
html (span, em, etc) - actually the inline pattern removes anything
wrapped in <> that still remains in the document. Which means it has
to run after the inlinepatterns for autolink and automail. However,
for reasons which should be obvious, escaping needs to happen before
autolink and automail. At the same time, escaping should not be run on
raw html.

I see to possible fixes:

(1) Move inline raw html detection to the preprocessor.

(2) Have the inline raw html  pattern check for any placeholders and
swap them out.

Option 2 would certainly be the easiest to implement. But there is a
part of me that wants to do option 1. A little while back I actually
started a new preprocessor that used Python's builtin real html
parser. It actually worked - until I included autolinks/automails in a
document. That broken implementation is in a branch on Gitorious if
anyone is interested. Unfortunately, I failed to commit when I had
working code and don't remember what I did that broke it when trying
(unsuccessfully) to work around the autolink issue. When I realized
what I did, I just committed everything as is and figured I come back
to it later.



>  - yuri
>
> On Mon, Feb 22, 2010 at 10:57 AM, Yuri Takhteyev <qar...@gm...> wrote:
>> Interesting. Here is a much simpler test case triggering this:
>>
>>    md.convert("<x\]>")
>>
>> Even without any extensions, using the version from git, we get:
>>
>>    u'<x\x02klzzwxh:0000\x03>'
>>
>> - yuri
>>
>> On Mon, Feb 22, 2010 at 9:08 AM, Tom Ritter <to...@ri...> wrote:
>>> klzzwxh:0000
>>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Interesting bug resulting in INLINE_PLACEHOLDER left in output

[Yuri T. <qar...@gm...>]

What happens is that \] is turned into placeholder, which gives us
"<x\x02klzzwxh:0000\x03>". Then <x\x02klzzwxh:0000\x03> is picked up
as an HTML pattern and stashed away, which prevents the placeholder
from being replaced back with "]".

I am guessing we should make HTML_RE more restrictive.

  - yuri

On Mon, Feb 22, 2010 at 10:57 AM, Yuri Takhteyev <qar...@gm...> wrote:
> Interesting. Here is a much simpler test case triggering this:
>
>    md.convert("<x\]>")
>
> Even without any extensions, using the version from git, we get:
>
>    u'<x\x02klzzwxh:0000\x03>'
>
> - yuri
>
> On Mon, Feb 22, 2010 at 9:08 AM, Tom Ritter <to...@ri...> wrote:
>> klzzwxh:0000
>

Re: [Python-markdown-discuss] Interesting bug resulting in INLINE_PLACEHOLDER left in output

[Yuri T. <qar...@gm...>]

Interesting. Here is a much simpler test case triggering this:

    md.convert("<x\]>")

Even without any extensions, using the version from git, we get:

    u'<x\x02klzzwxh:0000\x03>'

- yuri

On Mon, Feb 22, 2010 at 9:08 AM, Tom Ritter <to...@ri...> wrote:
> klzzwxh:0000

[Python-markdown-discuss] Interesting bug resulting in INLINE_PLACEHOLDER left in output

[Tom R. <to...@ri...>]

I submitted this bug:
http://www.freewisdom.org/projects/python-markdown/Tickets/000052

I'm using markdown to power a self-rolled comment system on my blog
(more info here: http://ritter.vg/code_adventures_site.html#rev6 ) and
I challenged people to exploit it.  Someone managed to submit a
comment that broke markdown.  The output comes back with klzzwxh:0000
which includes the u'\u0002' and u'\u0003' strings.  This in turn
broke chrome's json parser.

I tried fiddling with the input (changing brackets and the like) but
it's fairly specific.  Let me know if you need any more info - I'm on
gentoo with the latest ebuild:

[I] dev-python/markdown
     Installed versions:  2.0.3(15:21:58 02/06/10)(-pygments)

Although strangely I see this:

# python
Python 2.6.4 (r264:75706, Dec  7 2009, 18:16:11)
[GCC 4.1.2 (Gentoo 4.1.2 p1.3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import markdown
>>> markdown.version
'2.0.1'

I haven't investigated gentoo's ebuild but something's definetly fishy there...

-tom

Re: [Python-markdown-discuss] Proposal: Switching tests to nose

[Yuri T. <qar...@gm...>]

> Still not convinced?

Ok, convinced. :)

  - yuri

Re: [Python-markdown-discuss] Looking for advices to write a markdown plugin

[Yuri T. <qar...@gm...>]

> thanks a lot for your interest.
> I think it would be more like separators, as I don't have the need to
> exclude certain parts from being commented.

Using a single separator to sometimes break up paragraphs into smaller
units and sometimes merge them into larger ones is going to get ugly
quickly... I recommend thinking of an approach that will ensure that
commentable sections do not break the boundaries of HTML tags. I.e.,
if a section includes the start tag for an HTML block, it should
include the end tag as well. It also sounds like you want to have
commentable section overlap. So, a single character separator won't
work. You will at least need a pair of characters - one to open a
commentable section and one to close.

Anyway, if I were you, I would focus on identifying and re-using
existing document structure. E.g., allowing people to attach comments
to sections/headings, paragraphs, list items, sentences, quotations,
etc. If this isn't enough, then you can think of two separate markups
- one for identifying sub-paragraph sections, and another one for
grouping paragraphs.

  - yuri

Re: [Python-markdown-discuss] Looking for advices to write a markdown plugin

[Alexandre L. <new...@al...>]

Dear Ian,

thanks for pointing me to this project, I didn't know it and it seems 
very interesting!
However I don't think it covers my needs as I'd like to keep the 
comments along with the texts.

Best,

Alexandre

On 02/17/2010 09:42 PM, Ian Stokes-Rees wrote:
> Depending on your goals, you may find that simply using a system such as
> ReFrameit will provide similar functionality.
>    

Re: [Python-markdown-discuss] Looking for advices to write a markdown plugin

[Ian Stokes-R. <ijs...@hk...>]

Hello Alexandre,

On 2/17/10 10:56 AM, Alexandre Leray wrote:
> Hello,
>
> I wrote a simple text-publishing Django application enabling per 
> paragraph comments. It splits a texts into chunks and let people comment 
> them. The project is hosted at http://github.com/aleray/close-commenting 
> and a demo is visible there: 
> http://closecommenting.stdin.fr/one-possible-scenario-for-a-collective-future.html
>   

Depending on your goals, you may find that simply using a system such as
ReFrameit will provide similar functionality.

Ian

-- 
Ian Stokes-Rees, PhD                       W: http://abitibi.sbgrid.org
ijs...@hk...               T: +1.617.432.5608 x75
NEBioGrid, Harvard Medical School          C: +1.617.331.5993