python-markdown-discuss

[Python-markdown-discuss] Bug or Feature in link handling

[G. M. <gr...@mu...>]

Hello,
i recently tried if i could hijack my own site to test if it is secure enough.


I tried to post the following snippet:

    [What is the answer?](javascript:alert(42);)

but this would simply give me

    <p><a href="javascript:alert(1">test</a>;)\n</p>


Though i have discovered that this is achievable with the perl
implementation of markdown with a backslash in front of the first )
It looks like that:

    [What is the answer?](javascript:alert(42\);)

and it works!
Ok it's cool that there cannot be any javascript with parantheses...
but this would also prevent users to post links to wikipedia like
"http://en.wikipedia.org/wiki/Phone_(disambiguation)".


Is this a bug? Or a feature? Or a not quite well defined thing in the
markdown specs?


Thanks for your attention :-)

Gregor

Re: [Python-markdown-discuss] Bug or Feature in link handling

[Blake W. <bw...@la...>]

Gregor Müllegger wrote:
> I tried to post the following snippet:
>     [What is the answer?](javascript:alert(42);)
> but this would simply give me
>     <p><a href="javascript:alert(1">test</a>;)\n</p>
> but this would also prevent users to post links to wikipedia like
> "http://en.wikipedia.org/wiki/Phone_(disambiguation)".

I got hit by the same thing this morning, when I tried to post a link 
to: 
http://maps.google.com/maps?f=q&hl=en&geocode=&q=Summerhill+and+MacLennan&sll=43.687177,-79.371672&sspn=0.021661,0.037594&ie=UTF8&t=h&z=16 
and watched it completely fail.  I ended up posting a link to 
http://tinyurl.com/69xc2l which is kind of the same thing, but not really.

Later,
Blake.

Re: [Python-markdown-discuss] Bug or Feature in link handling

[Yuri T. <qar...@gm...>]

First to Gregor:

For [foo](bar(\)) Python Markdown actually behaves just like the most
recent Perl implementation.

http://babelmark.bobtfish.net/?markdown=[foo](bar(\))%0D%0A

For [foo](bar()), Python Markdown gives you different, and arguably
less intelligent HTML than other implementations:

http://babelmark.bobtfish.net/?markdown=[foo](bar())%0D%0A

However, other implementations only treat URLs with parenthesis
intelligently if the parentheses are balanced, and there is a simple
alternative way to link to URLs that have parentheses in them, which
is supported by all implementations:

    [foo][bar]

    [bar]: http://localhost/bar().html

(see http://babelmark.bobtfish.net/?markdown=[foo][bar]%0D%0A%0D%0A[bar]%3A+http%3A%2F%2Flocalhost%2Fbar().html%0D%0A)

Yes, this works for Javascript too:

    [foo][alert]

    [alert]: javascript:alert(42)

http://babelmark.bobtfish.net/?markdown=[foo][alert]%0D%0A%0D%0A[alert]%3A+javascript%3Aalert(42)%0D%0A

Does this allow people to do nasty stuff?  Yes.  However, the
consensus on the markdown-discuss list seems to be that preventing XSS
attacks is not Markdown's job.  The reason is that
javascript:alert(42) is just the tip of the iceberg when it comes to
cross-site scripting.  If you are worried about cross-site scripting,
you should get a good XSS filter and run markdown output through it.
And in my opinion, the only way to do it right is to parse the output
and filter it so that only stuff that you know is safe passes through.
 You can't fight XSS by black-listing a few keywords like
"javascript".

Now, given that we already have a "safe" option that filters out
user's HTML, I would be open to also stripping out (in "safe" mode)
any links that do not start with one of a small number of prefixes
known to be (relatively) safe (e.g., "/", "#", "http://", "https://",
"mailto://").  However, this would only make sense in "safe" mode,
when user-supplied HTML is already being removed.

To Blake:

> http://maps.google.com/maps?f=q&hl=en&geocode=&q=Summerhill+and+MacLennan&sll=43.687177,-79.371672&sspn=0.021661,0.037594&ie=UTF8&t=h&z=16
> and watched it completely fail

In this case, I don't actually see what the problem would be.  It
seems to work fine for me.

 - yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Bug or Feature in link handling

[Waylan L. <wa...@gm...>]

First of all, the parentheses in links issue is a known bug with an
existing ticket[1]. A patch is most welcome. That said, Yuri pointed
out a few ways to work around that limitation.

[1]: http://www.freewisdom.org/projects/python-markdown/Tickets/000004

On Thu, Jul 17, 2008 at 5:45 PM, Yuri Takhteyev <qar...@gm...> wrote:
>
> Now, given that we already have a "safe" option that filters out
> user's HTML, I would be open to also stripping out (in "safe" mode)
> any links that do not start with one of a small number of prefixes
> known to be (relatively) safe (e.g., "/", "#", "http://", "https://",
> "mailto://").  However, this would only make sense in "safe" mode,
> when user-supplied HTML is already being removed.
>

Something quite similar to this was checked in [2] a few months back.
I considered doing exactly as you suggested, but it seemed a little
too restrictive so I used pythons url parser to leave a little more
flexibility. In any event, it is only available in safe_mode. See the
docstring in the patch for an explanation.

I'm not completely convinced it covers every possibility. Actually as
http://ha.ckers.org/xss.html points out, there very well may be as yet
undiscovered possibilities that we don't know to check for. In any
event, for anyone that cares about this issue, that is an interesting
read. If anyone has any improvements and/or suggestions, I'm open.

[2]: http://gitorious.org/projects/python-markdown/repos/mainline/commits/2db5d1c8e469d2943a6a851bc0ff3ede070e448b

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Bug or Feature in link handling

[Yuri T. <qar...@gm...>]

> Something quite similar to this was checked in [2] a few months back.
> I considered doing exactly as you suggested, but it seemed a little
> too restrictive so I used pythons url parser to leave a little more
> flexibility. In any event, it is only available in safe_mode. See the
> docstring in the patch for an explanation.

Oh, indeed:

$ cat > test.txt
[foo][alert]

[alert]: javascript:alert(42)

$ python markdown.py -s remove test.txt
<p><a href="">foo</a>

Perhaps what we need is the documentation...

> I'm not completely convinced it covers every possibility. Actually as
> http://ha.ckers.org/xss.html points out, there very well may be as yet
> undiscovered possibilities that we don't know to check for.

Yes, I don't think this is safe - it assumes the behavior of a
standards-complient browser, but won't prevent some XSS attacks that
target IE6.

I think being both flexible and secure is a balancing act that is best
left for a good XSS filter.  I don't think it's our job to write one.
(Is there a good one for python that we can just recommend for people
to use?)  To the extent that we implement a "safe" mode, I think we
should go for the most restrictive approach.  If we are not (pretty
much) sure that it's safe, through it out in safe mode.  For me, this
means the URL should start with "http://", "https://", "/" or "#".

- yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Bug or Feature in link handling

[Yuri T. <qar...@gm...>]

> I have markdown installed with the easy_install command directly from the
> cheese shop, though I think I have a recent version (1.7). But while trying
> out your examples above, my installation fails:
>
>        >>> import markdown
>        >>> markdown.markdown(r'[foo](bar())')
>        u'<p><a href="bar(">foo</a>)\n</p>'
>        >>> markdown.markdown(r'[foo](bar(\))')
>        u'<p>[foo](bar())\n</p>'
>
> Does babelmark have anathor version installed? They say that they also have
> 1.7

I think you have the right version of Python markdown but the wrong
(old) version of Perl.

[foo](bar()) => <p><a href="bar(">foo</a>)\n</p>

This is the "bug" that Waylan mentioned.  That is, we differ from
other implementations here, in that <p><a href="bar()">foo</a></p>,
would be more desirable.

[foo](bar(\)) => <p>[foo](bar())\n</p>

Apart from the \n, this is what all implementations do now.  _Old_
version (1.0.1) of Perl markdown actually generated the link.  The new
one (1.0.2b8) does not. See:

http://babelmark.bobtfish.net/?markdown=[foo](bar(\))

PHP Markdown, which for me is a more important reference, produces this:

<p>[foo](bar(&#41;)</p>

Not to suggest that there is any logic to any of those behaviors.
Markdown syntax clearly doesn't consider parentheses in inline URLs.
So, while it would be nice to be on the same page with other
implementations for the first example, I can't say this bothers me too
much.

 - yuri

-- 
http://sputnik.freewisdom.org/