python-markdown-discuss

[Python-markdown-discuss] Performance issues with 2.1.0

[Alexandre L. <new...@al...>]

Dear all,

there seems to be pretty big performance issues in version 2.1.0, 
especially when dealing with raw HTML. Has anyone noticed it?

Have a look at the test script located at 
<http://stdin.fr/markdown/markdown_issue/>

Using markdown==2.0.3:

     python2 markdown_issue.py  0,17s user 0,01s system 94% cpu 0,191 total

Using markdown==2.1.0:

     python2 markdown_issue.py  3,00s user 0,03s system 99% cpu 3,048 total

And it gets worth with the longer files I deal with... I even get stuck 
indefinitely. Unfortunately it is very inconvenient for me switch back 
to 2.0.x as I rely a lot on new style attribute list... Do you have any 
idea on what is going on?

Thanks a lot,

-- Alex

Re: [Python-markdown-discuss] Performance issues with 2.1.0

[Waylan L. <wa...@gm...>]

On Wed, Dec 14, 2011 at 3:34 PM, Alexandre Leray
<new...@al...> wrote:
> Dear all,
>
> there seems to be pretty big performance issues in version 2.1.0,
> especially when dealing with raw HTML. Has anyone noticed it?
>
> Have a look at the test script located at
> <http://stdin.fr/markdown/markdown_issue/>

No I hadn't. My unscientific observations were that 2.1.0 might be
slightly faster. However, I don't do much raw html myself. In any
event, my first guess would be that the cause is the changes made here
[1] and here [2], which addressed this issue [3]. I'll try confirming
that later tonight.

I suggest reporting it on Github [4] so we don't forget about it.

[1]: https://github.com/waylan/Python-Markdown/commit/fc3020c68575042a1ff5426ddd94bc4be65bf77d
[2]: https://github.com/waylan/Python-Markdown/commit/c2d50b46b536a440b6d73c6bf309bdaf03b90abb
[3]: http://www.freewisdom.org/projects/python-markdown/Tickets/000048
[4]: https://github.com/waylan/Python-Markdown/issues/new


-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Performance issues with 2.1.0

[Waylan L. <wa...@gm...>]

Alex,

I just reverted the rawhtmlblock preprocessor and it is definitely
faster on docs with lots of html.

I pushed a commit to github on the slowhtml [1] branch.  See the diff
here [2]. I should note that a number of tests are failing and not
just the test for the new support of attr in raw html. One test is
even raising an exception. Every change to that file since the
original commit needs to be reviewed and possibly reimplemented before
this code could be considered stable.

That said, I'm not intending on committing this to master anyway. We
don't want to lose support for attributes in raw html. Perhaps a more
efficient solution than the current (long) regex would be the way
forward. Not something I have time for right now. At least we know
what is causing the slowdown.

[1]: https://github.com/waylan/Python-Markdown/tree/slowhtml
[2]: https://github.com/waylan/Python-Markdown/commit/790aeb56053940f299d600d8d364fc3fc8151085

On Wed, Dec 14, 2011 at 5:48 PM, Alexandre Leray
<new...@al...> wrote:
> Thanks Waylan for looking into this, much appreciate.
>
> waiting to hear from you,
>
> Thanks,
>
> Alex
>
>
> On 14/12/2011 22:03, Waylan Limberg wrote:
>>
>> No I hadn't. My unscientific observations were that 2.1.0 might be
>> slightly faster. However, I don't do much raw html myself. In any
>> event, my first guess would be that the cause is the changes made here
>> [1] and here [2], which addressed this issue [3]. I'll try confirming
>> that later tonight.
>>
>> I suggest reporting it on Github [4] so we don't forget about it.
>>
>> [1]:
>> https://github.com/waylan/Python-Markdown/commit/fc3020c68575042a1ff5426ddd94bc4be65bf77d
>> [2]:
>> https://github.com/waylan/Python-Markdown/commit/c2d50b46b536a440b6d73c6bf309bdaf03b90abb
>> [3]: http://www.freewisdom.org/projects/python-markdown/Tickets/000048
>> [4]: https://github.com/waylan/Python-Markdown/issues/new
>
>



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Performance issues with 2.1.0

[Waylan L. <wa...@gm...>]

On Wed, Dec 14, 2011 at 10:18 PM, Waylan Limberg <wa...@gm...> wrote:
> Not something I have time for right now. At least we know
> what is causing the slowdown.

Oh, I just created a issue on github to track this so I don't forget it:

https://github.com/waylan/Python-Markdown/issues/59

-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg