[Python-markdown-discuss] New Core Parser

[Waylan L. <wa...@gm...>]

On Mon, Nov 10, 2008 at 9:23 AM, Waylan Limberg <wa...@gm...> wrote:
> On Mon, Nov 10, 2008 at 4:37 AM, Yuri Takhteyev <qar...@gm...> wrote:
>> Perhaps we can find a way to make the parser a little more flexible,
>> so that def_list extension will just need to add a hander to it,
>> rather than having to swap in a new parser.
>
> Actually, over the last few days I've put together a completely new
> core processor which works very differently. Like the other pluggable
> parts of markdown, it loops through a list (OrderedDict actually) of
> BlockProcessors and parses the source text one block at a time. An
> extension could easily add, remove, replace individual pieces of the
> parser without the problems we currently have. Additionally, it runs
> faster (by about 1 second for 1000 iterations) than the current code
> and uses less recursion. It also overcomes some of the parsing
> differances unique to the python implementation (i.e.: <p>s in lists
> match pl and php behavior) although an extension could easily provide
> the current behavior instead.
>

Well, I'm satisfied enough with what I have to release it into the
wild. I'm not convinced it's ready for prime time just yet, but I
think it's mostly there. I started this as an extension that replaced
the core parser rather than hacking on the existing code in
markdown.py. For one, it made timing against the old code easier, and
two, with none of the old code in front of me, it forced me to start
from scratch. Unfortunately, after working out the remaining edge
cases, it is barely any faster than the old code, but it is much more
customizable, so we do gain a lot IMO.

Please note that I did not even try to match the old core's behavior.
I built this to match pl and php's behavior when parsing blocks (which
means ticket 1 can finally be closed). I suspect a number of tests
will need to be reviewed and updated once I copy this over into
markdown.py.

The extension is attached as block.py. Copy it to your
markdown_extensions dir and run markdown with the 'block' extension
for testing.

Any and all feedback is welcome.

Unless I hear some strong objections, I'll start migrating this code
to markdown.py and update the tests and extensions in a branch.

-- 
----
Waylan Limberg
wa...@gm...