python-markdown-discuss — A general purpose discussion list

Re: [Python-markdown-discuss] New Core Parser

[Yuri T. <qar...@gm...>]

> Well, I'm satisfied enough with what I have to release it into the
> wild. I'm not convinced it's ready for prime time just yet, but I
> think it's mostly there. I started this as an extension that replaced
> the core parser rather than hacking on the existing code in
> markdown.py. For one, it made timing against the old code easier, and
> two, with none of the old code in front of me, it forced me to start
> from scratch. Unfortunately, after working out the remaining edge
> cases, it is barely any faster than the old code, but it is much more
> customizable, so we do gain a lot IMO.

Yes, this certainly looks more customizable and is also easier to
understand.  Good job!  If we can switch to this implementation
without breaking too many tests, then let's just do this.

A few small details:

1. Now that this is fresh in your head, please add comments on how this works.
2. I would get rid of all abbreviations.  E.g., I would make it
"SetexHeaderProcessor" instead of "SHeaderProcessor".
3. I would replace 4 with TAB_LENGTH everywhere.

- yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] New Core Parser

[Waylan L. <wa...@gm...>]

On Tue, Nov 11, 2008 at 11:42 AM, Waylan Limberg <wa...@gm...> wrote:
>
> Well, I'm satisfied enough with what I have to release it into the
> wild. I'm not convinced it's ready for prime time just yet, but I
> think it's mostly there. I started this as an extension that replaced
> the core parser rather than hacking on the existing code in
> markdown.py. For one, it made timing against the old code easier, and
> two, with none of the old code in front of me, it forced me to start
> from scratch. Unfortunately, after working out the remaining edge
> cases, it is barely any faster than the old code, but it is much more
> customizable, so we do gain a lot IMO.
>

Oh, one thing I forgot to mention is that I have disabled the header
and hr preprocessors. The new core parser handles that stuff itself
and does so faster than the preprocessors did. A small, but noteworthy
detail I think.



-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] New Core Parser

[Waylan L. <wa...@gm...>]

On Mon, Nov 10, 2008 at 9:23 AM, Waylan Limberg <wa...@gm...> wrote:
> On Mon, Nov 10, 2008 at 4:37 AM, Yuri Takhteyev <qar...@gm...> wrote:
>> Perhaps we can find a way to make the parser a little more flexible,
>> so that def_list extension will just need to add a hander to it,
>> rather than having to swap in a new parser.
>
> Actually, over the last few days I've put together a completely new
> core processor which works very differently. Like the other pluggable
> parts of markdown, it loops through a list (OrderedDict actually) of
> BlockProcessors and parses the source text one block at a time. An
> extension could easily add, remove, replace individual pieces of the
> parser without the problems we currently have. Additionally, it runs
> faster (by about 1 second for 1000 iterations) than the current code
> and uses less recursion. It also overcomes some of the parsing
> differances unique to the python implementation (i.e.: <p>s in lists
> match pl and php behavior) although an extension could easily provide
> the current behavior instead.
>

Well, I'm satisfied enough with what I have to release it into the
wild. I'm not convinced it's ready for prime time just yet, but I
think it's mostly there. I started this as an extension that replaced
the core parser rather than hacking on the existing code in
markdown.py. For one, it made timing against the old code easier, and
two, with none of the old code in front of me, it forced me to start
from scratch. Unfortunately, after working out the remaining edge
cases, it is barely any faster than the old code, but it is much more
customizable, so we do gain a lot IMO.

Please note that I did not even try to match the old core's behavior.
I built this to match pl and php's behavior when parsing blocks (which
means ticket 1 can finally be closed). I suspect a number of tests
will need to be reviewed and updated once I copy this over into
markdown.py.

The extension is attached as block.py. Copy it to your
markdown_extensions dir and run markdown with the 'block' extension
for testing.

Any and all feedback is welcome.

Unless I hear some strong objections, I'll start migrating this code
to markdown.py and update the tests and extensions in a branch.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[Yuri T. <qar...@gm...>]

> Of course, there is the question as to why the header isn't parsed as
> a header, just *not* as a child of the list item.

Yes, this is the one that confused me originally.  But it all makes
sense now.  I agree, wontfix.  Please append your explanation to the
bug when changing the status.

- yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[Waylan L. <wa...@gm...>]

On Mon, Nov 10, 2008 at 10:34 AM, John Szakmeister <jo...@sz...> wrote:
[snip]
>
> I understand.  Thanks for clearing that up.  It's also weird that the
> space before the hash gets inserted into the HTML, rather than
> stripped.  I'm not sure why that was happening.
>

Well, I didn't write that code, but I suppose it's because whitespace
at the beginning of a line can be significant, so the parser is
careful to not strip any whitespace. Of course, once we get to the
paragraph processing code, that is no longer an issue, so we could
strip is there. However, then we only want to lstrip, so that we don't
strip manual line breaks (two spaces at end of line). Seeing
whitespace is insignificant in html, I'm guessing the thinking was
it's easier to leave whitespace alone. But, yeah, I probably would
have cleaned up that whitespace myself.



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[John S. <jo...@sz...>]

On Mon, Nov 10, 2008 at 10:12 AM, Waylan Limberg <wa...@gm...> wrote:
> On Thu, Nov 6, 2008 at 5:28 AM, John Szakmeister <jo...@sz...> wrote:
>> did find that if I embed a header into a list like so:
>>  1. My list
>>
>>     ## blah, blah
>>
>
> Ah, yes, this one. You only have three spaces of indent.
> Python-Markdown requires four or its not a child of the list item. If
> you check Babelmark [1], about half the implementations require four
> spaces and the other half just require indentation of any kind. This
> has been a hotly debated topic on the markdown dicussion list in the
> past which I'd rather not see repeated here.

Heh.  I didn't realize that... but you're right, it's there in the syntax guide.

> Regardless of which side of that argument you may fall on, I've taken
> the position that the current code base creates a technical limitation
> which forces us to require four spaces. So, I'm likely to set the
> ticket Yuri created for this as "wontfix" unless someone else can come
> up with a solution I'm not aware of. I realize that sounds a little
> harsh, but I don't have a better answer at this time.

No offense taken.

> Of course, there is the question as to why the header isn't parsed as
> a header, just *not* as a child of the list item. Well, all
> implementations require the hash to be the *very first* character on
> the line or it's not a header. Again, see Babelmark [2]. Obviously,
> that distinction becomes blurry when only whitespace defines children
> of list items. It's easy to see how there can be multiple views on the
> correct behavior here.

I understand.  Thanks for clearing that up.  It's also weird that the
space before the hash gets inserted into the HTML, rather than
stripped.  I'm not sure why that was happening.

-John

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[Waylan L. <wa...@gm...>]

On Thu, Nov 6, 2008 at 5:28 AM, John Szakmeister <jo...@sz...> wrote:
> did find that if I embed a header into a list like so:
>  1. My list
>
>     ## blah, blah
>

Ah, yes, this one. You only have three spaces of indent.
Python-Markdown requires four or its not a child of the list item. If
you check Babelmark [1], about half the implementations require four
spaces and the other half just require indentation of any kind. This
has been a hotly debated topic on the markdown dicussion list in the
past which I'd rather not see repeated here.

Regardless of which side of that argument you may fall on, I've taken
the position that the current code base creates a technical limitation
which forces us to require four spaces. So, I'm likely to set the
ticket Yuri created for this as "wontfix" unless someone else can come
up with a solution I'm not aware of. I realize that sounds a little
harsh, but I don't have a better answer at this time.

Of course, there is the question as to why the header isn't parsed as
a header, just *not* as a child of the list item. Well, all
implementations require the hash to be the *very first* character on
the line or it's not a header. Again, see Babelmark [2]. Obviously,
that distinction becomes blurry when only whitespace defines children
of list items. It's easy to see how there can be multiple views on the
correct behavior here.

[1]: http://babelmark.bobtfish.net/?markdown=1.+My+list%0D%0A%0D%0A+++%23%23+blah,+blah
[2]: http://babelmark.bobtfish.net/?markdown=+%23%23+blah,+blah
-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Waylan L. <wa...@gm...>]

On Mon, Nov 10, 2008 at 4:37 AM, Yuri Takhteyev <qar...@gm...> wrote:
>>> The main markdown parser marked some methods with __'s... which is
>>> supposed to be the indicator that the method is private.  So it's a
>>> little bit disturbing that DefListParser is drilling into those
>>> internals.  Perhaps it's indicative that those methods should be
>>> exposed more, or that we need an extra hook point into the code?
>>
>> That's a valid point and one I've thought about as well.
>> Unfortunately, I don't have a good answer yet. Part of my motivation
>> in doing this extension was to raise these questions and see how
>> things go.
>
> What worries me more, actually, is the fact that the extension ends up
> duplicating much of the code in markdown.py  and then substitutes the
> parser.  Substituting the parser should be a no-no for extensions.
> (Note that if several extensions do this, then one will break
> another.)

I am very aware of this problem and don't particularly like it myself.
However, the nature of the definition list syntax makes it much easier
to do within the core than elsewhere. Currently, the only way to add a
new block type to the core is to override the parseChunk method and
add a call to your code within the various if statements in that
method. And then the paragraph code is not really generic enough to
the point that many other block types deffer to it to do the right
thing.

> Can you explain what changes you are trying to make to the parser?
> Perhaps we can find a way to make the parser a little more flexible,
> so that def_list extension will just need to add a hander to it,
> rather than having to swap in a new parser.

Actually, over the last few days I've put together a completely new
core processor which works very differently. Like the other plugable
parts of markdown, it loops through a list (OrderedDict actually) of
BlockProcessors and parses the source text one block at a time. An
extension could easily add, remove, replace individual pieces of the
parser without the problems we currently have. Additionally, it runs
faster (by about 1 second for 1000 iterations) than the current code
and uses less recursion. It also overcomes some of the parsing
differances unique to the python implementation (i.e.: <p>s in lists
match pl and php behavior) although an extension could easily provide
the current behavior instead.

It's not quite ready yet, but in the next few days I'll post something
here for others to review.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Yuri T. <qar...@gm...>]

>> The main markdown parser marked some methods with __'s... which is
>> supposed to be the indicator that the method is private.  So it's a
>> little bit disturbing that DefListParser is drilling into those
>> internals.  Perhaps it's indicative that those methods should be
>> exposed more, or that we need an extra hook point into the code?
>
> That's a valid point and one I've thought about as well.
> Unfortunately, I don't have a good answer yet. Part of my motivation
> in doing this extension was to raise these questions and see how
> things go.

What worries me more, actually, is the fact that the extension ends up
duplicating much of the code in markdown.py  and then substitutes the
parser.  Substituting the parser should be a no-no for extensions.
(Note that if several extensions do this, then one will break
another.)

Can you explain what changes you are trying to make to the parser?
Perhaps we can find a way to make the parser a little more flexible,
so that def_list extension will just need to add a hander to it,
rather than having to swap in a new parser.

- yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[Yuri T. <qar...@gm...>]

Hi, John,

Since those tests fail at the moment, I won't integrate them yet, but
will treat them as bug reports.  I created a ticket for each of those
issues.  (http://www.freewisdom.org/projects/python-markdown/Tickets).
 The test suite is meant to be used for regression testing, so I would
rather not add failing test to it.

  - yuri

On Thu, Nov 6, 2008 at 2:28 AM, John Szakmeister <jo...@sz...> wrote:
> I was trying to ensure that a couple of code paths were actually being
> reached.  While in the process of doing that, I created a couple of
> tests that seem like they should work, but currently fail.

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Ben W. <da...@gm...>]

I'll have to check what I did with this last. I vaguely remember
running into a 2.3 problem, and thought I had it solved. :-) Can you
tell I've been out of the loop again?

However, I would say that my primary goal was to offer an elegant way
to add new markups. I care less about Treap being the implementation
than an elegant, pythonic solution. :) So, no bruised ego here. At
least the conversation has started.

That said, let me check when I get home on what I did on this. I
remember there being a problem, but not if it's the same as
encountered with 2.3 incompatibility. Entirely possible that I made a
personal decision not to care. I've not been able to visit the issue
in quite some time. However, OrderedDict looks very familiar.

I should say (I think I did once before) that my original inspiration
was from PmWiki, which is PHP-based. The author of that tool created a
wiki framework where everything could be overridden. The markup was
very flexible. Having grown accustomed to that flexibility, I'd wanted
to see it here.

Regards,
Ben

On Tue, Oct 28, 2008 at 3:13 AM, Yuri Takhteyev <qar...@gm...> wrote:
> I haven't had a chance to play with this, but in principle this looks
> reasonable.  I suggest that you merge this into mainline and let's
> assume that this is our solution for now.  I'll try to think a little
> more about it next week.
>
> No offense to Ben - his library looked great.  But it probably does
> makes more sense to go with a more standard solution.  Ben: if you
> think we are making a mistake here, please speak up. :)
>
>  - yuri
>
> On Wed, Oct 22, 2008 at 9:01 PM, Waylan Limberg <wa...@gm...> wrote:
>> I just pushed a new branch "dj-odict" [1] that implements a working
>> OrderedDict. You should see the last 2 commits in that branch. The
>> first of the 2 was an incomplete (it works but there is more work to
>> do) implementation of voidspace's OrderedDict with a few additional
>> methods. However, as that implementation included a lot of features
>> we'll never need (like comparison operators), I switched to the much
>> simpler Django implementation and improved the added methods. I've
>> reimplemented the ``add`` and ``link`` methods and they work with  the
>> ``_begin``, ``_end``, ``<key``, and ``>key`` syntax. While they are
>> not necessary, it does make the code in the extension shorter and to
>> the point. The tests in regression_tests.py cover the features we need
>> pretty well. I still need to write documentation, but that can wait
>> until I know we're going to go with this.
>>
>> I didn't do any real performance/memory usage testing, but the
>> regression_tests consistently run a little faster with more tests so
>> I'm thinking we have a minimal speed increase with this.
>>
>> Personally, I like it.
>>
>> As an aside, note the change in the tables.py extension in that last
>> commit. I had to do that to get the test to pass. Thing is, is was
>> passing with Treap and it shouldn't have been because Treap was
>> getting the order wrong (I think - I haven't actually checked this). I
>> know it was getting the order wrong on footnotes for sure, but that
>> ext doesn't have tests so it didn't show up. There are still some
>> other, unrelated problems with footnotes though, so they aren't
>> working right now (I think I broke them when I refactored the
>> processors/patterns this past weekend). But there wasn't much point in
>> working on that when things weren't in the right order so I did this
>> first.
>>
>> [1]: http://gitorious.org/projects/python-markdown/repos/mainline/logs/dj-odict
>>
>> On Wed, Oct 22, 2008 at 12:53 PM, Yuri Takhteyev <qar...@gm...> wrote:
>>>> Apparently Treap wasn't tested in Python 2.3 as the builtin function
>>>> ``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
>>>> not available until 2.4. We get errors on import. Can't even try
>>>> running the code.
>>>
>>> I did test it with 2.3, discovered that it wasn't working, but thought
>>> that this would be fixable, so this has been on my todo list.
>>>
>>>> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>>>
>>> We can definitely think about this.  I'll have to read up on odict to
>>> see if it does everything that we need.  Of top of my head, though,
>>> odict 1400 lines, too big to just include in our code, so we would
>>> have to worry about an extra dependency.  But let's think about this.
>>> Perhaps we should look at more options too.
>>>
>>>  - yuri
>>>
>>> --
>>> http://sputnik.freewisdom.org/
>>>
>>
>>
>>
>> --
>> ----
>> Waylan Limberg
>> wa...@gm...
>>
>
>
>
> --
> http://sputnik.freewisdom.org/
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
Ben Wilson
"Words are the only thing which will last forever" Churchill

Re: [Python-markdown-discuss] A couple more tests for some edge cases

[John S. <jo...@sz...>]

On Thu, Nov 6, 2008 at 5:28 AM, John Szakmeister <jo...@sz...> wrote:
> I was trying to ensure that a couple of code paths were actually being
> reached.  While in the process of doing that, I created a couple of
> tests that seem like they should work, but currently fail.
>
> para-with-hr-test.patch adds a really simple test case:
>   text, text, text
>   ***
>   more text
>
> The output that's produced is completely missing the first line.
>
> header-test.patch is one that I came up with while trying to make
> __processParagraph() hit the other conditional (one where it
> specifically looks for the start of a header).  Interestingly enough,
> I was never able to get something to run down that path.  However, I
> did find that if I embed a header into a list like so:
>  1. My list
>
>     ## blah, blah
>
> Then I don't get a header element in the list.

Whups!  There was more in the header-test patch than should have been.
 Try the attached instead.

-John

[Python-markdown-discuss] A couple more tests for some edge cases

[John S. <jo...@sz...>]

I was trying to ensure that a couple of code paths were actually being
reached.  While in the process of doing that, I created a couple of
tests that seem like they should work, but currently fail.

para-with-hr-test.patch adds a really simple test case:
   text, text, text
   ***
   more text

The output that's produced is completely missing the first line.

header-test.patch is one that I came up with while trying to make
__processParagraph() hit the other conditional (one where it
specifically looks for the start of a header).  Interestingly enough,
I was never able to get something to run down that path.  However, I
did find that if I embed a header into a list like so:
  1. My list

     ## blah, blah

Then I don't get a header element in the list.

-John

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Waylan L. <wa...@gm...>]

On Tue, Nov 4, 2008 at 10:56 PM, Waylan Limberg <wa...@gm...> wrote:
> ps: I believe the only other things we're missing from extra are
> abbreviations and markdown inside html. If I'm wrong, let me know and
> I'll add it to my todo list. I have an old abbreviations extension
> which needs to be refactored to run with the many recent changes to
> markdown. I haven't even looked at markdown inside html yet.
>

I just pushed the Abbreviation Extension. Note that this is a simple
extension that does nothing more than PHP Markdown Extra. If you want
the extra features of my old extension (including the one-of-a-kind
inline abbrs defs developed by Seemant Kulleen or loading abbrs from
external files) then you can always write your own. Feel free to use
my old one as a base.

What's really interesting to me, is that with both these recent
extensions, I had to do minor patches to markdown.py itself (which
were committed separately). However, in each case, the patch should
have happened regardless of the specific extension. I'll call it
testing the extension API.


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Waylan L. <wa...@gm...>]

On Wed, Nov 5, 2008 at 8:45 AM, Waylan Limberg <wa...@gm...> wrote:
>>
>> It looks like in _MarkdownParser__processParagraph() that
>> self.__processHR() and self.__processHeader() need to be
>> self._MarkdownParser__processHR() and
>> self._MarkdownParser__processHeader().  By default, python will try
>> calling self._DefListParser__processHR() and
>> self._DefListParser__processHeader(), which don't exist.
>
> Ah, right. It occurred to after I went to bed last night that I never
> tested to make sure all other markdown syntax was still parsd
> correctly. I'll get on that next.
>

I just pushed the fix for this and added a test. Thanks John.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Waylan L. <wa...@gm...>]

On Wed, Nov 5, 2008 at 3:14 AM, John Szakmeister <jo...@sz...> wrote:
>
> The main markdown parser marked some methods with __'s... which is
> supposed to be the indicator that the method is private.  So it's a
> little bit disturbing that DefListParser is drilling into those
> internals.  Perhaps it's indicative that those methods should be
> exposed more, or that we need an extra hook point into the code?

That's a valid point and one I've thought about as well.
Unfortunately, I don't have a good answer yet. Part of my motivation
in doing this extension was to raise these questions and see how
things go.

>
> It looks like in _MarkdownParser__processParagraph() that
> self.__processHR() and self.__processHeader() need to be
> self._MarkdownParser__processHR() and
> self._MarkdownParser__processHeader().  By default, python will try
> calling self._DefListParser__processHR() and
> self._DefListParser__processHeader(), which don't exist.

Ah, right. It occurred to after I went to bed last night that I never
tested to make sure all other markdown syntax was still parsd
correctly. I'll get on that next.

> It looks
> like the only difference in your process paragraph is that it also
> watches for 'dd'.  Is there some way to either expose that, or just
> add this edge to the actual MarkdownParser?  I realize that markdown
> doesn't natively produce those tags.  But it seems like a shame to
> have to edit the same code in two places if you find a bug (if you
> remember the other one exists!).

I had similar thoughts, but that was the easiest way to get things
working. At the same time, I don't look forward to maintaining this in
parallel with the markdown core. Certainly valid concerns.


>
> I think it'd be nice for the table extension to function like PHP
> markdown's implementation.  But that's just a nicety. :-)
>
Oh yeah, I forgot about that one.

Thanks for the comments John. You've given me some things to think
about and reminded me of a few others.



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[John S. <jo...@sz...>]

On Tue, Nov 4, 2008 at 8:38 AM, Waylan Limberg <wa...@gm...> wrote:
> On Tue, Nov 4, 2008 at 5:08 AM, Yuri Takhteyev <qar...@gm...> wrote:
>> Sorry about not getting back to you about this earlier, it's been a
>> crazy few days.
>>
>> Yes, test cases are good.  I will integrate yours the next time I boot
>> my brain into python markdown.  Probably later this week.
>>
>
> Sorry, I guess I forgot to mention it, but I already commited that
> test. You can find it here:
>
> http://gitorious.org/projects/python-markdown/repos/mainline/commits/c9389e8b33b5b6771f541c1ba47bb91abd9b3f6f
>

Thanks Waylan!  I'll make sure to try add future tests to the correct
area as well.

-John

Re: [Python-markdown-discuss] Announcing Definition List Extension - call for testing

[John S. <jo...@sz...>]

On Tue, Nov 4, 2008 at 10:56 PM, Waylan Limberg <wa...@gm...> wrote:
> I'm happy to report that I just pushed a Definition List extension [1]
> to the repo. I know this has been one of the most requested
> extensions. After Yuri's refactor of the core into a separate
> MarkdownParser, it was much easier to do.

Woot!  I've been eager to have this as well.

> I haven't written docs for it yet, but I have included tests [2],
> which show how it works. See PHP Markdown Extra's docs [3] for the
> hows and whys. Other than the included tests, I haven't really tested
> it much, so please run it through the paces. I'd like to say we have
> everything PHP Markdown Extra has when we release 2.0 and this was the
> biggest hurdle. Now we just need to wrap up the edge cases I'm sure I
> missed.

That would be awesome.

> Any bug reports, patches, test cases, comments, and criticisms are welcome.
>
> [1]: http://gitorious.org/projects/python-markdown/repos/mainline/blobs/master/markdown_extensions%2Fdef_list.py
> [2]: http://gitorious.org/projects/python-markdown/repos/mainline/trees/master/tests/extensions-x-def_list
> [3]: http://michelf.com/projects/php-markdown/extra/#def-list

I haven't tried testing it yet, but I'll definitely be trying it out
soon.  But I do have a couple other comments.

The main markdown parser marked some methods with __'s... which is
supposed to be the indicator that the method is private.  So it's a
little bit disturbing that DefListParser is drilling into those
internals.  Perhaps it's indicative that those methods should be
exposed more, or that we need an extra hook point into the code?

It looks like in _MarkdownParser__processParagraph() that
self.__processHR() and self.__processHeader() need to be
self._MarkdownParser__processHR() and
self._MarkdownParser__processHeader().  By default, python will try
calling self._DefListParser__processHR() and
self._DefListParser__processHeader(), which don't exist.  It looks
like the only difference in your process paragraph is that it also
watches for 'dd'.  Is there some way to either expose that, or just
add this edge to the actual MarkdownParser?  I realize that markdown
doesn't natively produce those tags.  But it seems like a shame to
have to edit the same code in two places if you find a bug (if you
remember the other one exists!).

> ps: I believe the only other things we're missing from extra are
> abbreviations and markdown inside html. If I'm wrong, let me know and
> I'll add it to my todo list. I have an old abbreviations extension
> which needs to be refactored to run with the many recent changes to
> markdown. I haven't even looked at markdown inside html yet.

I think it'd be nice for the table extension to function like PHP
markdown's implementation.  But that's just a nicety. :-)

-John

[Python-markdown-discuss] Announcing Definition List Extension - call for testing

[Waylan L. <wa...@gm...>]

I'm happy to report that I just pushed a Definition List extension [1]
to the repo. I know this has been one of the most requested
extensions. After Yuri's refactor of the core into a separate
MarkdownParser, it was much easier to do.

I haven't written docs for it yet, but I have included tests [2],
which show how it works. See PHP Markdown Extra's docs [3] for the
hows and whys. Other than the included tests, I haven't really tested
it much, so please run it through the paces. I'd like to say we have
everything PHP Markdown Extra has when we release 2.0 and this was the
biggest hurdle. Now we just need to wrap up the edge cases I'm sure I
missed.

Any bug reports, patches, test cases, comments, and criticisms are welcome.

[1]: http://gitorious.org/projects/python-markdown/repos/mainline/blobs/master/markdown_extensions%2Fdef_list.py
[2]: http://gitorious.org/projects/python-markdown/repos/mainline/trees/master/tests/extensions-x-def_list
[3]: http://michelf.com/projects/php-markdown/extra/#def-list

ps: I believe the only other things we're missing from extra are
abbreviations and markdown inside html. If I'm wrong, let me know and
I'll add it to my todo list. I have an old abbreviations extension
which needs to be refactored to run with the many recent changes to
markdown. I haven't even looked at markdown inside html yet.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Waylan L. <wa...@gm...>]

On Tue, Nov 4, 2008 at 5:08 AM, Yuri Takhteyev <qar...@gm...> wrote:
> Sorry about not getting back to you about this earlier, it's been a
> crazy few days.
>
> Yes, test cases are good.  I will integrate yours the next time I boot
> my brain into python markdown.  Probably later this week.
>

Sorry, I guess I forgot to mention it, but I already commited that
test. You can find it here:

http://gitorious.org/projects/python-markdown/repos/mainline/commits/c9389e8b33b5b6771f541c1ba47bb91abd9b3f6f


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[John S. <jo...@sz...>]

On Tue, Nov 4, 2008 at 5:08 AM, Yuri Takhteyev <qar...@gm...> wrote:
> Sorry about not getting back to you about this earlier, it's been a
> crazy few days.

Not a problem.  More than anything I was really looking to see if it's
useful. :-)

> Yes, test cases are good.  I will integrate yours the next time I boot
> my brain into python markdown.  Probably later this week.

Thanks!

-John

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Yuri T. <qar...@gm...>]

Sorry about not getting back to you about this earlier, it's been a
crazy few days.

Yes, test cases are good.  I will integrate yours the next time I boot
my brain into python markdown.  Probably later this week.

  - yuri

On Tue, Nov 4, 2008 at 1:29 AM, John Szakmeister <jo...@sz...> wrote:
> On Wed, Oct 29, 2008 at 4:41 AM, John Szakmeister <jo...@sz...> wrote:
>> On Tue, Oct 28, 2008 at 11:04 PM, Waylan Limberg <wa...@gm...> wrote:
>>> FYI, I pushed the merge earlier this evening.
>>
>> I actually had a test lying around in my working copy testing *...*
>> around a link (the old markdown code didn't work and neither did the
>> treap implementation).  The new baseline actually worked though!
>>
>> I've attached the test case.
>
> BTW, is this sort of thing of any use to the project?  I've tripped
> over several other edges that don't work, but I won't waste anyone's
> time (mine included) if it's not useful.
>
> -John
>



-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[John S. <jo...@sz...>]

On Wed, Oct 29, 2008 at 4:41 AM, John Szakmeister <jo...@sz...> wrote:
> On Tue, Oct 28, 2008 at 11:04 PM, Waylan Limberg <wa...@gm...> wrote:
>> FYI, I pushed the merge earlier this evening.
>
> I actually had a test lying around in my working copy testing *...*
> around a link (the old markdown code didn't work and neither did the
> treap implementation).  The new baseline actually worked though!
>
> I've attached the test case.

BTW, is this sort of thing of any use to the project?  I've tripped
over several other edges that don't work, but I won't waste anyone's
time (mine included) if it's not useful.

-John

Re: [Python-markdown-discuss] Footnote ext and random links

[Waylan L. <wa...@gm...>]

I just pushed my changes to footnotes.py. It's a significant refactor
under the hood. I threw out the old hash to keep the order of
footnotes and used OrderedDict. It worked quite well. I was about to
have the setFootnote method make sure the ID was always unique (by
appending a num), but then it wouldn't match the marker anymore. Not
sure how that would work.

Anyway, the output exactly matches PHP Markdown Extra's output now -
except we include a title attribute in the backlink. This addresses
various little oddities with the output we were getting before. And,
we have working tests now.

On Wed, Oct 29, 2008 at 2:49 PM, Yuri Takhteyev <qar...@gm...> wrote:
>> at the output. There are a few little oddities I'll fix, but the real
>> problem (IMO) is the random strings in the generated links from marker
>> to note and back. As they are randomly generated, they are never the
>> same. This means that if a user was to link to a specific footnote
>> (i.e.: ``http://example.com/page.html#fn1-xxxxxxxxx``), the link
>
> I did this to make it possible to display multiple markdown documents
> on the same HTML page.  E.g., several entries on the front page of the
> blog.  But I agree, this is an ugly solution.  You can get rid of the
> random suffix, and meanwhile I will think about a better way to deal
> with multiple documents on a page and similar issues.
>
> - yuri
>
> --
> http://sputnik.freewisdom.org/
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Footnote ext and random links

[Yuri T. <qar...@gm...>]

> at the output. There are a few little oddities I'll fix, but the real
> problem (IMO) is the random strings in the generated links from marker
> to note and back. As they are randomly generated, they are never the
> same. This means that if a user was to link to a specific footnote
> (i.e.: ``http://example.com/page.html#fn1-xxxxxxxxx``), the link

I did this to make it possible to display multiple markdown documents
on the same HTML page.  E.g., several entries on the front page of the
blog.  But I agree, this is an ugly solution.  You can get rid of the
random suffix, and meanwhile I will think about a better way to deal
with multiple documents on a page and similar issues.

- yuri

-- 
http://sputnik.freewisdom.org/