python-markdown-discuss — A general purpose discussion list

[Python-markdown-discuss] Footnote ext and random links

[Waylan L. <wa...@gm...>]

Yuri,

I got the footnote extension working again last night and was looking
at the output. There are a few little oddities I'll fix, but the real
problem (IMO) is the random strings in the generated links from marker
to note and back. As they are randomly generated, they are never the
same. This means that if a user was to link to a specific footnote
(i.e.: ``http://example.com/page.html#fn1-xxxxxxxxx``), the link
wouldn't work for future visits. Well, the correct page would load,
but the browser wouldn't jump to the proper footnote - which could be
a useful feature. I also noted that other implementations do not do
this. They simply use the key (i.e. ``fn1``) which should be unique
anyway. On top of all that, as the randomly generated links are
different each time, it is imposable to test with our current testing
framework (tests exist, but they're turned off) - which is why no one
noticed we broke footnotes when we implemented Treap.

So, my question is: what purpose do the random strings serve? Why
aren't we just using the key for each footnote?

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[John S. <jo...@sz...>]

On Tue, Oct 28, 2008 at 11:04 PM, Waylan Limberg <wa...@gm...> wrote:
> FYI, I pushed the merge earlier this evening.

I actually had a test lying around in my working copy testing *...*
around a link (the old markdown code didn't work and neither did the
treap implementation).  The new baseline actually worked though!

I've attached the test case.

-John

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Waylan L. <wa...@gm...>]

FYI, I pushed the merge earlier this evening.

On Tue, Oct 28, 2008 at 4:13 AM, Yuri Takhteyev <qar...@gm...> wrote:
> I haven't had a chance to play with this, but in principle this looks
> reasonable.  I suggest that you merge this into mainline and let's
> assume that this is our solution for now.  I'll try to think a little
> more about it next week.
>
> No offense to Ben - his library looked great.  But it probably does
> makes more sense to go with a more standard solution.  Ben: if you
> think we are making a mistake here, please speak up. :)
>
>  - yuri
>
> On Wed, Oct 22, 2008 at 9:01 PM, Waylan Limberg <wa...@gm...> wrote:
>> I just pushed a new branch "dj-odict" [1] that implements a working
>> OrderedDict. You should see the last 2 commits in that branch. The
>> first of the 2 was an incomplete (it works but there is more work to
>> do) implementation of voidspace's OrderedDict with a few additional
>> methods. However, as that implementation included a lot of features
>> we'll never need (like comparison operators), I switched to the much
>> simpler Django implementation and improved the added methods. I've
>> reimplemented the ``add`` and ``link`` methods and they work with  the
>> ``_begin``, ``_end``, ``<key``, and ``>key`` syntax. While they are
>> not necessary, it does make the code in the extension shorter and to
>> the point. The tests in regression_tests.py cover the features we need
>> pretty well. I still need to write documentation, but that can wait
>> until I know we're going to go with this.
>>
>> I didn't do any real performance/memory usage testing, but the
>> regression_tests consistently run a little faster with more tests so
>> I'm thinking we have a minimal speed increase with this.
>>
>> Personally, I like it.
>>
>> As an aside, note the change in the tables.py extension in that last
>> commit. I had to do that to get the test to pass. Thing is, is was
>> passing with Treap and it shouldn't have been because Treap was
>> getting the order wrong (I think - I haven't actually checked this). I
>> know it was getting the order wrong on footnotes for sure, but that
>> ext doesn't have tests so it didn't show up. There are still some
>> other, unrelated problems with footnotes though, so they aren't
>> working right now (I think I broke them when I refactored the
>> processors/patterns this past weekend). But there wasn't much point in
>> working on that when things weren't in the right order so I did this
>> first.
>>
>> [1]: http://gitorious.org/projects/python-markdown/repos/mainline/logs/dj-odict
>>
>> On Wed, Oct 22, 2008 at 12:53 PM, Yuri Takhteyev <qar...@gm...> wrote:
>>>> Apparently Treap wasn't tested in Python 2.3 as the builtin function
>>>> ``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
>>>> not available until 2.4. We get errors on import. Can't even try
>>>> running the code.
>>>
>>> I did test it with 2.3, discovered that it wasn't working, but thought
>>> that this would be fixable, so this has been on my todo list.
>>>
>>>> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>>>
>>> We can definitely think about this.  I'll have to read up on odict to
>>> see if it does everything that we need.  Of top of my head, though,
>>> odict 1400 lines, too big to just include in our code, so we would
>>> have to worry about an extra dependency.  But let's think about this.
>>> Perhaps we should look at more options too.
>>>
>>>  - yuri
>>>
>>> --
>>> http://sputnik.freewisdom.org/
>>>
>>
>>
>>
>> --
>> ----
>> Waylan Limberg
>> wa...@gm...
>>
>
>
>
> --
> http://sputnik.freewisdom.org/
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Link after two spaces

[Waylan L. <wa...@gm...>]

On Sat, Oct 25, 2008 at 7:47 AM, Torsten Bronger
<br...@ph...> wrote:
> Hallöchen!
>
> Two spaces, followed by a link, are converted to a line break, which
> is not the behaviour of the original implementation.
>

Torsten, I can't seem to reproduce this. Could you provide an example?
Are you using the latest code in the Git repo?




-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] New TOC extension.

[Waylan L. <wa...@gm...>]

On Tue, Oct 21, 2008 at 6:44 PM, Jack Miller <ja...@co...> wrote:
[snip]
>
> Attached is a patch that handles both of these issues. I didn't use HeaderID
[snip]

I just pushed this. Thanks Jack.


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Yuri T. <qar...@gm...>]

I haven't had a chance to play with this, but in principle this looks
reasonable.  I suggest that you merge this into mainline and let's
assume that this is our solution for now.  I'll try to think a little
more about it next week.

No offense to Ben - his library looked great.  But it probably does
makes more sense to go with a more standard solution.  Ben: if you
think we are making a mistake here, please speak up. :)

  - yuri

On Wed, Oct 22, 2008 at 9:01 PM, Waylan Limberg <wa...@gm...> wrote:
> I just pushed a new branch "dj-odict" [1] that implements a working
> OrderedDict. You should see the last 2 commits in that branch. The
> first of the 2 was an incomplete (it works but there is more work to
> do) implementation of voidspace's OrderedDict with a few additional
> methods. However, as that implementation included a lot of features
> we'll never need (like comparison operators), I switched to the much
> simpler Django implementation and improved the added methods. I've
> reimplemented the ``add`` and ``link`` methods and they work with  the
> ``_begin``, ``_end``, ``<key``, and ``>key`` syntax. While they are
> not necessary, it does make the code in the extension shorter and to
> the point. The tests in regression_tests.py cover the features we need
> pretty well. I still need to write documentation, but that can wait
> until I know we're going to go with this.
>
> I didn't do any real performance/memory usage testing, but the
> regression_tests consistently run a little faster with more tests so
> I'm thinking we have a minimal speed increase with this.
>
> Personally, I like it.
>
> As an aside, note the change in the tables.py extension in that last
> commit. I had to do that to get the test to pass. Thing is, is was
> passing with Treap and it shouldn't have been because Treap was
> getting the order wrong (I think - I haven't actually checked this). I
> know it was getting the order wrong on footnotes for sure, but that
> ext doesn't have tests so it didn't show up. There are still some
> other, unrelated problems with footnotes though, so they aren't
> working right now (I think I broke them when I refactored the
> processors/patterns this past weekend). But there wasn't much point in
> working on that when things weren't in the right order so I did this
> first.
>
> [1]: http://gitorious.org/projects/python-markdown/repos/mainline/logs/dj-odict
>
> On Wed, Oct 22, 2008 at 12:53 PM, Yuri Takhteyev <qar...@gm...> wrote:
>>> Apparently Treap wasn't tested in Python 2.3 as the builtin function
>>> ``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
>>> not available until 2.4. We get errors on import. Can't even try
>>> running the code.
>>
>> I did test it with 2.3, discovered that it wasn't working, but thought
>> that this would be fixable, so this has been on my todo list.
>>
>>> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>>
>> We can definitely think about this.  I'll have to read up on odict to
>> see if it does everything that we need.  Of top of my head, though,
>> odict 1400 lines, too big to just include in our code, so we would
>> have to worry about an extra dependency.  But let's think about this.
>> Perhaps we should look at more options too.
>>
>>  - yuri
>>
>> --
>> http://sputnik.freewisdom.org/
>>
>
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>



-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] Link after two spaces

[Waylan L. <wa...@gm...>]

Thanks for the report Torsten. I've filed this as a ticket [1] so we
don't forget about it. I have an idea what may be causing this, but
I'm not sure yet.

[1]: http://www.freewisdom.org/projects/python-markdown/Tickets/000020

On Sat, Oct 25, 2008 at 7:47 AM, Torsten Bronger
<br...@ph...> wrote:
> Hallöchen!
>
> Two spaces, followed by a link, are converted to a line break, which
> is not the behavior of the original implementation.
>
> Tschö,
> Torsten.
>
> --
> Torsten Bronger, aquisgrana, europa vetus
>                   Jabber ID: tor...@ja...
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] Link after two spaces

[Torsten B. <br...@ph...>]

Hallöchen!

Two spaces, followed by a link, are converted to a line break, which
is not the behaviour of the original implementation.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Waylan L. <wa...@gm...>]

I just pushed a new branch "dj-odict" [1] that implements a working
OrderedDict. You should see the last 2 commits in that branch. The
first of the 2 was an incomplete (it works but there is more work to
do) implementation of voidspace's OrderedDict with a few additional
methods. However, as that implementation included a lot of features
we'll never need (like comparison operators), I switched to the much
simpler Django implementation and improved the added methods. I've
reimplemented the ``add`` and ``link`` methods and they work with  the
``_begin``, ``_end``, ``<key``, and ``>key`` syntax. While they are
not necessary, it does make the code in the extension shorter and to
the point. The tests in regression_tests.py cover the features we need
pretty well. I still need to write documentation, but that can wait
until I know we're going to go with this.

I didn't do any real performance/memory usage testing, but the
regression_tests consistently run a little faster with more tests so
I'm thinking we have a minimal speed increase with this.

Personally, I like it.

As an aside, note the change in the tables.py extension in that last
commit. I had to do that to get the test to pass. Thing is, is was
passing with Treap and it shouldn't have been because Treap was
getting the order wrong (I think - I haven't actually checked this). I
know it was getting the order wrong on footnotes for sure, but that
ext doesn't have tests so it didn't show up. There are still some
other, unrelated problems with footnotes though, so they aren't
working right now (I think I broke them when I refactored the
processors/patterns this past weekend). But there wasn't much point in
working on that when things weren't in the right order so I did this
first.

[1]: http://gitorious.org/projects/python-markdown/repos/mainline/logs/dj-odict

On Wed, Oct 22, 2008 at 12:53 PM, Yuri Takhteyev <qar...@gm...> wrote:
>> Apparently Treap wasn't tested in Python 2.3 as the builtin function
>> ``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
>> not available until 2.4. We get errors on import. Can't even try
>> running the code.
>
> I did test it with 2.3, discovered that it wasn't working, but thought
> that this would be fixable, so this has been on my todo list.
>
>> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>
> We can definitely think about this.  I'll have to read up on odict to
> see if it does everything that we need.  Of top of my head, though,
> odict 1400 lines, too big to just include in our code, so we would
> have to worry about an extra dependency.  But let's think about this.
> Perhaps we should look at more options too.
>
>  - yuri
>
> --
> http://sputnik.freewisdom.org/
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Waylan L. <wa...@gm...>]

On Wed, Oct 22, 2008 at 12:53 PM, Yuri Takhteyev <qar...@gm...> wrote:
>> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>
> We can definitely think about this.  I'll have to read up on odict to
> see if it does everything that we need.  Of top of my head, though,
> odict 1400 lines, too big to just include in our code, so we would
> have to worry about an extra dependency.  But let's think about this.
> Perhaps we should look at more options too.
>
Actually that file has more than one implementation. We'd only need
the one  which we could include and modify within markdown.py. Or we
could do the same with Django's, or any other BSD licensed
implementation.

On Wed, Oct 22, 2008 at 1:22 PM,  <ne...@gm...> wrote:
> We can extend it to do insert not only by index but by keynames too.

That's exactly what I had in mind, regardless of which implementation
we start with.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[<ne...@gm...>]

2008/10/22 <ne...@gm...>

>
>
> 2008/10/22 Yuri Takhteyev <qar...@gm...>
>
>>
>> > Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>>
>> We can definitely think about this.  I'll have to read up on odict to
>> see if it does everything that we need.  Of top of my head, though,
>> odict 1400 lines, too big to just include in our code, so we would
>> have to worry about an extra dependency.  But let's think about this.
>> Perhaps we should look at more options too.
>>
>
> I think another one extra dependency is not very good. As an alternative we
> can look at SortedDict in Django [1]. We can extend it to do insert not only
> by index but by keynames too.
>
> [1]:
> http://code.djangoproject.com/browser/django/trunk/django/utils/datastructures.py#L53
>

Just noticed that in PEP 372, that Waylan referred,  SortedDict listed as
one of the implementations, and there are few others, I think we can find
something suitable here [1]

[1]: http://www.python.org/dev/peps/pep-0372/#example-implementation

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[<ne...@gm...>]

2008/10/22 Yuri Takhteyev <qar...@gm...>

>
> > Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
>
> We can definitely think about this.  I'll have to read up on odict to
> see if it does everything that we need.  Of top of my head, though,
> odict 1400 lines, too big to just include in our code, so we would
> have to worry about an extra dependency.  But let's think about this.
> Perhaps we should look at more options too.
>

I think another one extra dependency is not very good. As an alternative we
can look at SortedDict in Django [1]. We can extend it to do insert not only
by index but by keynames too.

[1]:
http://code.djangoproject.com/browser/django/trunk/django/utils/datastructures.py#L53

Re: [Python-markdown-discuss] Treap does not work in Python2.3

[Yuri T. <qar...@gm...>]

> Apparently Treap wasn't tested in Python 2.3 as the builtin function
> ``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
> not available until 2.4. We get errors on import. Can't even try
> running the code.

I did test it with 2.3, discovered that it wasn't working, but thought
that this would be fixable, so this has been on my todo list.

> Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]

We can definitely think about this.  I'll have to read up on odict to
see if it does everything that we need.  Of top of my head, though,
odict 1400 lines, too big to just include in our code, so we would
have to worry about an extra dependency.  But let's think about this.
Perhaps we should look at more options too.

  - yuri

-- 
http://sputnik.freewisdom.org/

[Python-markdown-discuss] Treap does not work in Python2.3

[Waylan L. <wa...@gm...>]

Apparently Treap wasn't tested in Python 2.3 as the builtin function
``sorted`` (as well as ``operator.itemgetter`` which sorted needs) are
not available until 2.4. We get errors on import. Can't even try
running the code.

Yuri, are we now dropping support for 2.3 or was this an oversite?

Personally, I'd prefer dropping Treap for an OrderedDict. odict [1]
seems like a reasonable solution and it's licensed under BSD so
there's no reason we couldn't include it. Interestingly, PEP 372 [2]
was recently introduced proposing an order dict for inclusion in the
standard library. That PEP points to a number of other
implementations, including one written by the pep author as well as a
much faster C clone of odict.

[1]: http://www.voidspace.org.uk/python/odict.html
[2]: http://www.python.org/dev/peps/pep-0372/

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] New TOC extension.

[Jack M. <ja...@co...>]

> Well, mostly. Now, if a header already has an id, both the header and
> anchor will get the id. As ids should be unique, this creates invalid
> html. I'd suggest *not* assigning an id to the anchor - just the
> header.
> 
> And speaking of unique, how are you guaranteeing that slugify does not
> create the same id for two different headers? Not a showstopper, but
> something to think about. You might be interested in how HeaderID
> addresses this.

Attached is a patch that handles both of these issues. I didn't use HeaderID
code, but I matched the style (_1, _2, etc) and it should produce globally
unique ids.

> I don't mean to be so nitpicky Jack, but I would like something like
> this packaged up with Markdown and it has to be right before that
> happens. When I'm happy with it and with your blessings (and no
> objection from Yuri) I'll commit it to the repo.
>

Hahaha. You're not being unreasonable, it's not a big deal. Trust me, I work
with much more nitpicky people at work =)

--Jack

Re: [Python-markdown-discuss] Bug in Treap??

[Waylan L. <wa...@gm...>]

On Mon, Oct 20, 2008 at 9:53 PM, Waylan Limberg <wa...@gm...> wrote:
[snip]
>
> If I understand things correctly, line 1667 is the problem. It currently is:
>
>    self._tree[parn]['heap'][1]+brace
>
> and should be:
>
>    self._tree[parn]['heap'][1].strip('<>')+brace
>
> The problem is, that takes "second" out of order (in the first test)
> which makes me wonder if ``B=>>`` has special meaning. If so, then
> line 1667 needs to be more sophisticated and do something different
> depending on the value of ``brace``.
>

Now that I've had time to sleep on it, I think that my patch might be
correct. "second" isn't really out of order. Both "second" and "third"
were set to come after "first", but they were never set in relation to
each other so "second" coming after "third" is not wrong, especially
considering that "second" was added later than "third".

If we really want "second" to come before "third", then we should tell
is so with "<third". As "third" is already set to come after "first",
this will force "second" to fall between "first" and "third". That
should provide enough control.

What this essentially means is that when adding items in order (as in
the Markdown class), they should be added using ">the_last_item"
(which is the default when no priority is set on add). However
whenever later inserting nodes, they should be added using
"<some_item" for precision placement.

Is this understanding correct? or am I crazy?

If I'm not crazy (at least with respect to this point), then I'll
commit the patch, update the tests and add a more polished version of
the above explanation to the docs so future users don't get surprised
like I did.


-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] Bug in Treap??

[Waylan L. <wa...@gm...>]

I think I found a bug in Treap when using the "<key" syntax but I'm
not sure. My understanding is that a node assigned such a priority
would be inserted *before* the "key" node. However it never is. In
fact, the last test in treap_test.py displays this same behavior. I
was confused at first thinking the test was right, but it is obviously
wrong.

    print ".. Change priority test."
    r.link('seventh', '<third')
    test('.... vals', r.heapsorted() == ['This', 'a', 'self', 'CRAZY', 'test'])
    test('.... keys', r.heapsorted(keys=1) == ['first', 'third',
'fourth','seventh', 'fifth'])
    test('.... items', r.heapsorted(items=1) == [('first', 'This'),
('third', 'a'), ('fourth', 'self'), ('seventh','CRAZY'), ('fifth',
'test')])

If "seventh" is linked *before* "third", then the keys should be
['first', 'seventh 'third', 'fourth', 'fifth'].

I think there may be a clue in ``r._tree`` (I added
``pprint(r._etree)`` to the end of treap_test.py:

    {'_begin': {'heap': ('_begin', 'B'), 'kids': {'first': '='}, 'prio': 'B'},
     '_end': {'heap': ('_end', 'E'), 'kids': {'seventh': '='}, 'prio': 'E'},
     'fifth': {'heap': ('fifth', 'B=>>'), 'kids': {}, 'prio': '>fourth'},
     'first': {'heap': ('first', 'B='),
               'kids': {'fourth': '>', 'third': '>'},
               'prio': '_begin'},
     'fourth': {'heap': ('fourth', 'B=>'),
                'kids': {'fifth': '>'},
                'prio': '>first'},
     'seventh': {'heap': ('seventh', 'B=><'), 'kids': {}, 'prio': '<third'},
     'third': {'heap': ('third', 'B=>'),
               'kids': {'seventh': '<'},
               'prio': '>first'}}

Notice that "seventh" has ``'B=><'`` and 'fifth' has ``'B=>>'``. In
further testing, I've noted this same pattern in each instance I can't
get Treap to sort the way I want.

If I understand things correctly, line 1667 is the problem. It currently is:

    self._tree[parn]['heap'][1]+brace

and should be:

    self._tree[parn]['heap'][1].strip('<>')+brace

The problem is, that takes "second" out of order (in the first test)
which makes me wonder if ``B=>>`` has special meaning. If so, then
line 1667 needs to be more sophisticated and do something different
depending on the value of ``brace``.

Or am I missing something obvious?
-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] refactoring and treap integration

[Waylan L. <wa...@gm...>]

On Mon, Oct 13, 2008 at 7:35 PM, Yuri Takhteyev <qar...@gm...> wrote:
>> Why not just make the InlineProcessor be one of the 'postprocessors'
>> and then extensions can add additional postprocessors either before or
>> after it as needed?
>
> Good point.  If we then also get rid of the
> preprocessor/textpreprocessor distinction, we can just reduce it all
> to three:
>

FYI, I just pushed the last of these changes. We now only have three
types of processors:

Preprocessor treap: HtmlBlock, Header, Line, Reference
Treeprocessors treap: Inline, Prettify
Postprocessors treap: RawHtml, AndSubstitute

If anyone has the old TextPostprocessors, or either of the old
postprocessors in your extensions, you'll need to make a few minor
updates for things to work. InlinePatterns should be unaffected - it's
just that now you can manipulate the tree before they are run if you
desire.

I should also mention that all this stuff is fully documented in
docs/writing_extensions.txt. Any improvements, corrections,
suggestions are welcome.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] New TOC extension.

[Yuri T. <qar...@gm...>]

> Just guessing, but did you turn on anchors in the config (it's off by
> default - and should be IMO)? And did you add the marker ``[TOC]`` to
> the document for placement of the TOC.

Turns out I was putting "[TOC]" into the wrong file. :)  Yes, I agree
that anchor links should be off by default.

> 1. Do nothing (current implementation). Without a marker you don't get
> a TOC - just ids. Although the HeaderID ext already gives you this, so
> the only value here is when an author wants some documents to have a
> TOC, but not others.
>
> 2. Insert the TOC into the beginning of the document by default.
> Although, this may be undesireable in some instances, so would need to
> be configurable.

I am a little divided between those, but if we go with #2, then we
would need a parameter to get #1 behavior, which I think may well be
needed at times.  If we stick with #1, again, a config parameter to
get #2 may be helpful, but less important.

> 3. Make the TOC available as an attribute on the Markdown class (much
> like Meta-Data ext does). This way a templating system could place the
> TOC elsewhere in a HTML template for a document if desired. Actually,
> this could happen regardless of 1 or 2.

Perhaps it would be a little cleaner to make those attributes of the
extensions?  I.e., have something like: md.extensions['toc'].tocTree

 - yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] New TOC extension.

[Yuri T. <qar...@gm...>]

> One thing that *might* be related to toc generation is more of a "tag
> cloud" generation thing... Heres a description of what I mean:

What may be useful for us to think about is how / whether both TOC and
some sort of tagcloud extension would work with multiple files.  I
would assume that tagcloud would usually be more useful when it covers
multiple pages.  TOC would _usually_ be used with a single page.
However, a multi-page TOC may be useful in some cases.

Up to now, the extensions has been allowed to be sticky by default.  I
haven't tested it recently, but the footnote extension is really
supposed to allow you to run multiple files collecting the footnote
info and then dump the actual footnotes into a third.   If you want to
forget what footnotes you've seen in earlier files, you need to call
Markdown.reset().  Two issues with this.  First, I've been wondering
if this is a good default solution.  Perhaps it would be best to
default to a clean-slate processing.  Instead of convert() and
reset(), we could have convert() and convertMore(), with convert()
always resetting the state completely.  So, your usage would be:

    m = Markdown(extensions=["tagcloud"])
    for x in pages:
          m.convertMore(x)
    tagcloud = m.convertMore("[TagCloud]")

Second, however, for this to be useful, extensions need to be able to
somehow keep track of what is coming from what file.  So, convert()
and convertMore() would need to take some sort of ID parameter.  In
other words, m.convertMore(page_source, id) would let the tagcloud
extension collect the tags knowing where those tags are coming from
and then generate proper links.  So, we would get something like:

    m = Markdown(extensions=["tagcloud"])
    for id in page_ids:
          converted[id] = m.convertMore(raw_pages[id], id)
    tagcloud = m.convertMore("[TagCloud]")

 - yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] New TOC extension.

[Waylan L. <wa...@gm...>]

On Fri, Oct 17, 2008 at 12:31 PM, Yuri Takhteyev <qar...@gm...> wrote:
> Thanks.  I appreciate you doing this work.
>
> I checked the extension in under markdown_extensions/toc.py and put a
> test file under tests/extensions-x-toc/syntax-toc.txt.
>
> I can't get it to work for some reason, though.  The file extension
> loads and does add the IDs, but doesn't seem to add the anchors or the
> actual TOC.  Am I missing something?

Just guessing, but did you turn on anchors in the config (it's off by
default - and should be IMO)? And did you add the marker ``[TOC]`` to
the document for placement of the TOC.

Which brings to mind another thought I had when I was planning my
approach to this. What should be the default behavior when the
document does not contain a marker? I can think of 3 reasonable
behaviors:

1. Do nothing (current implementation). Without a marker you don't get
a TOC - just ids. Although the HeaderID ext already gives you this, so
the only value here is when an author wants some documents to have a
TOC, but not others.

2. Insert the TOC into the beginning of the document by default.
Although, this may be undesireable in some instances, so would need to
be configurable.

3. Make the TOC available as an attribute on the Markdown class (much
like Meta-Data ext does). This way a templating system could place the
TOC elsewhere in a HTML template for a document if desired. Actually,
this could happen regardless of 1 or 2.


>
>  - yuri
>
> On Fri, Oct 17, 2008 at 8:13 AM, Jack Miller <ja...@co...> wrote:
>> On Fri, Oct 17, 2008 at 09:30:49AM -0400, Waylan Limberg wrote:
>>> Oh, one other thing, wrap the toctitle in an if statement and only add
>>> it if a title is set in the config settings. Right now your getting a
>>> blank title ``<span class="toctitle" />``.
>>>
>>
>> Yeah, I actually just forgot the [0] on the config, so typo fixed.
>>
>>> > * Why are you creating anchors in each header? Why not just assign
>>> > each header an "id" attribute? That is generally considered the
>>> > correct way to do things and works just the same.
>>> >
>>
>> Well, I like the headers to be turned into links. As with a lot of things
>> that are pure preference, I just turned it into a config option (anchorlink)
>> which defaults to 0, but if set makes the link.
>>
>>> > * It would be nice if it worked with the HeaderID extension. That is,
>>> > if a header already has an id attribute, that is used rather than
>>> > recreating one which may be different.
>>
>> Ah yes. Fixed.
>>
>> The code has been updated:
>> http://codezen.org/static/toc-updated.py
>>
>> --Jack
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Python-markdown-discuss mailing list
>> Pyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>>
>
>
>
> --
> http://sputnik.freewisdom.org/
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] New TOC extension.

[Magnus L. H. <ma...@he...>]

On Oct 17, 2008, at 18:31 , Yuri Takhteyev wrote:

> Thanks.  I appreciate you doing this work.
>
> I checked the extension in under markdown_extensions/toc.py and put a
> test file under tests/extensions-x-toc/syntax-toc.txt.


One thing that *might* be related to toc generation is more of a "tag  
cloud" generation thing... Heres a description of what I mean:

   Anyone got a good text-parser for tagged text?
   http://www.boingboing.net/2008/08/12/anyone-got-a-good-te.html

Maybe it's heretical to suggest something like this -- either some new  
syntax would have to be introduced, or some existing syntax would have  
to be hijacked -- but it's something I'm looking for, so I thought I'd  
mention it :)

(In some ways this is more related to generating an index -- or  
multiple indices. After all, a tag cloud is just a topic index, with  
hyperlinks instead of page numbers.)

-- 
Magnus Lie Hetland
http://hetland.org

Re: [Python-markdown-discuss] New TOC extension.

[Yuri T. <qar...@gm...>]

Thanks.  I appreciate you doing this work.

I checked the extension in under markdown_extensions/toc.py and put a
test file under tests/extensions-x-toc/syntax-toc.txt.

I can't get it to work for some reason, though.  The file extension
loads and does add the IDs, but doesn't seem to add the anchors or the
actual TOC.  Am I missing something?

 - yuri

On Fri, Oct 17, 2008 at 8:13 AM, Jack Miller <ja...@co...> wrote:
> On Fri, Oct 17, 2008 at 09:30:49AM -0400, Waylan Limberg wrote:
>> Oh, one other thing, wrap the toctitle in an if statement and only add
>> it if a title is set in the config settings. Right now your getting a
>> blank title ``<span class="toctitle" />``.
>>
>
> Yeah, I actually just forgot the [0] on the config, so typo fixed.
>
>> > * Why are you creating anchors in each header? Why not just assign
>> > each header an "id" attribute? That is generally considered the
>> > correct way to do things and works just the same.
>> >
>
> Well, I like the headers to be turned into links. As with a lot of things
> that are pure preference, I just turned it into a config option (anchorlink)
> which defaults to 0, but if set makes the link.
>
>> > * It would be nice if it worked with the HeaderID extension. That is,
>> > if a header already has an id attribute, that is used rather than
>> > recreating one which may be different.
>
> Ah yes. Fixed.
>
> The code has been updated:
> http://codezen.org/static/toc-updated.py
>
> --Jack
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] New TOC extension.

[Waylan L. <wa...@gm...>]

On Fri, Oct 17, 2008 at 11:13 AM, Jack Miller <ja...@co...> wrote:
> On Fri, Oct 17, 2008 at 09:30:49AM -0400, Waylan Limberg wrote:
>> Oh, one other thing, wrap the toctitle in an if statement and only add
>> it if a title is set in the config settings. Right now your getting a
>> blank title ``<span class="toctitle" />``.
>>
>
> Yeah, I actually just forgot the [0] on the config, so typo fixed.

Yeah, I've made that mistake before as well. Perhaps we need a better
config system. I'll have to think about that.

>
>> > * Why are you creating anchors in each header? Why not just assign
>> > each header an "id" attribute? That is generally considered the
>> > correct way to do things and works just the same.
>> >
>
> Well, I like the headers to be turned into links. As with a lot of things
> that are pure preference, I just turned it into a config option (anchorlink)
> which defaults to 0, but if set makes the link.

Fair enough. Personally, I like it when a little paragraph symbol
appears to the right of a header on hover, rather than the entire
header being an anchor, but that may be beyond this extension (unless
it was a third option) as the user would have to provide their own css
etc. The current implementation is certainly good enough (tm).

>
>> > * It would be nice if it worked with the HeaderID extension. That is,
>> > if a header already has an id attribute, that is used rather than
>> > recreating one which may be different.
>
> Ah yes. Fixed.

Well, mostly. Now, if a header already has an id, both the header and
anchor will get the id. As ids should be unique, this creates invalid
html. I'd suggest *not* assigning an id to the anchor - just the
header.

And speaking of unique, how are you guaranteeing that slugify does not
create the same id for two different headers? Not a showstopper, but
something to think about. You might be interested in how HeaderID
addresses this.

>
> The code has been updated:
> http://codezen.org/static/toc-updated.py
>

I don't mean to be so nitpicky Jack, but I would like something like
this packaged up with Markdown and it has to be right before that
happens. When I'm happy with it and with your blessings (and no
objection from Yuri) I'll commit it to the repo.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] New TOC extension.

[Jack M. <ja...@co...>]

On Fri, Oct 17, 2008 at 09:30:49AM -0400, Waylan Limberg wrote:
> Oh, one other thing, wrap the toctitle in an if statement and only add
> it if a title is set in the config settings. Right now your getting a
> blank title ``<span class="toctitle" />``.
>

Yeah, I actually just forgot the [0] on the config, so typo fixed.

> > * Why are you creating anchors in each header? Why not just assign
> > each header an "id" attribute? That is generally considered the
> > correct way to do things and works just the same.
> >

Well, I like the headers to be turned into links. As with a lot of things
that are pure preference, I just turned it into a config option (anchorlink)
which defaults to 0, but if set makes the link.

> > * It would be nice if it worked with the HeaderID extension. That is,
> > if a header already has an id attribute, that is used rather than
> > recreating one which may be different.

Ah yes. Fixed.

The code has been updated:
http://codezen.org/static/toc-updated.py

--Jack