python-markdown-discuss — A general purpose discussion list

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Yuri T. <qar...@gm...>]

> Or for any platform:
> easy_install http://splyer.com/markdown2.0-alpha.tar.gz
> easy_install elementtree

That too, though if you are limited to python2.3, chances are you
don't have easy_install pre-installed. :)

> What about AND_SUBSTITUTE ? I use it in some places instead of "&", now
> it's:
> AND_SUBSTITUTE = unichr(2) + unichr(4) + unichr(3)
> is it ok ?

Just for the sake of consistency, I would make it STX+"amp"+ETX.  And
I would call it AMPERSAND_SUBSTITUTE or AMP_SUBSTITUTE, since
AND_SUBSTITUTE can mean too many things.

Also, can you add more comments to the code?  (I mean especially the
parts that you changed, though you are welcome to comment old code as
well.)

Also, we had that discussion of recursion before.  I am in principle
all for getting rid of recursion as Waylan was suggesting, but I think
considering the time constraints, perhaps it would be more practical
to cap recursion by simply breaking the input into chunks.  This needs
to be done with a little bit of thought so that we don't break lists,
etc. into multiple chunks.  However, I think it will be much easier
than trying to rewrite the current _processSection logic and it should
get us the same benefits in terms of performance.

  - yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Yuri Takhteyev wrote:
>
> I tried running the new version with Python 2.5, 2.4 and 2.3.  For
> 2.5, all the tests pass out of the box, no element tree installation
> required.  For 2.4, I had to install element tree, after which all
> works fine.  So, we just need to make sure to document this.  I think
> we should include specific line-by-line instructions on the wiki,
> rather than just sending people to element tree website, which I
> myself found mildly confusing.  E.g., for UNIX:
>
>     wget http://splyer.com/markdown2.0.tar.gz
>     wget http://splyer.com/markdown2.0-alpha.tar.gz
>     tar xvzf markdown2.0-alpha.tar.gz
>     cd markdown2.0-alpha/
>     wget http://effbot.org/media/downloads/elementtree-1.2.6-20050316.tar.gz
>     tar xvzf elementtree-1.2.6-20050316.tar.gz
>     mv elementtree-1.2.6-20050316/elementtree .
>     python2.4 test-markdown.py
>   

Or for any platform:
easy_install http://splyer.com/markdown2.0-alpha.tar.gz
easy_install elementtree

> Also, another commit to correct placeholder codes to correct STX/ETX
> codes, and referring to them accordingly.  (Now using ASCII 02 and 03,
> rather than 01 and 02.)
>
>   

What about AND_SUBSTITUTE ? I use it in some places instead of "&", now 
it's:
AND_SUBSTITUTE = unichr(2) + unichr(4) + unichr(3)
is it ok ?

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Waylan Limberg wrote:
> On Wed, Aug 6, 2008 at 10:34 PM, Waylan Limberg <wa...@gm...> wrote:
>   
>> On Wed, Aug 6, 2008 at 6:41 PM, Artem Yunusov <ne...@gm...> wrote:
>>     
>>> Yep, I'll update doc strings by myself. Thanks for the link to PEP 257.
>>> I think extension writers need to know about markdown.etree module
>>> object, and also, that all data in <inline> tag will be processed by
>>> inline patterns, other functions, like indentETree is useless for them.
>>> I think we should write about all this on site in "Writing Extensions"
>>> section.
>>>       
>> I agree. Actually, that section needs to be completely reworked as a
>> number of additions/changes have happened to the extension API since
>> that page was last updated. The thing is, we don't want to update that
>> page until after 2.0 final is released or people may expect it to
>> apply to 1.7. I think maybe we should throw a text file in git for now
>> and copy it over the the wiki when we release. If you write up the
>> docs for the stuff you did Artem, I'll do the rest - unless you beat
>> me to it..
>>     
>
> FYI, I just committed a rough draft. Artem, I left the section on DOM
> manipulation for you. Any criticisms, suggestions or corrections are
> welcome. I'll copy it to the wiki when we release 2.0 final.
>   

Thanks, I'll write about it. One suggestion, I think we should not call 
ElementTree object - DOM, because it's slightly different things, as far 
as understand.

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Wed, Aug 6, 2008 at 10:34 PM, Waylan Limberg <wa...@gm...> wrote:
> On Wed, Aug 6, 2008 at 6:41 PM, Artem Yunusov <ne...@gm...> wrote:
>> Yep, I'll update doc strings by myself. Thanks for the link to PEP 257.
>> I think extension writers need to know about markdown.etree module
>> object, and also, that all data in <inline> tag will be processed by
>> inline patterns, other functions, like indentETree is useless for them.
>> I think we should write about all this on site in "Writing Extensions"
>> section.
>
> I agree. Actually, that section needs to be completely reworked as a
> number of additions/changes have happened to the extension API since
> that page was last updated. The thing is, we don't want to update that
> page until after 2.0 final is released or people may expect it to
> apply to 1.7. I think maybe we should throw a text file in git for now
> and copy it over the the wiki when we release. If you write up the
> docs for the stuff you did Artem, I'll do the rest - unless you beat
> me to it..

FYI, I just committed a rough draft. Artem, I left the section on DOM
manipulation for you. Any criticisms, suggestions or corrections are
welcome. I'll copy it to the wiki when we release 2.0 final.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Wed, Aug 6, 2008 at 6:41 PM, Artem Yunusov <ne...@gm...> wrote:
> Yep, I'll update doc strings by myself. Thanks for the link to PEP 257.
> I think extension writers need to know about markdown.etree module
> object, and also, that all data in <inline> tag will be processed by
> inline patterns, other functions, like indentETree is useless for them.
> I think we should write about all this on site in "Writing Extensions"
> section.

I agree. Actually, that section needs to be completely reworked as a
number of additions/changes have happened to the extension API since
that page was last updated. The thing is, we don't want to update that
page until after 2.0 final is released or people may expect it to
apply to 1.7. I think maybe we should throw a text file in git for now
and copy it over the the wiki when we release. If you write up the
docs for the stuff you did Artem, I'll do the rest - unless you beat
me to it..



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Yuri T. <qar...@gm...>]

> Concerning ElementTree version, I made some tests, all versions since
> 1.1(including) suitable for us. That means that only 1.0 failed. I'll
> add version check to importETree function.
> Python 2.5 uses ElementTree 1.2.6.

I tried running the new version with Python 2.5, 2.4 and 2.3.  For
2.5, all the tests pass out of the box, no element tree installation
required.  For 2.4, I had to install element tree, after which all
works fine.  So, we just need to make sure to document this.  I think
we should include specific line-by-line instructions on the wiki,
rather than just sending people to element tree website, which I
myself found mildly confusing.  E.g., for UNIX:

    wget http://splyer.com/markdown2.0.tar.gz
    wget http://splyer.com/markdown2.0-alpha.tar.gz
    tar xvzf markdown2.0-alpha.tar.gz
    cd markdown2.0-alpha/
    wget http://effbot.org/media/downloads/elementtree-1.2.6-20050316.tar.gz
    tar xvzf elementtree-1.2.6-20050316.tar.gz
    mv elementtree-1.2.6-20050316/elementtree .
    python2.4 test-markdown.py

Now, for 2.3 I am getting an error, however:

Traceback (most recent call last):
  File "test-markdown.py", line 358, in ?
    testDirectory("tests/markdown-test", measure_time=True)
  File "test-markdown.py", line 246, in testDirectory
    actual_output = md.convert(input)
  File "/usr/local/tmp/markdown2.0-alpha/markdown.py", line 1929, in convert
    xml = codecs.decode(etree.tostring(root, encoding="utf8"), "utf8")

As I realize, this is a line I added myself.  Apparently, the codecs
module in 2.3 does not have a "decode" function.  I replaced line 1929
with:

    xml, length = codecs.utf_8_decode(etree.tostring(root, encoding="utf8"))

after which it seems to all work with python2.3 as well.

I commited the change with

http://gitorious.org/projects/python-markdown/repos/mainline/commits/c78a1e7

Also, another commit to correct placeholder codes to correct STX/ETX
codes, and referring to them accordingly.  (Now using ASCII 02 and 03,
rather than 01 and 02.)

http://gitorious.org/projects/python-markdown/repos/mainline/commits/a2e4db6

 - yuri

-- 
http://sputnik.freewisdom.org/

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Yep, I'll update doc strings by myself. Thanks for the link to PEP 257.
I think extension writers need to know about markdown.etree module 
object, and also, that all data in <inline> tag will be processed by 
inline patterns, other functions, like indentETree is useless for them. 
I think we should write about all this on site in "Writing Extensions" 
section.
Concerning ElementTree version, I made some tests, all versions since 
1.1(including) suitable for us. That means that only 1.0 failed. I'll 
add version check to importETree function.
Python 2.5 uses ElementTree 1.2.6.

Waylan Limberg wrote:
> Artem,
>
> I have finally got a chance to start looking at your work more closely
> and really appreciate the time and effort you put into it. I still
> need to wrap my head around a few things, but it looks good so far.
>
> Looking through your code, I noticed a few missing or problem doc
> strings. I thought you might want to clean them up yourself. If you
> haven't already, I'd suggest reading [PEP 257][] first as a guideline.
> No doubt you'll notice that we haven't kept to that strictly, but I've
> been making an effort to improve in that area recently. In fact, a few
> months back I went through and added a significant amount of missing
> doc strings.
>
> My policy has been that any class/method/function that extension
> writers are likely to use should get a very thorough docstring
> spelling everything out, and anything else can have a one-liner
> (except that _private methods do not need one). So, for example, would
> an extension writer need to use some of your etree helper functions to
> build his/her elements for inclusion in the tree? I'm not sure, and I
> don't find any hints in the existing docstrings.
>
> Also, in PEP 257 you may want to note this comment with regard to
> one-liners (although the principle should apply to all docstrings):
>
>   
>> The docstring is a phrase ending in a period. It prescribes the function
>> or method's effect as a command ("Do this", "Return that"), not as a
>> description; e.g. don't write "Returns the pathname ...".
>>     
>
> It appears that most of your docstrings get that wrong, although its
> not too hard to fix. You might want to run a spellcheck on your
> docstrings as well ( I saw a few "retruns").
>
> Oh, and one final point, the docstrings should all use markdown
> formatting - especially the long ones - for reasons that should be
> obvious. I didn't notice any specific problems there, but figured it's
> worth pointing out.
>
> I realize this may seem a little nitpicky, and I get the impression
> Yuri isn't too worried about it, but it has been something others have
> complained about in the past and I really want to get it right before
> we release 2.0 final.
>
> [PEP 257]: http://www.python.org/dev/peps/pep-0257/
>
>
>   

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

While I'm thinking of it, one other thing that needs to be documented
is the dependency on ElementTree. What's the oldest required version?

Obviously it should be a non-issue for Python2.5, but for those who
have 1.3 or 2.4, they need to know that external library needs to be
installed, and if they happen to have a too-old version already
installed, we should warn them upfront. Perhaps on a successful
import, we should check the version and exit with an appropriate error
message if it's too old.

On Wed, Aug 6, 2008 at 11:37 AM, Waylan Limberg <wa...@gm...> wrote:
> Artem,
>
> I have finally got a chance to start looking at your work more closely
> and really appreciate the time and effort you put into it. I still
> need to wrap my head around a few things, but it looks good so far.
>
> Looking through your code, I noticed a few missing or problem doc
> strings. I thought you might want to clean them up yourself. If you
> haven't already, I'd suggest reading [PEP 257][] first as a guideline.
> No doubt you'll notice that we haven't kept to that strictly, but I've
> been making an effort to improve in that area recently. In fact, a few
> months back I went through and added a significant amount of missing
> doc strings.
>
> My policy has been that any class/method/function that extension
> writers are likely to use should get a very thorough docstring
> spelling everything out, and anything else can have a one-liner
> (except that _private methods do not need one). So, for example, would
> an extension writer need to use some of your etree helper functions to
> build his/her elements for inclusion in the tree? I'm not sure, and I
> don't find any hints in the existing docstrings.
>
> Also, in PEP 257 you may want to note this comment with regard to
> one-liners (although the principle should apply to all docstrings):
>
>> The docstring is a phrase ending in a period. It prescribes the function
>> or method's effect as a command ("Do this", "Return that"), not as a
>> description; e.g. don't write "Returns the pathname ...".
>
> It appears that most of your docstrings get that wrong, although its
> not too hard to fix. You might want to run a spellcheck on your
> docstrings as well ( I saw a few "retruns").
>
> Oh, and one final point, the docstrings should all use markdown
> formatting - especially the long ones - for reasons that should be
> obvious. I didn't notice any specific problems there, but figured it's
> worth pointing out.
>
> I realize this may seem a little nitpicky, and I get the impression
> Yuri isn't too worried about it, but it has been something others have
> complained about in the past and I really want to get it right before
> we release 2.0 final.
>
> [PEP 257]: http://www.python.org/dev/peps/pep-0257/
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

Artem,

I have finally got a chance to start looking at your work more closely
and really appreciate the time and effort you put into it. I still
need to wrap my head around a few things, but it looks good so far.

Looking through your code, I noticed a few missing or problem doc
strings. I thought you might want to clean them up yourself. If you
haven't already, I'd suggest reading [PEP 257][] first as a guideline.
No doubt you'll notice that we haven't kept to that strictly, but I've
been making an effort to improve in that area recently. In fact, a few
months back I went through and added a significant amount of missing
doc strings.

My policy has been that any class/method/function that extension
writers are likely to use should get a very thorough docstring
spelling everything out, and anything else can have a one-liner
(except that _private methods do not need one). So, for example, would
an extension writer need to use some of your etree helper functions to
build his/her elements for inclusion in the tree? I'm not sure, and I
don't find any hints in the existing docstrings.

Also, in PEP 257 you may want to note this comment with regard to
one-liners (although the principle should apply to all docstrings):

> The docstring is a phrase ending in a period. It prescribes the function
> or method's effect as a command ("Do this", "Return that"), not as a
> description; e.g. don't write "Returns the pathname ...".

It appears that most of your docstrings get that wrong, although its
not too hard to fix. You might want to run a spellcheck on your
docstrings as well ( I saw a few "retruns").

Oh, and one final point, the docstrings should all use markdown
formatting - especially the long ones - for reasons that should be
obvious. I didn't notice any specific problems there, but figured it's
worth pointing out.

I realize this may seem a little nitpicky, and I get the impression
Yuri isn't too worried about it, but it has been something others have
complained about in the past and I really want to get it right before
we release 2.0 final.

[PEP 257]: http://www.python.org/dev/peps/pep-0257/


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Tue, Aug 5, 2008 at 5:28 PM, Artem Yunusov <ne...@gm...> wrote:
>
> Done. What should I do with ticket#4 ? it's fixed only for not nested
> parenthesis.
> Also in 2.0-alpha version any number of parenthesis can be inserted in
> that way: [link](<()()((()>)
>

I'd add a comment to the bottom of the report to that effect. Point
out that the given example works but nesting does not. I'm tempted to
say close it as the report didn't specifically address nesting anyway
- and in fact, we may never get it given pythons lack of recursion in
the re module.


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Waylan Limberg wrote:
> On Mon, Aug 4, 2008 at 2:07 PM, Artem Yunusov <ne...@gm...> wrote:
>   
>> Hello all,
>>
>> I merged GSoC version to our mainline repository and now we close to
>> release 2.0 version(after some testing).
>>     
>
> Artem, now that your patches are merged to trunk, could you close the
> related tickets. I could do it, but I'm not sure which tickets you've
> addressed and which you haven't. Thanks.
>
>
>   

Done. What should I do with ticket#4 ? it's fixed only for not nested 
parenthesis.
Also in 2.0-alpha version any number of parenthesis can be inserted in 
that way: [link](<()()((()>)

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Mon, Aug 4, 2008 at 2:07 PM, Artem Yunusov <ne...@gm...> wrote:
> Hello all,
>
> I merged GSoC version to our mainline repository and now we close to
> release 2.0 version(after some testing).

Artem, now that your patches are merged to trunk, could you close the
related tickets. I could do it, but I'm not sure which tickets you've
addressed and which you haven't. Thanks.


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Thanks for the advice, new link is:

http://splyer.com/markdown2.0-alpha.tar.gz


Waylan Limberg wrote:
> On Mon, Aug 4, 2008 at 3:20 PM, Waylan Limberg <wa...@gm...> wrote:
>   
>> On Mon, Aug 4, 2008 at 2:07 PM, Artem Yunusov <ne...@gm...> wrote:
>>     
>>> Hello all,
>>>
>>> I merged GSoC version to our mainline repository and now we close to
>>> release 2.0 version(after some testing).
>>> Please, try it out. You can take it from repository or just download
>>> tarball file:
>>>
>>> http://splyer.com/markdown2.0.tar.gz
>>>
>>>       
>> I hate to nitpick, but I wouldn't call this 2.0 yet. Perhaps
>> 2.0-alpha. It gets confusing later when trying to debug bug reports
>> and different people have different versions all labeled the same. I'd
>> suggest renaming that file on your server before to many people
>> download it and it creates a real problem.
>>
>>     
>
> Oh, I see you set the version_info to 2.0.0.beta in the repo, just not
> in the filename (I still recommend renaming the file). I haven't
> downloaded the tar.gz file yet (I can't behind the firewall here at
> work) so I don't know what all it includes, but I prefer to use
> setup.py to create the distribution files. As long as the setup.py
> script has the right version info (it appears you did not update that
> in the repo) it will generate the appropriate file names with all the
> version info included.
>
>
>   

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Mon, Aug 4, 2008 at 3:20 PM, Waylan Limberg <wa...@gm...> wrote:
> On Mon, Aug 4, 2008 at 2:07 PM, Artem Yunusov <ne...@gm...> wrote:
>> Hello all,
>>
>> I merged GSoC version to our mainline repository and now we close to
>> release 2.0 version(after some testing).
>> Please, try it out. You can take it from repository or just download
>> tarball file:
>>
>> http://splyer.com/markdown2.0.tar.gz
>>
> I hate to nitpick, but I wouldn't call this 2.0 yet. Perhaps
> 2.0-alpha. It gets confusing later when trying to debug bug reports
> and different people have different versions all labeled the same. I'd
> suggest renaming that file on your server before to many people
> download it and it creates a real problem.
>

Oh, I see you set the version_info to 2.0.0.beta in the repo, just not
in the filename (I still recommend renaming the file). I haven't
downloaded the tar.gz file yet (I can't behind the firewall here at
work) so I don't know what all it includes, but I prefer to use
setup.py to create the distribution files. As long as the setup.py
script has the right version info (it appears you did not update that
in the repo) it will generate the appropriate file names with all the
version info included.


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC version merged to mainline

[Waylan L. <wa...@gm...>]

On Mon, Aug 4, 2008 at 2:07 PM, Artem Yunusov <ne...@gm...> wrote:
> Hello all,
>
> I merged GSoC version to our mainline repository and now we close to
> release 2.0 version(after some testing).
> Please, try it out. You can take it from repository or just download
> tarball file:
>
> http://splyer.com/markdown2.0.tar.gz
>
I hate to nitpick, but I wouldn't call this 2.0 yet. Perhaps
2.0-alpha. It gets confusing later when trying to debug bug reports
and different people have different versions all labeled the same. I'd
suggest renaming that file on your server before to many people
download it and it creates a real problem.


-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] GSoC version merged to mainline

[Artem Y. <ne...@gm...>]

Hello all,

I merged GSoC version to our mainline repository and now we close to 
release 2.0 version(after some testing).
Please, try it out. You can take it from repository or just download 
tarball file:

http://splyer.com/markdown2.0.tar.gz

Re: [Python-markdown-discuss] Command line script

[Yuri T. <qar...@gm...>]

> their own code, so I'll probably at least leave that. But should I
> leave all the command line stuff in markdown.py and just do a short
> script something like:

I can see the value of having an executable script installed in a
proper place, but I don't think we should remove the command-line
stuff from the main (="module") script.   We could add an extra file,
but I would really prefer to still have a combined script that can be
either called from the command line or imported as a module.  I know
that such combined script is the only version that I would ever use
myself.

>    #!/usr/bin/env python
>    if __name__ == '__main__':
>        import markdown
>        options = markdown.parse_options()
>
>        if not options:
>            sys.exit(0)
>
>        markdown.markdownFromFile(**options)

Could be.

  - yuri


-- 
http://sputnik.freewisdom.org/

[Python-markdown-discuss] Command line script

[Waylan L. <wa...@gm...>]

Something that has bothered me for a while now is that when we run
`setup.py install`, markdown is installed just fine for module use.
However, if one wants to use it on the command line, then markdown.py
needs to be copied or linked to the system path and made executable.
The thing is, if a separate comand line script is provided with the
distribution, then the install will copy that to the path and make it
executable by default. Usually `/usr/bin/` or `/usr/local/bin/` or
some similar variant on *nix systems (depending on distribution) and
assuming a default install, `C:\\Python\scripts\` on windows[^1]. That
way, markdown.py doesn't need to be in the same dir as the source file
and there's no need to append `python` to the front of the command
(windows uses the file extension and *nix the shebang line). Much more
useful IMO.

The question is, what all should that command line script contain? I'm
tempted to rip everything out of markdown.py that is related to the
command line stuff and only have that in the script. But I also
realize that some people may what to still use `markdownFromFile()` in
their own code, so I'll probably at least leave that. But should I
leave all the command line stuff in markdown.py and just do a short
script something like:

    #!/usr/bin/env python
    if __name__ == '__main__':
        import markdown
        options = markdown.parse_options()
				
	if not options:
            sys.exit(0)
		
        markdown.markdownFromFile(**options)

or the more drastic example here:
http://gitorious.org/projects/python-markdown/repos/waylans-sandbox/commits/9f8232a0235ee95873c9d53531f23eabd8df33c4
and all that stuff stripped from markdown.py here:
http://gitorious.org/projects/python-markdown/repos/waylans-sandbox/commits/30f0464a90a71dd2fbefa9f5c7403f0f831c05f0

Any preferences?

Oh, and I should mention that, unlike in the above linked example, I'd
name the script markdown.py for consistency.

[^1]: I realize `C:\\Python\scripts\` is not on the system path by
default, but you really should add it. There are all sorts of handy
tools in there. I should also note the I may have the path slightly
wrong, but I don't have a windows install handy at the moment to
check. Don't worry, I'll check before updating the docs.

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Python-markdown wikilinks extension

[David W. <wo...@cs...>]

On 13:00 Mon 21 Jul     , Waylan Limberg wrote:
> However, it would be very easy to either build your own extension or
> perhaps even use my extension with a different regex which provides
> the features you desire. I encourage you to give it a shot. If you
> need any help, feel free to ask. I imagine a "bracketed links"
> extension (perhaps with a better name) would be a valuable asset to
> the Python-Markdown community.
When I implemented bracketed (think: wikipedia) links in
DrProject, I don't remember it taking much more than an
afternoon.  
Now, of course, I can't find that code right now... But
if you're having trouble with it, though, I can look harder.

David

Re: [Python-markdown-discuss] Python-markdown wikilinks extension

[Waylan L. <wa...@gm...>]

Sean,

First of all, thanks for your feedback. I appreciate hearing from my users.

Second, as the wikilink extension is now part of the Markdown package
(and will likely be included in the next release), I'm copying the
python-markdown list with this response.

Personally, I don't care for CamelCase links in my documents for the
same reasons you mention, among others (and therefore don't use the
extension). However, the WikiLink extension is specifically to add
CamelCase links, so I won't be removing that feature. I wrote the
WikiLink extension as a learning exercise and have maintained it since
because some seem to find is useful. At the very least, others have
found it to be simple starting point for creating their own Inline
extensions. Therefore, I'm inclined to not add any additional features
and over complicate it for the beginner.

However, it would be very easy to either build your own extension or
perhaps even use my extension with a different regex which provides
the features you desire. I encourage you to give it a shot. If you
need any help, feel free to ask. I imagine a "bracketed links"
extension (perhaps with a better name) would be a valuable asset to
the Python-Markdown community.

Btw, some of your objections have already been addressed in the
existing extension. For instance, an underscore in a Wiki_Link will be
converted to a space ("Wiki Link" in this case). Additionally, a
\CamelCase word can be escaped with a backslash. I understand these
solutions may not be ideal for everyone, but that is why we try to
keep Markdown's extension api easy to use. Anyone can add new or
override existing behavior.

On Mon, Jul 21, 2008 at 11:42 AM, Sean Hammond <S.P...@sm...> wrote:
> Like this extension, but I wonder can it be extended to allow wikilinking to
> pages whose name is only one word? e.g. if a page is simply called "About" that
> cannot be spelled in CamelCase. Perhaps to make a wikilink to this page the
> user could write [[About]]. Double brackets is generally a better syntax
> because it allows one-word page names and page names with spaces in them, and
> CamelCase will sometimes match things that are not page names that then have to
> be escaped somehow. An option could then turn off the linkifying of CamelCase
> words and use only [[Double Brackets]]. I think it could be done by modifying
> the regular expression WIKILINK_RE. What do you think?
>
> Thanks
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] GSoC current progress

[Artem Y. <ne...@gm...>]

Waylan Limberg wrote:
> On Sat, Jul 19, 2008 at 9:28 AM, Artem Yunusov <ne...@gm...> wrote:
>   
>> I deleted Pattern.contentGroup attribute, that I previously added,
>> because I solved this problem using regexps. I also created new regexp
>> for links, it partly solves ticket#4 because it works fine, until you'll
>> try to insert nested parenthesis in link, in Perl  implementation it's
>> solved with recursive  regexp, but I don't see any way of doing it in
>> Python using regexps.
>>
>>     
>
> Yeah I was just looking at that yesterday. A small annoyance.
>
> [snip]
>   

Maybe we should use some combination of parsing and regexps here. But I 
think it's not very important bug now.

>   
>> Maybe some refactoring. For instance class CorePatterns scheduled for
>> refactoring, but now I don't have an idea what can be a better
>> replacement for it.
>>
>>     
>
> I'd leave this alone for now. IMO, what needs to happen is the current
> code should be (mostly) thrown out (sorry Yuri) and we need to create
> some easily extendable API where each block-level parser is its own
> class or something (perhaps a state machine or similar). And in the
> process, we need to lose the recursion that we have now. I suspect
> this would be a major undertaking and an entire summer project in
> itself.
>
> The recursion issue could probably be dealt with on it's own, but
> while we're in there, we might as well make is extendable. Currently,
> if you wanted to write an extension that added another block-level
> type (say definition lists) you would have to both create a processor
> for them and then override the _processSection method (through a
> monkey patch) with your own that reimplements the entire existing
> method with the appropriate calls to your new definition list
> processor added into the logic. It's ugly and complicated and doesn't
> work if multiple extensions take the same approach.
>   

Hmm, so then there will be new type of extensions, something called 
block level patterns.
I think it'll slow down markdown, but sure it'll be quite useful.
Yuri, what do you think about it?

> However, that does bring to mind something you could do. Perhaps try
> creating a definition list extension as a preproccessor. Not sure how
> you'd get it into the DOM, but I'm sure there are a few possibilities.
> Just be sure to follow the syntax used by PHP Markdown Extra. That
> seems to be the community accepted syntax.
>
> Oh, and while I'm at it, I should mention that I committed a small
> patch [1] for an undocumented bug that David had reintroduced while
> refactoring the extension api. Perhaps you're keeping an eye on the
> master repo and already know about it, but you may want to merge that
> into your code.
>
> [1]: http://gitorious.org/projects/python-markdown/repos/mainline/commits/bf7cf776daa26d734c10a6039efe64113f066045
>   

Yep, I saw it, I think I'll integrate it in GSoC version.

Re: [Python-markdown-discuss] List bug

[Yuri T. <qar...@gm...>]

Hmm.  Actually, perl markdown and most other implementations do more
or less the same thing, but this is not much of an excuse, considering
that PHP Markdown does the right thing here.  Luckily, it seems like
there is an easy fix in this case.

http://gitorious.org/projects/python-markdown/repos/mainline/commits/caef3e9

- yuri

On Sat, Jul 19, 2008 at 7:27 PM, Rob van der Linde <ro...@gm...> wrote:
> Hi, I have found this bug with lists in python-markdown and don't know
> where else to report this. I've actually known about it for a while, but
> only just managed to put together a test case. The bug exists in 1.6 and
> 1.7 as far as I know.
>
> Ok, so here's my test case (take out the --snip-- off course):
>
> -- snip --
> Title
> -----
>
> 1. dgfg
> 2. dg
> 3. dsfg
>
> Something in between and it's ok...
>
> * ddfgd
> * ddgdfg
> * dfgdfg
>
> Something else in between...
>
> 1. dg
> 2. dfgdfg
> 3. dgdg
>
> * dgdg
> * dgdfg
> * dgfdg
>
> The above will produce the error
> -- snip --
>
> Now this will produce the following HTML (I cleaned it up only for readability):
>
> <h2>Title</h2>
>
> <ol>
> <li>dgfg</li>
> <li>dg</li>
> <li>dsfg</li>
> </ol>
>
> <p>Something in between and it's ok...</p>
>
> <ul>
> <li>ddfgd</li>
> <li>ddgdfg</li>
> <li>dfgdfg</li>
> </ul>
>
> <p>Something else in between...</p>
>
> <ol>
> <li><p>dg</p></li>
> <li><p>dfgdfg</p></li>
> <li><p>dgdg</p></li>
> <li><p>dgdg</p></li>
> <li><p>dgdfg</p></li>
> <li><p>dgfdg</p></li>
> </ol>
>
> <p>The above will produce the error</p>
>
> Notice how the bulleted and numbered lists will merge together in the
> last example, as well as it put a paragraph inside each list item. This
> does not happen if I have a divider paragraph in between each list as
> the first two lists show.
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>
>



-- 
http://sputnik.freewisdom.org/

[Python-markdown-discuss] List bug

[Rob v. d. L. <ro...@gm...>]

Hi, I have found this bug with lists in python-markdown and don't know
where else to report this. I've actually known about it for a while, but
only just managed to put together a test case. The bug exists in 1.6 and
1.7 as far as I know.

Ok, so here's my test case (take out the --snip-- off course):

-- snip --
Title
-----

1. dgfg
2. dg
3. dsfg

Something in between and it's ok...

* ddfgd
* ddgdfg
* dfgdfg

Something else in between...

1. dg
2. dfgdfg
3. dgdg

* dgdg
* dgdfg
* dgfdg

The above will produce the error
-- snip --

Now this will produce the following HTML (I cleaned it up only for readability):

<h2>Title</h2>

<ol>
<li>dgfg</li>
<li>dg</li>
<li>dsfg</li>
</ol>

<p>Something in between and it's ok...</p>

<ul>
<li>ddfgd</li>
<li>ddgdfg</li>
<li>dfgdfg</li>
</ul>

<p>Something else in between...</p>

<ol>
<li><p>dg</p></li>
<li><p>dfgdfg</p></li>
<li><p>dgdg</p></li>
<li><p>dgdg</p></li>
<li><p>dgdfg</p></li>
<li><p>dgfdg</p></li>
</ol>

<p>The above will produce the error</p>

Notice how the bulleted and numbered lists will merge together in the
last example, as well as it put a paragraph inside each list item. This
does not happen if I have a divider paragraph in between each list as
the first two lists show.

Re: [Python-markdown-discuss] GSoC current progress

[Waylan L. <wa...@gm...>]

On Sat, Jul 19, 2008 at 9:28 AM, Artem Yunusov <ne...@gm...> wrote:
> I deleted Pattern.contentGroup attribute, that I previously added,
> because I solved this problem using regexps. I also created new regexp
> for links, it partly solves ticket#4 because it works fine, until you'll
> try to insert nested parenthesis in link, in Perl  implementation it's
> solved with recursive  regexp, but I don't see any way of doing it in
> Python using regexps.
>

Yeah I was just looking at that yesterday. A small annoyance.

[snip]

> Maybe some refactoring. For instance class CorePatterns scheduled for
> refactoring, but now I don't have an idea what can be a better
> replacement for it.
>

I'd leave this alone for now. IMO, what needs to happen is the current
code should be (mostly) thrown out (sorry Yuri) and we need to create
some easily extendable API where each block-level parser is its own
class or something (perhaps a state machine or similar). And in the
process, we need to lose the recursion that we have now. I suspect
this would be a major undertaking and an entire summer project in
itself.

The recursion issue could probably be dealt with on it's own, but
while we're in there, we might as well make is extendable. Currently,
if you wanted to write an extension that added another block-level
type (say definition lists) you would have to both create a processor
for them and then override the _processSection method (through a
monkey patch) with your own that reimplements the entire existing
method with the appropriate calls to your new definition list
processor added into the logic. It's ugly and complicated and doesn't
work if multiple extensions take the same approach.

However, that does bring to mind something you could do. Perhaps try
creating a definition list extension as a preproccessor. Not sure how
you'd get it into the DOM, but I'm sure there are a few possibilities.
Just be sure to follow the syntax used by PHP Markdown Extra. That
seems to be the community accepted syntax.

Oh, and while I'm at it, I should mention that I committed a small
patch [1] for an undocumented bug that David had reintroduced while
refactoring the extension api. Perhaps you're keeping an eye on the
master repo and already know about it, but you may want to merge that
into your code.

[1]: http://gitorious.org/projects/python-markdown/repos/mainline/commits/bf7cf776daa26d734c10a6039efe64113f066045


-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] GSoC current progress

[Artem Y. <ne...@gm...>]

I deleted Pattern.contentGroup attribute, that I previously added, 
because I solved this problem using regexps. I also created new regexp 
for links, it partly solves ticket#4 because it works fine, until you'll 
try to insert nested parenthesis in link, in Perl  implementation it's 
solved with recursive  regexp, but I don't see any way of doing it in 
Python using regexps.

Perl code:
$g_nested_parens = qr{
    (?>                                 # Atomic matching
       [^()\s]+                            # Anything other than parens 
or whitespace
     |
       \(
         (??{ $g_nested_parens })        # Recursive set of nested brackets
       \)
    )*
}x;

Now link regexp works for angled links too, so I deleted 
LINK_ANGLED_PATTERN from patterns list. Now link regexp is quite 
complicated, so what about of using re.VERBOSE flag? I tried it without 
any changes to regexps, but it's not working, seems that Python goes to 
infinite loop after adding this flag.
Also I created aggregated regexp for STRONG_RE and STRONG_2_RE and 
aggregated regexp for STRONG_EM_RE and STRONG_EM_2_RE. So STRONG_2_RE 
and STRONG_EM_2_RE can be deleted form patterns list.

Now I'm gathering different issues/bugs, I think I'll post it on Monday 
for discussion, which of them we want to fix.
Another thing I plan to do - port extensions to ElementTree.
Maybe some refactoring. For instance class CorePatterns scheduled for 
refactoring, but now I don't have an idea what can be a better 
replacement for it.