python-markdown-discuss — A general purpose discussion list

Re: [Python-markdown-discuss] Markdown encoding

[Yuri T. <qar...@gm...>]

I haven't had a chance to look at the specific problem, but in
general, here how it is _supposed_ to work.

The Markdown class is unicode-in-unicode-out.   It can take a simple
string as input, but one should never pass an encoded string to it, be
it utf8 or whatever.
It's the callers responsibility to decode their text into unicode from
utf8 or whatever it is that they have it encoded as, and they can then
encode the output into whatever encoding they want.  Then I got a
patch for removing BOM and integrated it without thinking, which
required passing "encoding" to it.  Looking at it now I realize that
that was quite stupid.  Since removeBOM() should never get encoded
strings, should _assume_ that the input is unicode, so presumably it
should suffice to have:

    def removeBOM(text, encoding):
         return text.lstrip(u'\ufeff')

In fact, we should just get rid of this function and put
text.lstrip(u'\ufeff') in the place where it is called.  (BTW, should
we put it back into the output?)

Again, if you are using markdown as a module, you should decode your
content yourself, run it through md.convert(), and then use the
resulting unicode as you wish:

     input_file =3D codecs.open("test.txt", mode=3D"r", encoding=3D"utf16")
     text =3D input_file.read()
     html_unicode =3D Markdown.markdown(text, extensions)
     output_file =3D codecs.open("test.html", "w", encoding=3D"utf8")
     output_file.write(html_unicode)

Perhaps we should raise an error if we get an encoded string?  I.e.,
check that either the string is of type unicode _or_ it has no special
characters.

Markdown.markdown does have an obvious bug in that it accepts an
encoding argument and doesn't pass it to Markdown.__init__.  I suppose
we should just get of this parameter altogether.

There is also another utility function - markdownFromFile.  This one
does the encoding and decoding for you.  For simplicity, it uses only
one encoding argument, which is used for both decoding the input and
encoding output.  I suppose that this might be confusing.  Should we
add an extra argument "output_encoding" making it optional?  I.e.:

    def markdownFromFile(input =3D None,
                         output =3D None,
                         extensions =3D [],
                         encoding =3D None,
                         output_encoding =3D None,
                         message_threshold =3D CRITICAL,
                         safe =3D False) :
        if not output_encoding:
           output encoding =3D encoding

I must admit here that I just went to look at the documentation on the
wiki and am realizing that that's what is responsible for much of the
confusion.  We have a new wiki at http://markdown.freewisdom.org/ and
I am slowly moving content there.  In particular, I copied over the
content of http://markdown.freewisdom.org/Using_as_a_Module and
updated it with the example above.

We should perhaps create a page called "BOMs" to archive there the
design decisions related to BOM removal, etc.

  - yuri


On 10/30/07, Waylan Limberg <wa...@gm...> wrote:
> Kent, thanks for the info. We'll look at this further.
>
> On 10/30/07, Kent Johnson <ke...@td...> wrote:
> > Waylan Limberg wrote:
> > > Kent,
> > >
> > > Could you verify that revision 46 fixes the problem for you?
> >
> > It will fix my problem but it won't work correctly with all unicode
> > text. For example if the original text contains a BOM and it is
> > converted with utf-16be or utf-16le encoding then the unicode string
> > still contains a BOM which will not be removed by this patch.
>
> My testing shows this works with utf-16. Could you provide a simple test =
case?
>
> >
> > Also it still seems a bit strange that the encoding argument to
> > markdown() is not used at all and the encoding argument to
> > Markdown.__init__() is the encoding that the data was in *before* it wa=
s
> > converted to unicode.
> >
> > I would write removeBOM() as
> >
> >   def removeBOM(text, encoding):
> >       if isinstance(text, unicode):
> >           boms =3D [u'\ufeff']
> >       else:
> >           boms =3D BOMS[encoding]
> >       for bom in boms:
> >           if text.startswith(bom):
> >               return text.lstrip(bom)
> >       return text
> >
> > and I would change the rest of the code to use encoding=3DNone when the
> > text is actually unicode.
> >
> > Kent
> >
> > >
> > > We can thank the very smart Malcolm Tredinnick for providing a patch.
> > > See bug report [1817528] for more.
> > >
> > > On 9/12/07, Kent Johnson <ke...@td...> wrote:
> > >> Hi,
> > >>
> > >> Markdown 1.6b doesn't work with UTF-8-encoded text. It fails with a
> > >> UnicodeDecodeError in removeBOM():
> > >>
> > >> In [3]: import markdown
> > >> In [4]: text =3D u'\xe2'.encode('utf-8')
> > >> In [6]: print text
> > >> =E2
> > >> In [7]: print markdown.markdown(text)
> > >> ------------------------------------------------------------
> > >> Traceback (most recent call last):
> > >>    File "<ipython console>", line 1, in <module>
> > >>    File
> > >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/sit=
e-packages/markdown.py",
> > >> line 1722, in markdown
> > >>      return md.convert(text)
> > >>    File
> > >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/sit=
e-packages/markdown.py",
> > >> line 1614, in convert
> > >>      self.source =3D removeBOM(self.source, self.encoding)
> > >>    File
> > >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/sit=
e-packages/markdown.py",
> > >> line 74, in removeBOM
> > >>      if text.startswith(bom):
> > >> <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode b=
yte
> > >> 0xc3 in position 0: ordinal not in range(128)
> > >>
> > >> The problem is that the BOM being tested is unicode so to execute
> > >>    text.startswith(bom)
> > >> Python tries to convert text to Unicode using the default encoding
> > >> (ascii). This fails because the text is not ascii.
> > >>
> > >> I'm trying to understand what the encoding parameter is for; it does=
n't
> > >> seem to do much. There also seems to be some confusion with the use =
of
> > >> encoding in markdownFromFile() vs markdown(); the file is converted =
to
> > >> Unicode on input so I don't understand why the same encoding paramet=
er
> > >> is passed to markdown()?
> > >>
> > >> ISTM the encoding passed to markdown should match the encoding of th=
e
> > >> text passed to markdown, and the values in the BOMS table should be =
in
> > >> the encoding of the key, not in unicode. Then the __unicode__() meth=
od
> > >> should actually decode. Or is the intent that the text passed to
> > >> markdown() should always be ascii or unicode?
> > >>
> > >> I can put together a patch if you like but I wanted to make sure tha=
t I
> > >> am not missing some grand plan...
> > >>
> > >> Kent
> > >>
> > >> --------------------------------------------------------------------=
-----
> > >> This SF.net email is sponsored by: Microsoft
> > >> Defy all challenges. Microsoft(R) Visual Studio 2005.
> > >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > >> _______________________________________________
> > >> Python-markdown-discuss mailing list
> > >> Pyt...@li...
> > >> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> > >>
> > >
> > >
> >
> >
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


--=20
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] Markdown encoding

[Waylan L. <wa...@gm...>]

Kent, thanks for the info. We'll look at this further.

On 10/30/07, Kent Johnson <ke...@td...> wrote:
> Waylan Limberg wrote:
> > Kent,
> >
> > Could you verify that revision 46 fixes the problem for you?
>
> It will fix my problem but it won't work correctly with all unicode
> text. For example if the original text contains a BOM and it is
> converted with utf-16be or utf-16le encoding then the unicode string
> still contains a BOM which will not be removed by this patch.

My testing shows this works with utf-16. Could you provide a simple test ca=
se?

>
> Also it still seems a bit strange that the encoding argument to
> markdown() is not used at all and the encoding argument to
> Markdown.__init__() is the encoding that the data was in *before* it was
> converted to unicode.
>
> I would write removeBOM() as
>
>   def removeBOM(text, encoding):
>       if isinstance(text, unicode):
>           boms =3D [u'\ufeff']
>       else:
>           boms =3D BOMS[encoding]
>       for bom in boms:
>           if text.startswith(bom):
>               return text.lstrip(bom)
>       return text
>
> and I would change the rest of the code to use encoding=3DNone when the
> text is actually unicode.
>
> Kent
>
> >
> > We can thank the very smart Malcolm Tredinnick for providing a patch.
> > See bug report [1817528] for more.
> >
> > On 9/12/07, Kent Johnson <ke...@td...> wrote:
> >> Hi,
> >>
> >> Markdown 1.6b doesn't work with UTF-8-encoded text. It fails with a
> >> UnicodeDecodeError in removeBOM():
> >>
> >> In [3]: import markdown
> >> In [4]: text =3D u'\xe2'.encode('utf-8')
> >> In [6]: print text
> >> =E2
> >> In [7]: print markdown.markdown(text)
> >> ------------------------------------------------------------
> >> Traceback (most recent call last):
> >>    File "<ipython console>", line 1, in <module>
> >>    File
> >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-=
packages/markdown.py",
> >> line 1722, in markdown
> >>      return md.convert(text)
> >>    File
> >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-=
packages/markdown.py",
> >> line 1614, in convert
> >>      self.source =3D removeBOM(self.source, self.encoding)
> >>    File
> >> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-=
packages/markdown.py",
> >> line 74, in removeBOM
> >>      if text.startswith(bom):
> >> <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byt=
e
> >> 0xc3 in position 0: ordinal not in range(128)
> >>
> >> The problem is that the BOM being tested is unicode so to execute
> >>    text.startswith(bom)
> >> Python tries to convert text to Unicode using the default encoding
> >> (ascii). This fails because the text is not ascii.
> >>
> >> I'm trying to understand what the encoding parameter is for; it doesn'=
t
> >> seem to do much. There also seems to be some confusion with the use of
> >> encoding in markdownFromFile() vs markdown(); the file is converted to
> >> Unicode on input so I don't understand why the same encoding parameter
> >> is passed to markdown()?
> >>
> >> ISTM the encoding passed to markdown should match the encoding of the
> >> text passed to markdown, and the values in the BOMS table should be in
> >> the encoding of the key, not in unicode. Then the __unicode__() method
> >> should actually decode. Or is the intent that the text passed to
> >> markdown() should always be ascii or unicode?
> >>
> >> I can put together a patch if you like but I wanted to make sure that =
I
> >> am not missing some grand plan...
> >>
> >> Kent
> >>
> >> ----------------------------------------------------------------------=
---
> >> This SF.net email is sponsored by: Microsoft
> >> Defy all challenges. Microsoft(R) Visual Studio 2005.
> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> >> _______________________________________________
> >> Python-markdown-discuss mailing list
> >> Pyt...@li...
> >> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> >>
> >
> >
>
>


--=20
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Re-organization of tracker.

[Anand <ana...@gm...>]

On 30-Oct-07, at 9:55 AM, Waylan Limberg wrote:

> On 10/30/07, Yuri Takhteyev <yu...@cs...> wrote:
>>> The limitations of this site are rather annoying. It doesn't help  
>>> that
>>> it's painfully slow.
>>
>> Should we move?
>
> That would be fine by me.
>
>> I am not exactly a big fan of sourceforge myself and
>> would have moved earlier, if I had the opportunity to research the
>> alternatives.
>
> If I have some time, I'll do some research myself. If anyone else out
> there has some input, please do speak up.

http://launchpad.net is worth exploring.

Re: [Python-markdown-discuss] Re-organization of tracker.

[Herbert P. <her...@gm...>]

i've moved away from sourceforge a long time ago (for bug tracking)
because it was (and obviously is) extremely slow.. first i've used my
own implementation of a bug tracker.. but since i actually wanted to
track bugs, and not fix the bugtracker.. i eventually moved to google
code hosting .. the issue tracker is very minimalistic but still
powerful enough - http://code.google.com/p/sct-project/issues/list
(the only thing which is annoying to me is that users who are not part
of the project can't add 'Labels' and since everything is done by
labels (including the categories feature request, bug report, etc.) i
basically have to edit all incoming bug reports to add the correct
labels.. but it is very fast and all labels can be customized by the
project admin ..)

On 10/30/07, Waylan Limberg <wa...@gm...> wrote:
> On 10/30/07, Yuri Takhteyev <yu...@cs...> wrote:
> > > The limitations of this site are rather annoying. It doesn't help that
> > > it's painfully slow.
> >
> > Should we move?
>
> That would be fine by me.
>
> > I am not exactly a big fan of sourceforge myself and
> > would have moved earlier, if I had the opportunity to research the
> > alternatives.
>
> If I have some time, I'll do some research myself. If anyone else out
> there has some input, please do speak up.
>
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>

Re: [Python-markdown-discuss] Re-organization of tracker.

[Waylan L. <wa...@gm...>]

On 10/30/07, Yuri Takhteyev <yu...@cs...> wrote:
> > The limitations of this site are rather annoying. It doesn't help that
> > it's painfully slow.
>
> Should we move?

That would be fine by me.

> I am not exactly a big fan of sourceforge myself and
> would have moved earlier, if I had the opportunity to research the
> alternatives.

If I have some time, I'll do some research myself. If anyone else out
there has some input, please do speak up.



-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Re-organization of tracker.

[Yuri T. <yu...@cs...>]

> The limitations of this site are rather annoying. It doesn't help that
> it's painfully slow.

Should we move?  I am not exactly a big fan of sourceforge myself and
would have moved earlier, if I had the opportunity to research the
alternatives.

  - yuri

>
> [1]: http://sourceforge.net/tracker/?group_id=153041
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

[Python-markdown-discuss] Re-organization of tracker.

[Waylan L. <wa...@gm...>]

This is just an FYI that I have made some administrative changes to
the tracker on Sourceforge for Python-Markdown [1].  If you've
previously submitted a report and can't find it now, this may be of
interest to you.

Previously, the tracker included 4 sections (Bugs, Feature Requests,
Patches, and Support Requests). Given the small size of the project
and the low number of items in the tracker, having so many sections
doesn't make sense. Actually, I kept forgetting sections other than
"bugs" existed and tend to ignore them.

Therefore, all open non-bug items (all 2 of them) have been moved to
the "Bugs" section and all other sections have been hidden
(Sourceforge won't allow deleting). I'd rename the "Bugs" section to
something more generic but that's not possible either. I've considered
hiding the "Bugs" section as well and creating a new, generically
named section, but then only admins (Yuri and myself) could view the
old closed tickets. As there's no 'mass move to another section'
feature, I'm not manually moving all those old tickets (currently 36
plus 5 open). Perhaps I'll make this change after the next release.

Oh, and I also added a setting so that I am notified via email for
anything submitted to the tracker. Previously, I'd have to go check
the site manually which I may or may not remember regularly.
Unfortunately, I don't see a way to have more than one email address
notified so I couldn't add Yuri also. I'll try to keep him updated of
anything of interest.

The limitations of this site are rather annoying. It doesn't help that
it's painfully slow.

[1]: http://sourceforge.net/tracker/?group_id=153041

-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] infinite loop in python-markdown

[Waylan L. <wa...@gm...>]

FYI, I've applied this patch and tests in r47.

On 10/12/07, Anand <ana...@gm...> wrote:
>
> On 08-Oct-07, at 9:33 AM, Yuri Takhteyev wrote:
>
> > I found a fix for this particular
> > problem:
> >
> > replace
> >
> >         + (NOBRACKET + r'(\['+NOBRACKET)*6
> >         + (NOBRACKET+ r'\])*'+NOBRACKET)*6
> >
> > with:
> >
> >         + (NOBRACKET + r'(\[')*6
> >         + (NOBRACKET+ r'\])*')*6
> >
> > in the defintion of "BRK".  Which as far as my test suite shows does
> > not causes any problems (and really shouldn't, since the second
> > NOBRACKET was redundant in both cases).
>
> Thanks Yuri, that solved the problem.
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] Trac with Markdown

[Yuri T. <qar...@gm...>]

Sure, it does seem that they are not interested in supporting it.  But if
trac is designed well, it shouldn't be that hard to make this chance without
their support.

However, I spent way too much time today trying to install trac on dreamhost
and am ready to give up on it.  I then tried MoinMoin and remembered why I
stopped using it earlier.  I was going to try Infogami next, but the fact
that I would need to install Mercurial on dreamhost makes me suspect that
there would be a lot more of wasted time ahead.  Unless someone can suggest
some other option that I haven't considered yet, I think I'll just go and
fix the current wiki by installing a new version.

- yuri

On 10/18/07, Brian Jaress <bri...@gn...> wrote:
>
> That's right, it needs the curly braces.  I'm actually not using Trac
> right now, and I was never very familiar with the internals.
>
> One thing I remember is that the Trac devs seemed scornful of people
> wanting to get rid of the curly braces.
>
> A lot of users wanted that feature, so the devs may have changed their
> mind since then, but I doubt it.
>
>
> On Wed, Oct 17, 2007 at 02:18:04AM -0500, Yuri Takhteyev wrote:
> > It doesn't seem that ugly, once I managed to parse it. :)
> >
> > A question though: this being a "plugin" means that all Markdown
> formatting
> > would need to be surrounded by {{{...}}}, right?  Is there a way of
> avoiding
> > this?
> >
> > Judging by http://trac.edgewall.org/ticket/615, I am guessing that there
> > isn't any blessed solution.  But one would think that there should be a
> way
> > to hack it to default to Markdown?
> >
> >   - yuri
> >
> > On 10/16/07, Brian Jaress <bri...@gn...> wrote:
> > >
> > > A while back I did an ugly hack that mostly worked (but I haven't used
> > > it in a while).  I just uses a giant regular expresion to extract link
> > > targets, run them through the wiki engine, and then process the whole
> > > thing with markdown.py.
> > >
> > > There must be a better way.
> > >
> > >
> > > """Trac plugin for Markdown Syntax (with links)
> > >
> > > Everything markdown-ed as a link target is run through Trac's wiki
> > > formatter to get a substitute url.
> > >
> > > Tested with Trac 0.8.1 and python-markdown 1.4 on Debian GNU/Linux.
> > >
> > > Brian Jaress
> > > 2007-01-04
> > > """
> > >
> > > from re import sub, compile, search, I
> > > from markdown import markdown
> > > from trac.WikiFormatter import wiki_to_oneliner
> > >
> > > #links, autolinks, and reference-style links
> > > LINK = compile(
> > >     r'(\]\()([^) ]+)([^)]*\))|(<)([^>]+)(>)|(\n\[[^]]+\]: *)([^
> > > \n]+)(.*\n)'
> > > )
> > >
> > > HREF = compile(r'href=[\'"]?([^\'" ]*)')
> > >
> > > def execute(hdf, txt, env):
> > >     abs = env.abs_href.base
> > >     abs = abs[:len(abs) - len(env.href.base)]
> > >     def convert(m):
> > >         pre, target, suf = filter(None, m.groups())
> > >         url = search(
> > >                 HREF,
> > >                 wiki_to_oneliner(target, hdf, env, env.get_db_cnx()),
> > >                 I).groups()[0]
> > >         #Trac creates relative links, which markdown won't touch
> inside
> > >         # <autolinks> because they look like HTML
> > >         if pre == '<' and url != target:
> > >             pre += abs
> > >         return pre + str(url) + suf
> > >
> > >     return markdown(sub(LINK, convert, txt))
> > >
> > >
> > >
> > > On Tue, Oct 16, 2007 at 03:45:46PM -0500, Yuri Takhteyev wrote:
> > > > Has anyone gotten Trac working with Markdown?  What exactly does it
> > > take?
> > > >
> > > >   - yuri
> > > >
> > > > --
> > > > http://www.freewisdom.org/
> > >
> > > >
> > >
> -------------------------------------------------------------------------
> > > > This SF.net email is sponsored by: Splunk Inc.
> > > > Still grepping through log files to find problems?  Stop.
> > > > Now Search log events and configuration files using AJAX and a
> browser.
> > > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > > _______________________________________________
> > > > Python-markdown-discuss mailing list
> > > > Pyt...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> > >
> > >
> > > --
> > > Brian Jaress
> > > bri...@gn...
> > > http://brian-jaress.livejournal.com
> > >
> > >
> -------------------------------------------------------------------------
> > > This SF.net email is sponsored by: Splunk Inc.
> > > Still grepping through log files to find problems?  Stop.
> > > Now Search log events and configuration files using AJAX and a
> browser.
> > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > _______________________________________________
> > > Python-markdown-discuss mailing list
> > > Pyt...@li...
> > > https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> > >
> >
> >
> >
> > --
> > Yuri Takhteyev
> > Ph.D. Candidate, UC Berkeley School of Information
> > http://takhteyev.org/, http://www.freewisdom.org/
>
> --
> Brian Jaress
> bri...@gn...
> http://brian-jaress.livejournal.com
>



-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] Trac with Markdown

[Brian J. <bri...@gn...>]

That's right, it needs the curly braces.  I'm actually not using Trac
right now, and I was never very familiar with the internals.

One thing I remember is that the Trac devs seemed scornful of people
wanting to get rid of the curly braces.

A lot of users wanted that feature, so the devs may have changed their
mind since then, but I doubt it.


On Wed, Oct 17, 2007 at 02:18:04AM -0500, Yuri Takhteyev wrote:
> It doesn't seem that ugly, once I managed to parse it. :)
> 
> A question though: this being a "plugin" means that all Markdown formatting
> would need to be surrounded by {{{...}}}, right?  Is there a way of avoiding
> this?
> 
> Judging by http://trac.edgewall.org/ticket/615, I am guessing that there
> isn't any blessed solution.  But one would think that there should be a way
> to hack it to default to Markdown?
> 
>   - yuri
> 
> On 10/16/07, Brian Jaress <bri...@gn...> wrote:
> >
> > A while back I did an ugly hack that mostly worked (but I haven't used
> > it in a while).  I just uses a giant regular expresion to extract link
> > targets, run them through the wiki engine, and then process the whole
> > thing with markdown.py.
> >
> > There must be a better way.
> >
> >
> > """Trac plugin for Markdown Syntax (with links)
> >
> > Everything markdown-ed as a link target is run through Trac's wiki
> > formatter to get a substitute url.
> >
> > Tested with Trac 0.8.1 and python-markdown 1.4 on Debian GNU/Linux.
> >
> > Brian Jaress
> > 2007-01-04
> > """
> >
> > from re import sub, compile, search, I
> > from markdown import markdown
> > from trac.WikiFormatter import wiki_to_oneliner
> >
> > #links, autolinks, and reference-style links
> > LINK = compile(
> >     r'(\]\()([^) ]+)([^)]*\))|(<)([^>]+)(>)|(\n\[[^]]+\]: *)([^
> > \n]+)(.*\n)'
> > )
> >
> > HREF = compile(r'href=[\'"]?([^\'" ]*)')
> >
> > def execute(hdf, txt, env):
> >     abs = env.abs_href.base
> >     abs = abs[:len(abs) - len(env.href.base)]
> >     def convert(m):
> >         pre, target, suf = filter(None, m.groups())
> >         url = search(
> >                 HREF,
> >                 wiki_to_oneliner(target, hdf, env, env.get_db_cnx()),
> >                 I).groups()[0]
> >         #Trac creates relative links, which markdown won't touch inside
> >         # <autolinks> because they look like HTML
> >         if pre == '<' and url != target:
> >             pre += abs
> >         return pre + str(url) + suf
> >
> >     return markdown(sub(LINK, convert, txt))
> >
> >
> >
> > On Tue, Oct 16, 2007 at 03:45:46PM -0500, Yuri Takhteyev wrote:
> > > Has anyone gotten Trac working with Markdown?  What exactly does it
> > take?
> > >
> > >   - yuri
> > >
> > > --
> > > http://www.freewisdom.org/
> >
> > >
> > -------------------------------------------------------------------------
> > > This SF.net email is sponsored by: Splunk Inc.
> > > Still grepping through log files to find problems?  Stop.
> > > Now Search log events and configuration files using AJAX and a browser.
> > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > _______________________________________________
> > > Python-markdown-discuss mailing list
> > > Pyt...@li...
> > > https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> >
> >
> > --
> > Brian Jaress
> > bri...@gn...
> > http://brian-jaress.livejournal.com
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > _______________________________________________
> > Python-markdown-discuss mailing list
> > Pyt...@li...
> > https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
> >
> 
> 
> 
> -- 
> Yuri Takhteyev
> Ph.D. Candidate, UC Berkeley School of Information
> http://takhteyev.org/, http://www.freewisdom.org/

-- 
Brian Jaress
bri...@gn...
http://brian-jaress.livejournal.com

Re: [Python-markdown-discuss] Trac with Markdown

[Yuri T. <qar...@gm...>]

It doesn't seem that ugly, once I managed to parse it. :)

A question though: this being a "plugin" means that all Markdown formatting
would need to be surrounded by {{{...}}}, right?  Is there a way of avoiding
this?

Judging by http://trac.edgewall.org/ticket/615, I am guessing that there
isn't any blessed solution.  But one would think that there should be a way
to hack it to default to Markdown?

  - yuri

On 10/16/07, Brian Jaress <bri...@gn...> wrote:
>
> A while back I did an ugly hack that mostly worked (but I haven't used
> it in a while).  I just uses a giant regular expresion to extract link
> targets, run them through the wiki engine, and then process the whole
> thing with markdown.py.
>
> There must be a better way.
>
>
> """Trac plugin for Markdown Syntax (with links)
>
> Everything markdown-ed as a link target is run through Trac's wiki
> formatter to get a substitute url.
>
> Tested with Trac 0.8.1 and python-markdown 1.4 on Debian GNU/Linux.
>
> Brian Jaress
> 2007-01-04
> """
>
> from re import sub, compile, search, I
> from markdown import markdown
> from trac.WikiFormatter import wiki_to_oneliner
>
> #links, autolinks, and reference-style links
> LINK = compile(
>     r'(\]\()([^) ]+)([^)]*\))|(<)([^>]+)(>)|(\n\[[^]]+\]: *)([^
> \n]+)(.*\n)'
> )
>
> HREF = compile(r'href=[\'"]?([^\'" ]*)')
>
> def execute(hdf, txt, env):
>     abs = env.abs_href.base
>     abs = abs[:len(abs) - len(env.href.base)]
>     def convert(m):
>         pre, target, suf = filter(None, m.groups())
>         url = search(
>                 HREF,
>                 wiki_to_oneliner(target, hdf, env, env.get_db_cnx()),
>                 I).groups()[0]
>         #Trac creates relative links, which markdown won't touch inside
>         # <autolinks> because they look like HTML
>         if pre == '<' and url != target:
>             pre += abs
>         return pre + str(url) + suf
>
>     return markdown(sub(LINK, convert, txt))
>
>
>
> On Tue, Oct 16, 2007 at 03:45:46PM -0500, Yuri Takhteyev wrote:
> > Has anyone gotten Trac working with Markdown?  What exactly does it
> take?
> >
> >   - yuri
> >
> > --
> > http://www.freewisdom.org/
>
> >
> -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > _______________________________________________
> > Python-markdown-discuss mailing list
> > Pyt...@li...
> > https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>
>
> --
> Brian Jaress
> bri...@gn...
> http://brian-jaress.livejournal.com
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] Trac with Markdown

[Brian J. <bri...@gn...>]

A while back I did an ugly hack that mostly worked (but I haven't used
it in a while).  I just uses a giant regular expresion to extract link
targets, run them through the wiki engine, and then process the whole
thing with markdown.py.

There must be a better way.


"""Trac plugin for Markdown Syntax (with links)

Everything markdown-ed as a link target is run through Trac's wiki
formatter to get a substitute url.

Tested with Trac 0.8.1 and python-markdown 1.4 on Debian GNU/Linux.

Brian Jaress
2007-01-04
"""

from re import sub, compile, search, I
from markdown import markdown
from trac.WikiFormatter import wiki_to_oneliner

#links, autolinks, and reference-style links
LINK = compile(
    r'(\]\()([^) ]+)([^)]*\))|(<)([^>]+)(>)|(\n\[[^]]+\]: *)([^ \n]+)(.*\n)'
)

HREF = compile(r'href=[\'"]?([^\'" ]*)')

def execute(hdf, txt, env):
    abs = env.abs_href.base
    abs = abs[:len(abs) - len(env.href.base)]
    def convert(m):
        pre, target, suf = filter(None, m.groups())
        url = search(
                HREF,
                wiki_to_oneliner(target, hdf, env, env.get_db_cnx()),
                I).groups()[0]
        #Trac creates relative links, which markdown won't touch inside
        # <autolinks> because they look like HTML
        if pre == '<' and url != target:
            pre += abs
        return pre + str(url) + suf

    return markdown(sub(LINK, convert, txt))



On Tue, Oct 16, 2007 at 03:45:46PM -0500, Yuri Takhteyev wrote:
> Has anyone gotten Trac working with Markdown?  What exactly does it take?
> 
>   - yuri
> 
> -- 
> http://www.freewisdom.org/

> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss


-- 
Brian Jaress
bri...@gn...
http://brian-jaress.livejournal.com

[Python-markdown-discuss] Trac with Markdown

[Yuri T. <qar...@gm...>]

Has anyone gotten Trac working with Markdown?  What exactly does it take?

  - yuri

-- 
http://www.freewisdom.org/

Re: [Python-markdown-discuss] Herbert is your Yaar! :)

[Herbert P. <her...@gm...>]

waaaa .. sorry .. this was NOT initiated by myself ! i haven't visited this
damn site for 2 months and suddenly they start sending invitations to my
gmail contacts.. i have no idea why the hell they are doing this ..

sorry again, please ignore it !
  herbert

On 10/15/07, Herbert <her...@gm...> wrote:
>
>     Herbert
> Vienna
>  Herbert Poul wants you to join Yaari!
> Is Herbert your friend?
>  [image: Yes]<http://www.yaari.com/y-register.php?i=MTAPV10IL7EEB95B1188806929> [image:
> No] <http://www.yaari.com/y-register.php>
> Please respond or Herbert might think you said no :(
>
>
> To stop receiving emails from Yaari.com, click here<http://yaari.com/y-email-opt-out.php?param=cHl0aG9uLW1hcmtkb3duLWRpc2N1c3NAbGlzdHMuc291cmNlZm9yZ2UubmV0>.
>
> If you have any concerns regarding the content of this message, please
> email ab...@ya....
> Yaari LLC, 358 Angier Ave, Atlanta, GA 30312
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>
>

[Python-markdown-discuss] Herbert is your Yaar! :)

[Herbert <her...@gm...>]

CjxodG1sPgo8aGVhZD4KPC9oZWFkPgo8Ym9keT4KPHRhYmxlIGFsaWduPSJjZW50ZXIiIHdpZHRo
PSI1NTAiIGhlaWdodD0iMjQ2IgpiYWNrZ3JvdW5kPSJodHRwOi8veWFhcmkuY29tL3ktdmlyYWwv
aW1nL2JveC1hbGwuanBnIj48dHI+PHRkPgogIDxkaXYgaWQ9ImNvbnRhaW5lciIKc3R5bGU9InRl
eHQtYWxpZ246Y2VudGVyO2ZvbnQtZmFtaWx5OlZlcmRhbmEsQXJpYWw7Zm9udC13ZWlnaHQ6Ym9s
ZDtmb250LXNpemU6MTFwdDsiPgogICAgPHRhYmxlIHdpZHRoPSI5NyUiIGhlaWdodD0iOTAlIiBh
bGlnbj0iY2VudGVyIiBzdHlsZT0ibWFyZ2luLXRvcDoxMHB4OyI+CiAgICAgIDx0ciB2YWxpZ249
Im1pZGRsZSI+CiAgICAgICAgPHRkIHdpZHRoPSIyMCUiPgogICAgICAgICAgPGltZyBzcmM9Imh0
dHA6Ly93d3cueWFhcmkuY29tL3VzZXJfMTAwcHhwaWMvbm9faW1nX2JpZy5qcGciCnN0eWxlPSJk
aXNwbGF5OmJsb2NrO21hcmdpbjphdXRvO21hcmdpbi1ib3R0b206MTBweDtib3JkZXI6bm9uZTsi
IC8+CiAgICAgICAgICA8ZGl2IHN0eWxlPSJmb250LXNpemU6OXB0O3RleHQtYWxpZ246Y2VudGVy
OyI+SGVyYmVydDwvZGl2PgogICAgICAgICAgPGRpdiBzdHlsZT0iZm9udC1zaXplOjlwdDtjb2xv
cjojQjAwO3RleHQtYWxpZ246Y2VudGVyOyI+Vmllbm5hPC9kaXY+CiAgICAgICAgPC90ZD4KICAg
ICAgICA8dGQ+CiAgICAgICAgICA8ZGl2IHN0eWxlPSJ0ZXh0LWFsaWduOmNlbnRlcjsiPkhlcmJl
cnQgUG91bCB3YW50cyB5b3UgdG8gam9pbiBZYWFyaSE8L2Rpdj4KICAgICAgICAgIDxkaXYKc3R5
bGU9ImZvbnQtc2l6ZToxM3B0O21hcmdpbi10b3A6MzVweDttYXJnaW4tYm90dG9tOjEwcHg7dGV4
dC1hbGlnbjpjZW50ZXI7Zm9udC13ZWlnaHQ6Ym9sZDsiPklzCkhlcmJlcnQgeW91ciBmcmllbmQ/
PC9kaXY+CiAgICAgICAgICA8ZGl2IGFsaWduPSJjZW50ZXIiIHN0eWxlPSJtYXJnaW46MjVweDt0
ZXh0LWFsaWduOmNlbnRlcjsiPgogICAgICAgICAgICA8YSBocmVmPSJodHRwOi8vd3d3LnlhYXJp
LmNvbS95LXJlZ2lzdGVyLnBocD9pPU1UQVBWMTBJTDdFRUI5NUIxMTg4ODA2OTI5Ij4KICAgICAg
ICAgICAgICA8aW1nIHNyYz0iaHR0cDovL3lhYXJpLmNvbS95LXZpcmFsL2ltZy9idXR0b24teWVz
LmdpZiIgYWx0PSJZZXMiCnN0eWxlPSJib3JkZXI6bm9uZTttYXJnaW46MCA1cHg7Ii8+PC9hPgog
ICAgICAgICAgICA8YSBocmVmPSJodHRwOi8vd3d3LnlhYXJpLmNvbS95LXJlZ2lzdGVyLnBocCI+
CiAgICAgICAgICAgICAgPGltZyBzcmM9Imh0dHA6Ly95YWFyaS5jb20veS12aXJhbC9pbWcvYnV0
dG9uLW5vLmdpZiIgYWx0PSJObyIKc3R5bGU9ImJvcmRlcjpub25lO21hcmdpbjowIDVweDsiIC8+
PC9hPgogICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8ZGl2IHN0eWxlPSJmb250LXNpemU6OXB0
O3RleHQtYWxpZ246Y2VudGVyOyI+UGxlYXNlIHJlc3BvbmQgb3IgSGVyYmVydCBtaWdodCB0aGlu
ayB5b3UKc2FpZCBubyA6KDwvZGl2PgogICAgICAgIDwvdGQ+CiAgICAgIDwvdHI+CiAgICA8L3Rh
YmxlPgogIDwvZGl2Pgo8L3RkPjwvdHI+PC90YWJsZT4KPHAgYWxpZ249ImNlbnRlciIgc3R5bGU9
ImZvbnQtc2l6ZToxMHB0Ij4KICA8YnIgLz4KICBUbyBzdG9wIHJlY2VpdmluZyBlbWFpbHMgZnJv
bSBZYWFyaS5jb20sIGNsaWNrIDxhCmhyZWY9Imh0dHA6Ly95YWFyaS5jb20veS1lbWFpbC1vcHQt
b3V0LnBocD9wYXJhbT1jSGwwYUc5dUxXMWhjbXRrYjNkdUxXUnBjMk4xYzNOQWJHbHpkSE11YzI5
MWNtTmxabTl5WjJVdWJtVjAiPmhlcmU8L2E+LgogIDxiciAvPgogIElmIHlvdSBoYXZlIGFueSBj
b25jZXJucyByZWdhcmRpbmcgdGhlIGNvbnRlbnQgb2YgdGhpcyBtZXNzYWdlLCBwbGVhc2UgZW1h
aWwgYWJ1c2VAeWFhcmkuY29tLgogIDxiciAvPgogIDxzcGFuIHN0eWxlPSJmb250LXNpemU6OHB0
Ij5ZYWFyaSBMTEMsIDM1OCBBbmdpZXIgQXZlLCBBdGxhbnRhLCBHQSAKMzAzMTI8L3NwYW4+Cjwv
cD4KPC9ib2R5Pgo8L2h0bWw+Cg==

Re: [Python-markdown-discuss] Missing Simple Tables extension

[Andrej P. <ap...@gm...>]

The file is attached. I've added a license (same as Python Markdown)  
and a usage example in the doc string.

Cheers,
Andrej

Re: [Python-markdown-discuss] Missing Simple Tables extension

[Yuri T. <qar...@gm...>]

Ok, send me all the files and I will put it up on the site once I set
up the new wiki, which will hopefully be soon.

  - yuri

On 10/12/07, Andrej Primc <ap...@gm...> wrote:
> Hi Waylan,
>
> I'd be happy if you hosted it.
>
> (My site is down, and will remain so for a while.)
>
> Cheers,
> Andrej
>
> On Oct 9, 2007, at 7:18 PM, Waylan Limberg wrote:
>
> > I recently noticed that the "Simple Tables" extension listed in
> > Available Extensions [1] is no longer available at the provided link.
> > However, I did find it with the Way Back Machine here [2].
> >
> >
> > Andrej, would you mind is we hosted a copy on the project site? Or,
> > perhaps it's moved and you, like me, are waiting for Yuri to fix the
> > login so you can edit the link.
> >
> > [1]: http://www.freewisdom.org/projects/python-markdown/
> > Available_Extensions
> > [2]: http://web.archive.org/web/20070708205838rn_1/k7a.org/2006/oct/
> > 30/table-extension-python-markdown/
> > --
> > ----
> > Waylan Limberg
> > wa...@gm...
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] Missing Simple Tables extension

[Andrej P. <ap...@gm...>]

Hi Waylan,

I'd be happy if you hosted it.

(My site is down, and will remain so for a while.)

Cheers,
Andrej

On Oct 9, 2007, at 7:18 PM, Waylan Limberg wrote:

> I recently noticed that the "Simple Tables" extension listed in
> Available Extensions [1] is no longer available at the provided link.
> However, I did find it with the Way Back Machine here [2].
>
>
> Andrej, would you mind is we hosted a copy on the project site? Or,
> perhaps it's moved and you, like me, are waiting for Yuri to fix the
> login so you can edit the link.
>
> [1]: http://www.freewisdom.org/projects/python-markdown/ 
> Available_Extensions
> [2]: http://web.archive.org/web/20070708205838rn_1/k7a.org/2006/oct/ 
> 30/table-extension-python-markdown/
> -- 
> ----
> Waylan Limberg
> wa...@gm...

Re: [Python-markdown-discuss] infinite loop in python-markdown

[Anand <ana...@gm...>]

On 08-Oct-07, at 9:33 AM, Yuri Takhteyev wrote:

> I found a fix for this particular
> problem:
>
> replace
>
>         + (NOBRACKET + r'(\['+NOBRACKET)*6
>         + (NOBRACKET+ r'\])*'+NOBRACKET)*6
>
> with:
>
>         + (NOBRACKET + r'(\[')*6
>         + (NOBRACKET+ r'\])*')*6
>
> in the defintion of "BRK".  Which as far as my test suite shows does
> not causes any problems (and really shouldn't, since the second
> NOBRACKET was redundant in both cases).

Thanks Yuri, that solved the problem. 

Re: [Python-markdown-discuss] Proposed changes to linebreaks

[Yuri T. <qar...@gm...>]

Those changes all make sense to me.  The first and the last ones are
just plain bugs.  The second one doesn't make much difference in HTML,
but I suppose it would be cleaner if the extra spaces are removed.

  - yuri

On 10/10/07, Waylan Limberg <wa...@gm...> wrote:
> I've committed these changes in r41. I suspect item (1) would be the
> only item that would cause any real issues. If it's a problem, I have
> a workaround. It just seemed hackish and unnecessary so I left it out
> barring any complaints. Please let me know if any issues are
> encountered.
>
> On 10/9/07, Waylan Limberg <wa...@gm...> wrote:
> > I've been working on a few different bugs related to line breaks. In
> > doing so, I've found some questionable side effects of the current
> > implementation that do not always make sense or correspond with perl
> > or php variations. I realize that just because markdown.pl or php do
> > something a certain way, doesn't mean we should, but I think it makes
> > sense here.
> >
> > So my question is, would anyone have any objection to the following changes?
> >
> > (1) Currently, in python two spaces are converted to a linebreak at
> > the end of a *header*. Seeing headers are always one line, what value
> > does this add? Neither php nor perl support this. Would anyone miss
> > this if I removed it? (It would make an inlinePattern for linebreaks
> > **much** easier.)
> >
> > (2) I noticed that the two spaces are always preserved in output ("
> > <br />\n"). Is this necessary? For example (spaces replaced with +s
> > for clarity):
> >
> >     Line with a break.++
> >
> > currently becomes:
> >
> >     Line with a break.++<br />
> >
> > instead of:
> >
> >     Line with a break.<br />
> >
> > IMO this doesn't really matter one way or the other, but I'm getting
> > test failures over this and am inclined to strip the extra whitepsace
> > from the tests. Any thoughts?
> >
> > (1). Currently a blank line composed of at least two spaces becomes a
> > line break "<br />". This is not inline with either perl or php and
> > can have some interesting side effects. Note this example from
> > "tests/misc/some-tests.txt" (I've replaced the indentation with '+'s
> > for clarity):
> >
> >     * Python is
> >     ok
> >     ++++Or not?
> >     +++
> >     Here is a normal paragraph
> >
> >     1. Another list
> >
> > I would expect the output given by perl or php:
> >
> >     <ul>
> >     <li>Python is
> >     ok
> >     Or not?</li>
> >     </ul>
> >
> >     <p>Here is a normal paragraph</p>
> >
> >     <ol>
> >     <li>Another list</li>
> >     </ol>
> >
> > Instead python-markdown gives us:
> >
> >     <ul>
> >      <li><p>Python is
> >        ok
> >        Or not?
> >           <br />
> >        Here is a normal paragraph
> >     </p>
> >
> >      </li>
> >
> >      <li><p>Another list
> >     </p>
> >
> >      </li>
> >     </ul>
> >
> > Removing the 'blank line with spaces becomes a linebreak' rule
> > addresses this issue. Is there a legitimate use case for these lines
> > that I'm missing?
> >
> > If no one has any persuasive arguments for keeping the current
> > behaviour, I'll update the tests to reflect the changes and then
> > finish up my patches for the various linebreak bugs.
> >
> >
> > --
> > ----
> > Waylan Limberg
> > wa...@gm...
> >
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] Proposed changes to linebreaks

[Waylan L. <wa...@gm...>]

I've committed these changes in r41. I suspect item (1) would be the
only item that would cause any real issues. If it's a problem, I have
a workaround. It just seemed hackish and unnecessary so I left it out
barring any complaints. Please let me know if any issues are
encountered.

On 10/9/07, Waylan Limberg <wa...@gm...> wrote:
> I've been working on a few different bugs related to line breaks. In
> doing so, I've found some questionable side effects of the current
> implementation that do not always make sense or correspond with perl
> or php variations. I realize that just because markdown.pl or php do
> something a certain way, doesn't mean we should, but I think it makes
> sense here.
>
> So my question is, would anyone have any objection to the following changes?
>
> (1) Currently, in python two spaces are converted to a linebreak at
> the end of a *header*. Seeing headers are always one line, what value
> does this add? Neither php nor perl support this. Would anyone miss
> this if I removed it? (It would make an inlinePattern for linebreaks
> **much** easier.)
>
> (2) I noticed that the two spaces are always preserved in output ("
> <br />\n"). Is this necessary? For example (spaces replaced with +s
> for clarity):
>
>     Line with a break.++
>
> currently becomes:
>
>     Line with a break.++<br />
>
> instead of:
>
>     Line with a break.<br />
>
> IMO this doesn't really matter one way or the other, but I'm getting
> test failures over this and am inclined to strip the extra whitepsace
> from the tests. Any thoughts?
>
> (1). Currently a blank line composed of at least two spaces becomes a
> line break "<br />". This is not inline with either perl or php and
> can have some interesting side effects. Note this example from
> "tests/misc/some-tests.txt" (I've replaced the indentation with '+'s
> for clarity):
>
>     * Python is
>     ok
>     ++++Or not?
>     +++
>     Here is a normal paragraph
>
>     1. Another list
>
> I would expect the output given by perl or php:
>
>     <ul>
>     <li>Python is
>     ok
>     Or not?</li>
>     </ul>
>
>     <p>Here is a normal paragraph</p>
>
>     <ol>
>     <li>Another list</li>
>     </ol>
>
> Instead python-markdown gives us:
>
>     <ul>
>      <li><p>Python is
>        ok
>        Or not?
>           <br />
>        Here is a normal paragraph
>     </p>
>
>      </li>
>
>      <li><p>Another list
>     </p>
>
>      </li>
>     </ul>
>
> Removing the 'blank line with spaces becomes a linebreak' rule
> addresses this issue. Is there a legitimate use case for these lines
> that I'm missing?
>
> If no one has any persuasive arguments for keeping the current
> behaviour, I'll update the tests to reflect the changes and then
> finish up my patches for the various linebreak bugs.
>
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>


-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] Proposed changes to linebreaks

[Waylan L. <wa...@gm...>]

I've been working on a few different bugs related to line breaks. In
doing so, I've found some questionable side effects of the current
implementation that do not always make sense or correspond with perl
or php variations. I realize that just because markdown.pl or php do
something a certain way, doesn't mean we should, but I think it makes
sense here.

So my question is, would anyone have any objection to the following changes?

(1) Currently, in python two spaces are converted to a linebreak at
the end of a *header*. Seeing headers are always one line, what value
does this add? Neither php nor perl support this. Would anyone miss
this if I removed it? (It would make an inlinePattern for linebreaks
**much** easier.)

(2) I noticed that the two spaces are always preserved in output ("
<br />\n"). Is this necessary? For example (spaces replaced with +s
for clarity):

    Line with a break.++

currently becomes:

    Line with a break.++<br />

instead of:

    Line with a break.<br />

IMO this doesn't really matter one way or the other, but I'm getting
test failures over this and am inclined to strip the extra whitepsace
from the tests. Any thoughts?

(1). Currently a blank line composed of at least two spaces becomes a
line break "<br />". This is not inline with either perl or php and
can have some interesting side effects. Note this example from
"tests/misc/some-tests.txt" (I've replaced the indentation with '+'s
for clarity):

    * Python is
    ok
    ++++Or not?
    +++
    Here is a normal paragraph

    1. Another list

I would expect the output given by perl or php:

    <ul>
    <li>Python is
    ok
    Or not?</li>
    </ul>

    <p>Here is a normal paragraph</p>

    <ol>
    <li>Another list</li>
    </ol>

Instead python-markdown gives us:

    <ul>
     <li><p>Python is
       ok
       Or not?
          <br />
       Here is a normal paragraph
    </p>

     </li>

     <li><p>Another list
    </p>

     </li>
    </ul>

Removing the 'blank line with spaces becomes a linebreak' rule
addresses this issue. Is there a legitimate use case for these lines
that I'm missing?

If no one has any persuasive arguments for keeping the current
behaviour, I'll update the tests to reflect the changes and then
finish up my patches for the various linebreak bugs.


-- 
----
Waylan Limberg
wa...@gm...

[Python-markdown-discuss] Missing Simple Tables extension

[Waylan L. <wa...@gm...>]

I recently noticed that the "Simple Tables" extension listed in
Available Extensions [1] is no longer available at the provided link.
However, I did find it with the Way Back Machine here [2].


Andrej, would you mind is we hosted a copy on the project site? Or,
perhaps it's moved and you, like me, are waiting for Yuri to fix the
login so you can edit the link.

[1]: http://www.freewisdom.org/projects/python-markdown/Available_Extensions
[2]: http://web.archive.org/web/20070708205838rn_1/k7a.org/2006/oct/30/table-extension-python-markdown/
-- 
----
Waylan Limberg
wa...@gm...

Re: [Python-markdown-discuss] differance between svn and 1.6b

[Yuri T. <qar...@gm...>]

No good reason, and no good excuse either :(  Sorry about this.

Yes, please merge it.

  - yuri

On 10/7/07, Waylan Limberg <wa...@gm...> wrote:
> I recently noticed that some of the changes in 1.6b never made it to
> the svn repo. A few were simple, like the HRs and BRs being stripped
> in safemode and the whitespace issues affecting a codeblock on the
> first line. I've since committed patches to those issues.
>
> However, the big differance is the addition of textPreprocessors with
> the HTML_BLOCK_PREPROCESSOR being moved there from the preprocessors.
>
> Is there any reason this never made it to the repo? Would you like me
> to merge it in?
>
> --
> ----
> Waylan Limberg
> wa...@gm...
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>


-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/

Re: [Python-markdown-discuss] infinite loop in python-markdown

[Yuri T. <qar...@gm...>]

Aaron and Anand found a serious bug with one of the regular
expressions that causes python markdown to hang on certain.  This
email is to announce a tentative solution and to check if anyone has
better ideas.

The attached script.py is the test case that Aaron and Anand sent in;
script2.py is my variation on it just includes the relevant parts of
markdown.py.  The regular expression in question is supposed to catch
patterns like [title][link_id].  The catch is that it supposed to
allow for balanced parentheses in the title part, i.e., one should be
able to write [one [[[[[two]]]]] three][link_id]

To the best of my knowledge, python's regular expressions do not offer
support balanced parentheses, so markdown.py handles this somewhat
with a rather ugly regular expression (see script2.py) which catches
matching brackets up two 6 levels deep.  The resulting expressing is
quite big, so it is perhaps not too surprising that this causes
problems on long paragraph.   I found a fix for this particular
problem:

replace

        + (NOBRACKET + r'(\['+NOBRACKET)*6
        + (NOBRACKET+ r'\])*'+NOBRACKET)*6

with:

        + (NOBRACKET + r'(\[')*6
        + (NOBRACKET+ r'\])*')*6

in the defintion of "BRK".  Which as far as my test suite shows does
not causes any problems (and really shouldn't, since the second
NOBRACKET was redundant in both cases).

However, I thought I would use this as an opportunity to check if
anyone has any suggestions for a better way to handle this.

Does this warrant a new release?

  - yuri

-- 
Yuri Takhteyev
Ph.D. Candidate, UC Berkeley School of Information
http://takhteyev.org/, http://www.freewisdom.org/