python-markdown-discuss

[Python-markdown-discuss] Changes to make it easier to bundle markdown

[Toshio K. <a.b...@gm...>]

Hi,  I've been working on getting python-webhelpers copy of markdown updated
to a more recent version and found that the cyclic imports in the code were
making it hard to do so.  Would you take a patch to break the cyclic import?


The current cycles work like this:

  import markdown => imports (blockparser, blockprocessors, inlinparser, etc)
    => import markdown

When markdown is a subpackage, this fails:

  >>> from webhelpers import markdown
  Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "webhelpers/markdown/__init__.py", line 161, in <module>
      import preprocessors
  File "webhelpers/markdown/preprocessors.py", line 11, in <module>
      import webhelpers.markdown as markdown
  AttributeError: 'module' object has no attribute 'markdown'

Moving a portion of the code from markdown into some new files breaks the
cyclic import and allows this to work:

  import markdown => imports blockparser => imports misc

I have a patch here that seems to work:
http://gitorious.org/python-markdown/toshios-mainline/commit/b50560edc707241b236f9d6c3f33edcf09d7cf33

If the idea behind this seems acceptable, I'll propose it for merging.

-Toshio

Re: [Python-markdown-discuss] Changes to make it easier to bundle markdown

[Waylan L. <wa...@gm...>]

On Mon, Jul 5, 2010 at 2:43 PM, Toshio Kuratomi <a.b...@gm...> wrote:
[snip]
> I have a patch here that seems to work:
> http://gitorious.org/python-markdown/toshios-mainline/commit/b50560edc707241b236f9d6c3f33edcf09d7cf33
>

Thanks for the feedback.  We are always looking for ways to make
python-markdown better.

I'm not so sure about your move of things to misc.  Personally, I
wonder why not "util", but that's just bike shedding.

What really concerns me is that there are a number of third party
extensions which import these things.  Many of these extensions have
never been published publicly and we have promised that they will
continue to work.  So my question is, will these changes only require
a small adjustment in an import line, or more work.  And if they do
require just the change in the import line, will the extension only be
able to work on either one version of markdown or the other?

In fact, a number of the things you've moved to misc, I had intended
to most to a util file.  But I never got to it before the release of
2.0 and now I've considered us stuck with things the way they are for
the above reasons.  However, if you can convince me that that will not
be a problem, then I will gladly merge such a change.

However, regarding the global variables.  Most of those do not need to
be global variables, but properties on an instance of the Markdown
class.  In fact, this will also make it easier for users and/or
extensions to override with their own settings. I've been meaning to
do this.  However, the few that do need to be global variables, I
don't see why they can't stay where they are.

Personally, I find it strange that you would need to ship markdown
inside your package.  Why can't markdown be a separate library in it's
own namespace? That way users can upgrade to newer versions without
waiting for you to. Sure, I realize you may be offering wrappers and
want to offer a compatibility guarantee, but I'd personally prefer the
option of stepping outside that if I wanted. But maybe that's just me.

-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] Changes to make it easier to bundle markdown

[Toshio K. <a.b...@gm...>]

On Mon, Jul 05, 2010 at 09:47:59PM -0400, Waylan Limberg wrote:
> On Mon, Jul 5, 2010 at 2:43 PM, Toshio Kuratomi <a.b...@gm...> wrote:
> [snip]
> > I have a patch here that seems to work:
> > http://gitorious.org/python-markdown/toshios-mainline/commit/b50560edc707241b236f9d6c3f33edcf09d7cf33
> >
> 
> Thanks for the feedback.  We are always looking for ways to make
> python-markdown better.
> 
> I'm not so sure about your move of things to misc.  Personally, I
> wonder why not "util", but that's just bike shedding.
> 
Sure, that's not an issue at all :-)
Would you also like misc_logging renamed to something like logging_util.py?

> What really concerns me is that there are a number of third party
> extensions which import these things.  Many of these extensions have
> never been published publicly and we have promised that they will
> continue to work.  So my question is, will these changes only require
> a small adjustment in an import line, or more work.  And if they do
> require just the change in the import line, will the extension only be
> able to work on either one version of markdown or the other?
> 
I think we can make this compatible by adding two lines in my changes.
In __init__.py:

  from misc import *
  from misc_logging import *

This doesn't reintroduce a cyclic dependency because we're still doing
importing things in misc into __init__.py; not importing things in
__init__.py into other parts of the package.

> In fact, a number of the things you've moved to misc, I had intended
> to most to a util file.  But I never got to it before the release of
> 2.0 and now I've considered us stuck with things the way they are for
> the above reasons.  However, if you can convince me that that will not
> be a problem, then I will gladly merge such a change.
> 
> However, regarding the global variables.  Most of those do not need to
> be global variables, but properties on an instance of the Markdown
> class.  In fact, this will also make it easier for users and/or
> extensions to override with their own settings. I've been meaning to
> do this.  However, the few that do need to be global variables, I
> don't see why they can't stay where they are.
> 
I do agree that setting things as class variables would be better :-)

> Personally, I find it strange that you would need to ship markdown
> inside your package.  Why can't markdown be a separate library in it's
> own namespace? That way users can upgrade to newer versions without
> waiting for you to. Sure, I realize you may be offering wrappers and
> want to offer a compatibility guarantee, but I'd personally prefer the
> option of stepping outside that if I wanted. But maybe that's just me.
> 
Oh, I definitely agree here :-)  I'm a packager for Fedora Linux and we're
constantly fighting against bundled libraries for a huge variety of
troublesome issues.  What I hear from upstreams, though, is that they want
to offer users something that works out of the box.  For them that means
shipping with a bundled version of the libraries they depend on.

So what I generally end up having to do is allow upstream to keep their
bundled copy but change their imports so that they first try to import from
a spearate library and if that's not found, fallback to the version that
they have bundled.  This is what I've been attempting to do with webhelpers
(which currently bundles 1.7.0).

-Toshio

Re: [Python-markdown-discuss] Changes to make it easier to bundle markdown

[Toshio K. <a.b...@gm...>]

On Mon, Jul 05, 2010 at 09:47:59PM -0400, Waylan Limberg wrote:
> On Mon, Jul 5, 2010 at 2:43 PM, Toshio Kuratomi <a.b...@gm...> wrote:
> [snip]
> > I have a patch here that seems to work:
> > http://gitorious.org/python-markdown/toshios-mainline/commit/b50560edc707241b236f9d6c3f33edcf09d7cf33
> >
> 
> Thanks for the feedback.  We are always looking for ways to make
> python-markdown better.
> 
> I'm not so sure about your move of things to misc.  Personally, I
> wonder why not "util", but that's just bike shedding.
>
Branch updated to use "util".  I also noticed that I hadn't committed the
files where I moved the global variables.  They're in there now.

Made a merge request but I can still update it if you want something else
added like renaming misc_logging.py to logging_util.py.

http://gitorious.org/python-markdown/mainline/merge_requests/6

-Toshio