Menu
▾
▴
matplotlib-devel
|
From: Gael V. <gae...@no...> - 2008-06-08 20:58:33
|
Hey guys,
Just got back from 3 weeks holydays (that feels really good, I should try
this more often). I a fighting with a mountain of emails, but I just
wanted to give a little heads up. Tout is working on the codebase that I
originally wrote and got me addicted to the ETS, at the university of
Toronto.
He tried updating and did run in some minor incompatibility (some ".api"
added, not the end of the word, I believe).
However, more working (Darren Dale is Cced about that) is that matplotib
includes its own version of Traits and tends to be quite light on the
policy to decide when to overide the system one. As a result, and recent
version of MPL make the code base die in the ETS parts. I have already
mentioned this problem, and I believe it is a really evil one. I don't
have time to do some lobbying about this right now, could someone make
sure this is solved (Darren?).
IMHO one temporary solution, less ugly then the current one, is the one
we used in nipy: define a matplotlib.externals.traits that can point either to a
system-wide traits, or to the embedded traits. Using some code in
matplotlib.externals similar to:
"""
def import_traits():
""" Import traits, either from a system-wide location, or from our
copy.
"""
try:
from enthought import traits
except ImportError:
from matplotlib.externals.enthought import traits
return traits
traits = import_traits()
"""
You can put as much logics as you want in "import_traits" to check eg for
version numbers. If in MPL you only import traits from matplotlib.traits,
you can thus use traits and not have side effects on other libraries.
IMHO, the current situation is untenable.
Cheers,
Gaël
On Tue, Jun 03, 2008 at 01:51:40PM -0500, Dave Peterson wrote:
> Tout Wang wrote:
> That worked to resolve the ImportError for View but now it gets stuck
> at ui. I think ui is still imported from enthought.traits.ui because
> from enthought.traits.ui import ui
> works just fine. It seems that the main problem is that the latest
> version of Enthought has changed how things are imported, which is a
> little annoying because I expected that newer versions would be almost
> entirely backwards compatible. I wonder why this is?
> Anyway, I have reverted to Enthon 1.0.0 (Python 2.4.3) and everything
> imports just fine now from TraitsUI with the original code.
> Enthon 1.0.0 was released in August of 2006. At that time (about 7,000
> svn revisions ago!) Traits was reporting itself as version 1.0.2. EPD
> includes enthought.traits 2.0.4 as you've already seen. Even though our
> policy on version numbers didn't come into being until well after Enthon
> 1.0.0 was released, I think it reasonable that there would be some
> API-incompatible changes between version 1 and version 2.
> IIRC, the changes between these versions are more about adding features
> than they are about breaking or leaving behind functionality. Yes, the
> locations of where to import some things may have changed (centralized to
> an api.py rather than in __init__.py) but in general most everything else
> should work pretty much the same.
|
|
From: Darren D. <dar...@co...> - 2008-06-08 22:13:29
|
On Sunday 08 June 2008 16:57:56 Gael Varoquaux wrote: > Hey guys, > > Just got back from 3 weeks holydays (that feels really good, I should try > this more often). I a fighting with a mountain of emails, but I just > wanted to give a little heads up. Tout is working on the codebase that I > originally wrote and got me addicted to the ETS, at the university of > Toronto. > > He tried updating and did run in some minor incompatibility (some ".api" > added, not the end of the word, I believe). > > However, more working (Darren Dale is Cced about that) is that matplotib > includes its own version of Traits and tends to be quite light on the > policy to decide when to overide the system one. As a result, and recent > version of MPL make the code base die in the ETS parts. I have already > mentioned this problem, and I believe it is a really evil one. I don't > have time to do some lobbying about this right now, could someone make > sure this is solved (Darren?). Matplotlib's setup scripts are designed to avoid this problem. There are three conditions under which we install traits: 1) Traits is not installed 2) A previous version of traits is installed, but it is a version installed by matplotlib. I added an "-mpl" to the end of traits __version__ string so we can keep track. 3) The user explicitly askes for it in setup.cfg So if matplotlib is overwriting traits when it should not, I want to fix it. But I need more information about what is causing it, because I don't see how it could happen. > IMHO one temporary solution, less ugly then the current one, is the one > we used in nipy: define a matplotlib.externals.traits that can point either > to a system-wide traits, or to the embedded traits. Using some code in > matplotlib.externals similar to: > > """ > def import_traits(): > """ Import traits, either from a system-wide location, or from our > copy. > """ > try: > from enthought import traits > except ImportError: > from matplotlib.externals.enthought import traits > return traits > > traits = import_traits() > """ > > You can put as much logics as you want in "import_traits" to check eg for > version numbers. If in MPL you only import traits from matplotlib.traits, > you can thus use traits and not have side effects on other libraries. I tried this when I first started working with TConfig, and concluded that it was not possible because there are too many places where traits expects enthought to be a top level package. There were all kind of errors, exceptions being raised that were not named as expected, extension code that would need to be modified, so we settled on the current solution. > IMHO, the current situation is untenable. Please, provide details so I can understand the problem. Darren |
|
From: Gael V. <gae...@no...> - 2008-06-09 05:28:19
|
On Sun, Jun 08, 2008 at 06:13:24PM -0400, Darren Dale wrote:
> Matplotlib's setup scripts are designed to avoid this problem. There are three
> conditions under which we install traits:
> 1) Traits is not installed
> 2) A previous version of traits is installed, but it is a version installed by
> matplotlib. I added an "-mpl" to the end of traits __version__ string so we
> can keep track.
> 3) The user explicitly askes for it in setup.cfg
> So if matplotlib is overwriting traits when it should not, I want to fix it.
> But I need more information about what is causing it, because I don't see how
> it could happen.
If Traits is installed after MPL, if I get it right, then the problem
occurs. IIRC, this is the problem I stumbled upon once.
> I tried this when I first started working with TConfig, and concluded
> that it was not possible because there are too many places where traits
> expects enthought to be a top level package. There were all kind of
> errors, exceptions being raised that were not named as expected,
> extension code that would need to be modified, so we settled on the
> current solution.
OK. You are right. In nipy I modified enthought.traits and
enthought.etsconfig. This was not a beautiful job, I must admit. Maybe
monkey patching sys.path is the option (it is the way eggs do it) thought
I must admit I hate it, because it puts a lot of magic, that will
take the user by surprise. Anyway, the criteria for monkey-patching
sys.path must be improved, I feel.
If I understand things correctly, the current problem can be described
by:
* User has an old version of ETS, (the one in Enthon 1.0.4, that is a
very old one), his code needs the old version.
* User installs a new version of MPL.
* His code stops working.
I agree that as you describe things, this should not happen. Maybe I have
gotten wrong the order in which the user did things. I have the feeling
things shouldn't be dependent on the order in which you do the steps
(maybe the test for monkey-patching sys.path should not be at
install time, but at load time).
Maybe the test fails for a very old version of traits. I wanted to have a
quick look at this code, but I can't find it after a quick scan of the
MPL source, and I can't devote much time to this right now.
Cheers,
Gaël
|
|
From: Darren D. <dar...@co...> - 2008-06-09 10:41:30
|
On Monday 09 June 2008 01:27:57 Gael Varoquaux wrote: > On Sun, Jun 08, 2008 at 06:13:24PM -0400, Darren Dale wrote: > > Matplotlib's setup scripts are designed to avoid this problem. There are > > three conditions under which we install traits: > > > > 1) Traits is not installed > > 2) A previous version of traits is installed, but it is a version > > installed by matplotlib. I added an "-mpl" to the end of traits > > __version__ string so we can keep track. > > 3) The user explicitly askes for it in setup.cfg > > > > So if matplotlib is overwriting traits when it should not, I want to fix > > it. But I need more information about what is causing it, because I don't > > see how it could happen. > > If Traits is installed after MPL, if I get it right, then the problem > occurs. IIRC, this is the problem I stumbled upon once. I don't see why that would cause a problem. > > I tried this when I first started working with TConfig, and concluded > > that it was not possible because there are too many places where traits > > expects enthought to be a top level package. There were all kind of > > errors, exceptions being raised that were not named as expected, > > extension code that would need to be modified, so we settled on the > > current solution. > > OK. You are right. In nipy I modified enthought.traits and > enthought.etsconfig. This was not a beautiful job, I must admit. Maybe > monkey patching sys.path is the option (it is the way eggs do it) thought > I must admit I hate it, because it puts a lot of magic, that will > take the user by surprise. Anyway, the criteria for monkey-patching > sys.path must be improved, I feel. I also considered modifying sys.patch, this solution was roundly booed. > If I understand things correctly, the current problem can be described > by: > > * User has an old version of ETS, (the one in Enthon 1.0.4, that is a > very old one), his code needs the old version. > * User installs a new version of MPL. > * His code stops working. > > I agree that as you describe things, this should not happen. Maybe I have > gotten wrong the order in which the user did things. I have the feeling > things shouldn't be dependent on the order in which you do the steps > (maybe the test for monkey-patching sys.path should not be at > install time, but at load time). What test? I don't follow. > Maybe the test fails for a very old version of traits. I wanted to have a > quick look at this code, but I can't find it after a quick scan of the > MPL source, and I can't devote much time to this right now. Its a short section in setupext.py (which is in the same directory a setup.py), the function is called check_provide_traits. I think we would rather make traits an external dependency, if it could be easily installed as a separate package by a novice python user. Would it be possible for http://code.enthought.com/projects/traits/ to list specific instructions and links to the downloads? Or to list traits on the Python Package Index? |
|
From: John H. <jd...@gm...> - 2008-06-09 13:52:49
|
On Mon, Jun 9, 2008 at 5:41 AM, Darren Dale <dar...@co...> wrote: > I think we would rather make traits an external dependency, if it could be > easily installed as a separate package by a novice python user. Would it be > possible for http://code.enthought.com/projects/traits/ to list specific > instructions and links to the downloads? Or to list traits on the Python > Package Index? I also agree that we should not install a modified version of traits in the top level namespace. Let's disable all default installs of enthought traits until we work this out. We only need it for the optional rc config and so let's require that someone manually turn it on in setup.cfg if they want it. In particular, let's fix this before the next bugfix release. If we opt to depend on traits for the new rc, we will either need to require and external dependency, figure out a way to install a completely working and compatible version, or do the extra work to install it inside the matplotlib namespace. I prefer 1 or 3, but at this point I think whether we will migrate to the traited rc system is an open question so disabling is the right solution for now. Darren, can you take care of this? JDH |
|
From: Darren D. <dar...@co...> - 2008-06-09 16:05:06
|
On Monday 09 June 2008 09:52:46 am John Hunter wrote: > On Mon, Jun 9, 2008 at 5:41 AM, Darren Dale <dar...@co...> wrote: > > I think we would rather make traits an external dependency, if it could > > be easily installed as a separate package by a novice python user. Would > > it be possible for http://code.enthought.com/projects/traits/ to list > > specific instructions and links to the downloads? Or to list traits on > > the Python Package Index? > > I also agree that we should not install a modified version of traits > in the top level namespace. Let's disable all default installs of > enthought traits until we work this out. We only need it for the > optional rc config and so let's require that someone manually turn it > on in setup.cfg if they want it. In particular, let's fix this before > the next bugfix release. If we opt to depend on traits for the new > rc, we will either need to require and external dependency, figure out > a way to install a completely working and compatible version, or do > the extra work to install it inside the matplotlib namespace. I > prefer 1 or 3, but at this point I think whether we will migrate to > the traited rc system is an open question so disabling is the right > solution for now. > > Darren, can you take care of this? Its done. Gael, maybe the following situation caused the trouble: 1) user downloads mpl source 2) builds matplotlib - traits now exists in the temproary build directory 3) installs enthought traits 4) installs matplotlib - traits would not have been built in this case, its already installed on the system, but it still exists in the build directory and gets installed anyway. Seems unlikely, but who knows. |
|
From: Gael V. <gae...@no...> - 2008-06-09 16:14:02
|
On Mon, Jun 09, 2008 at 12:04:58PM -0400, Darren Dale wrote: > Gael, maybe the following situation caused the trouble: > 1) user downloads mpl source > 2) builds matplotlib - traits now exists in the temproary build directory > 3) installs enthought traits > 4) installs matplotlib - traits would not have been built in this case, its > already installed on the system, but it still exists in the build directory > and gets installed anyway. Actually, after looking at the code and thinking a bit more, I think the problem might simply be with different version of traits installed in different directories in the sys.path, with Python's import mechanism picking up the MPL-installed one, rather then the user-installed one. Tricky problems that I see more and more happenning. But I don't have an example box to test this hypothesis, so we are still clueless. Cheers, Gaël |
|
From: Dave P. <dpe...@en...> - 2008-06-09 18:55:37
|
Gael Varoquaux wrote: > On Mon, Jun 09, 2008 at 12:04:58PM -0400, Darren Dale wrote: > >> Gael, maybe the following situation caused the trouble: >> > > >> 1) user downloads mpl source >> 2) builds matplotlib - traits now exists in the temproary build directory >> 3) installs enthought traits >> 4) installs matplotlib - traits would not have been built in this case, its >> already installed on the system, but it still exists in the build directory >> and gets installed anyway. >> > > Actually, after looking at the code and thinking a bit more, I think the > problem might simply be with different version of traits installed in > different directories in the sys.path, with Python's import mechanism > picking up the MPL-installed one, rather then the user-installed one. > Tricky problems that I see more and more happenning. > > But I don't have an example box to test this hypothesis, so we are still > clueless. > In my experience, the current method really only falls down for those trying to publish binary distributions who don't realize the potential consequences of this build-time detection, or that there is even a decision being made there about the content of what is being built. Say, perhaps those building EPD. :-) Luckily, once you know what is going on during the build, it's a pretty easy problem to solve. That said, given the upcoming release of Traits 3 this situation may get a little crazier. T2 and T3 are not fully api compatible, though they are very close. So I suspect version numbers are going to play a larger role in the future. Is there anything we can do in the T3 release to make resolution of this upcoming issue easier to deal with for the matplotlib team? One point probably worth mentioning is that, IIRC, we currently rely on T3 being installed with egg meta-data in order to determine an accurate version number. -- Dave |
|
From: Gael V. <gae...@no...> - 2008-06-09 16:16:36
|
On Mon, Jun 09, 2008 at 06:41:08AM -0400, Darren Dale wrote: > I think we would rather make traits an external dependency, if it could be > easily installed as a separate package by a novice python user. Would it be > possible for http://code.enthought.com/projects/traits/ to list specific > instructions and links to the downloads? Or to list traits on the Python > Package Index? If I get things correctly, having traits on PyPI is planned for the version 3 release, which should happen when the guys at Enthought find time to make it happen. However I cannot speak for them, as I am not employed by Enthought, sitting in Austin (Yet !). I will also be very happy the day this happens. Gaël |
|
From: Dave P. <dpe...@en...> - 2008-06-09 18:46:34
|
Darren Dale wrote: > I think we would rather make traits an external dependency, if it could be > easily installed as a separate package by a novice python user. Would it be > possible for http://code.enthought.com/projects/traits/ to list specific > instructions and links to the downloads? Or to list traits on the Python > Package Index? > We have now scheduled resources to finish the ETS refactoring that will allow us to release Traits 3, which means publishing it into PyPI. The effort won't start until next week at the earliest though. Even though I'm a bit gun-shy about doing this, I'd forecast we have Traits 3.0 officially released and on PyPI by the end of June, perhaps a week into July. -- Dave |
|
From: Eric F. <ef...@ha...> - 2008-06-09 19:22:47
|
Dave Peterson wrote: > That said, given the upcoming release of Traits 3 this situation may get > a little crazier. T2 and T3 are not fully api compatible, though they > are very close. So I suspect version numbers are going to play a > larger role in the future. Is there anything we can do in the T3 > release to make resolution of this upcoming issue easier to deal with > for the matplotlib team? One point probably worth mentioning is that, > IIRC, we currently rely on T3 being installed with egg meta-data in > order to determine an accurate version number. Whether, or to what extent, mpl starts depending on traits is still open; but if we do depend on it, I think we should simply require T3 as an external dependency. If that requires some slight modifications of Darren's code, which was written for T2, I expect the changes will be easy. Three questions: 1) To what extent would the range of T3 eggs cover the various platforms on which people run mpl? 2) For uncovered cases, should T3 be easy to build and install? 3) My recollection is that setuptools was determined to be causing a hit to the startup time, and mpl is already sluggish in starting up. Is there any more insight or progress on this front? Is there a way to use traits in mpl without increasing the startup time? I gather from the lists that we are still not out of the woods with respect to making mpl (and scipy, even more) easy to install, and ideally easy to build from svn, for nearly all users; I would not want to see us start depending on traits if this will make the present bad situation worse. Thanks. Eric |
|
From: Dave P. <dpe...@en...> - 2008-06-09 20:04:54
|
Eric Firing wrote: > Dave Peterson wrote: > >> That said, given the upcoming release of Traits 3 this situation may >> get a little crazier. T2 and T3 are not fully api compatible, though >> they are very close. So I suspect version numbers are going to play >> a larger role in the future. Is there anything we can do in the T3 >> release to make resolution of this upcoming issue easier to deal with >> for the matplotlib team? One point probably worth mentioning is >> that, IIRC, we currently rely on T3 being installed with egg >> meta-data in order to determine an accurate version number. > > Whether, or to what extent, mpl starts depending on traits is still > open; but if we do depend on it, I think we should simply require T3 > as an external dependency. If that requires some slight modifications > of Darren's code, which was written for T2, I expect the changes will > be easy. > > Three questions: > > 1) To what extent would the range of T3 eggs cover the various > platforms on which people run mpl? Not quite sure on this one as I don't know what platforms are most used by mpl. What I can say is that we've worked very hard to minimize the dependencies Traits has on other things in order to make it as easy as possible for people to install. We'll definitely be uploading a source tarball, which should meet most people's needs, and a Windows binary (since not all users there have a c compiler.) We may or may not put up OS X, and a couple linux distribution, binaries. > 2) For uncovered cases, should T3 be easy to build and install? T3 proper needs a c compiler, gcc seems to work fine. TraitsGUI and the backend projects seem to be pure-python though clearly you'll need libs for the chosen backend. > 3) My recollection is that setuptools was determined to be causing a > hit to the startup time, and mpl is already sluggish in starting up. > Is there any more insight or progress on this front? Is there a way > to use traits in mpl without increasing the startup time? I'm not sure it was setuptools' egg features that were the problem so much as I thought it was the use of namespace packaging we have embedded all over in ETS. I don't see any significant efforts underway at this time that are trying to speed this up, but perhaps I'm just uninformed about any such efforts. I don't see anything on the horizon that would let us remove this from the ETS projects either. The end result is I don't see any way mpl could work around this and still treat Traits as an external dependency. -- Dave |
|
From: Darren D. <dar...@co...> - 2008-06-09 21:19:16
|
On Monday 09 June 2008 04:04:47 pm Dave Peterson wrote: > Eric Firing wrote: > > Dave Peterson wrote: > >> That said, given the upcoming release of Traits 3 this situation may > >> get a little crazier. T2 and T3 are not fully api compatible, though > >> they are very close. So I suspect version numbers are going to play > >> a larger role in the future. Is there anything we can do in the T3 > >> release to make resolution of this upcoming issue easier to deal with > >> for the matplotlib team? One point probably worth mentioning is > >> that, IIRC, we currently rely on T3 being installed with egg > >> meta-data in order to determine an accurate version number. > > > > Whether, or to what extent, mpl starts depending on traits is still > > open; but if we do depend on it, I think we should simply require T3 > > as an external dependency. If that requires some slight modifications > > of Darren's code, which was written for T2, I expect the changes will > > be easy. I think T2->T3 would not be a difficult transition for us. It may not even require any modifications, I seem to remember the traited config stuff just worked with traits3 last time I tried it, I just dont remember when I tried it last. > > Three questions: > > > > 1) To what extent would the range of T3 eggs cover the various > > platforms on which people run mpl? > > Not quite sure on this one as I don't know what platforms are most used > by mpl. What I can say is that we've worked very hard to minimize the > dependencies Traits has on other things in order to make it as easy as > possible for people to install. We'll definitely be uploading a source > tarball, which should meet most people's needs, and a Windows binary > (since not all users there have a c compiler.) We may or may not put up > OS X, and a couple linux distribution, binaries. > > > 2) For uncovered cases, should T3 be easy to build and install? > > T3 proper needs a c compiler, gcc seems to work fine. TraitsGUI and the > backend projects seem to be pure-python though clearly you'll need libs > for the chosen backend. > > > 3) My recollection is that setuptools was determined to be causing a > > hit to the startup time, and mpl is already sluggish in starting up. > > Is there any more insight or progress on this front? Is there a way > > to use traits in mpl without increasing the startup time? > > I'm not sure it was setuptools' egg features that were the problem so > much as I thought it was the use of namespace packaging we have embedded > all over in ETS. I don't see any significant efforts underway at this > time that are trying to speed this up, but perhaps I'm just uninformed > about any such efforts. I don't see anything on the horizon that would > let us remove this from the ETS projects either. The end result is I > don't see any way mpl could work around this and still treat Traits as > an external dependency. Fernando saw a big performance hit due to namespace packaging, but I never saw it. I think we concluded that the presence of large NFS mounts were causing the performance hit in namespace packages that Fernando reported at Scipy last year (Dave, was it you who worked with him?). I did some profiling a while back also, after we added traits to the mpl codebase and stripped out the namespace packaging. We still saw something like a 30% performance hit, which I tracked back to regular expressions being compiled in traits and in configobj. See: http://thread.gmane.org/gmane.comp.python.enthought.devel/10908/focus=10989 Darren |
|
From: Dave P. <dpe...@en...> - 2008-06-09 22:22:35
|
Darren Dale wrote: > On Monday 09 June 2008 04:04:47 pm Dave Peterson wrote: > >> Eric Firing wrote: >> >>> Dave Peterson wrote: >>> >>>> That said, given the upcoming release of Traits 3 this situation may >>>> get a little crazier. T2 and T3 are not fully api compatible, though >>>> they are very close. So I suspect version numbers are going to play >>>> a larger role in the future. Is there anything we can do in the T3 >>>> release to make resolution of this upcoming issue easier to deal with >>>> for the matplotlib team? One point probably worth mentioning is >>>> that, IIRC, we currently rely on T3 being installed with egg >>>> meta-data in order to determine an accurate version number. >>>> >>> Whether, or to what extent, mpl starts depending on traits is still >>> open; but if we do depend on it, I think we should simply require T3 >>> as an external dependency. If that requires some slight modifications >>> of Darren's code, which was written for T2, I expect the changes will >>> be easy. >>> > > I think T2->T3 would not be a difficult transition for us. It may not even > require any modifications, I seem to remember the traited config stuff just > worked with traits3 last time I tried it, I just dont remember when I tried > it last. > Yes, the API breaks are not large at all. They're actually fairly small, the bulk of the effort on T3 was behind the scenes and added functionality. But there are some there so it may not be clear sailing for everyone. >>> Three questions: >>> >>> 1) To what extent would the range of T3 eggs cover the various >>> platforms on which people run mpl? >>> >> Not quite sure on this one as I don't know what platforms are most used >> by mpl. What I can say is that we've worked very hard to minimize the >> dependencies Traits has on other things in order to make it as easy as >> possible for people to install. We'll definitely be uploading a source >> tarball, which should meet most people's needs, and a Windows binary >> (since not all users there have a c compiler.) We may or may not put up >> OS X, and a couple linux distribution, binaries. >> >> >>> 2) For uncovered cases, should T3 be easy to build and install? >>> >> T3 proper needs a c compiler, gcc seems to work fine. TraitsGUI and the >> backend projects seem to be pure-python though clearly you'll need libs >> for the chosen backend. >> >> >>> 3) My recollection is that setuptools was determined to be causing a >>> hit to the startup time, and mpl is already sluggish in starting up. >>> Is there any more insight or progress on this front? Is there a way >>> to use traits in mpl without increasing the startup time? >>> >> I'm not sure it was setuptools' egg features that were the problem so >> much as I thought it was the use of namespace packaging we have embedded >> all over in ETS. I don't see any significant efforts underway at this >> time that are trying to speed this up, but perhaps I'm just uninformed >> about any such efforts. I don't see anything on the horizon that would >> let us remove this from the ETS projects either. The end result is I >> don't see any way mpl could work around this and still treat Traits as >> an external dependency. >> > > Fernando saw a big performance hit due to namespace packaging, but I never saw > it. I think we concluded that the presence of large NFS mounts were causing > the performance hit in namespace packages that Fernando reported at Scipy > last year (Dave, was it you who worked with him?). > Yes, he and I sat together in a sprint room at Scipy and refactored Traits code (and everything else using setuptools and namespace packages) until we removed it all, taking timing runs each step of the way. I'd actually be surprised if Fernando was using NSF mounts during this effort, but I don't remember asking. I do remember him pointing out how bad it *would* be for anyone using network drives to suffer through some of the API call counts we saw when using namespace packages. > I did some profiling a while back also, after we added traits to the mpl > codebase and stripped out the namespace packaging. We still saw something > like a 30% performance hit, which I tracked back to regular expressions being > compiled in traits and in configobj. See: > http://thread.gmane.org/gmane.comp.python.enthought.devel/10908/focus=10989 > I didn't intend to start up the root cause discussion again. :-) I only wanted to point out that I hadn't heard of anyone working on improving whatever performance problems exist / existed. -- Dave > Darren > > |
|
From: Darren D. <dar...@co...> - 2008-06-09 22:35:02
|
On Monday 09 June 2008 6:22:28 pm Dave Peterson wrote: > I didn't intend to start up the root cause discussion again. :-) I only > wanted to point out that I hadn't heard of anyone working on improving > whatever performance problems exist / existed. I just wanted to avoid having a misperception turn into conventional wisdom. We see that enough in political discourse. |
|
From: Fernando P. <fpe...@gm...> - 2008-06-09 23:06:28
|
On Mon, Jun 9, 2008 at 3:22 PM, Dave Peterson <dpe...@en...> wrote: > Yes, he and I sat together in a sprint room at Scipy and refactored Traits > code (and everything else using setuptools and namespace packages) until we > removed it all, taking timing runs each step of the way. I'd actually be > surprised if Fernando was using NSF mounts during this effort, but I don't > remember asking. I do remember him pointing out how bad it *would* be for > anyone using network drives to suffer through some of the API call counts we > saw when using namespace packages. You are correct: I was NOT using an NFS mount, it was all done on my (slow) laptop. We were seeing enormous numbers (tens of thousands) of disc/directory access calls that I was very worried about if said disk was on NFS. Part of this was due to the fact that back at work in Colorado, I *did* have my $HOME on NFS and I'd recalled seeing some slowness here and there, but I never went back to repeat our profiling exercise at work, so I can't provide numbers on that. Cheers, f |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X