Menu
▾
▴
matplotlib-devel
|
From: Nicolas G. <nic...@ga...> - 2007-02-12 00:14:39
|
Hello all, When you are working on matplotlib, after a checkout from the SVN repository, which method do you use to compile extensions without re-building and re-installing everything each time? For now, I use this method: - I checkout matplotlib in directory c:\projects\matplotlib. - I add c:\projects\matplotlib\lib to my PYTHONPATH. - I use the command "python setup.py build_ext --inplace" each time I modify a C/C++ file. - Without anything else, I can use matplotlib in my scripts. The problem: This method builds smoothly my extensions (.pyd files) alongside my Python files (.py files). But I have no directory c:\projects\matplotlib\lib\matplotlib\mpl-data, like in a standard installation. As a workaround, I've created it manually by copying files from directory fonts and images. My question: Matplotlib's developers, how do you proceed on your own computer? Thanks a lot for your answers, Nicolas Grilly |
|
From: John H. <jd...@gm...> - 2007-02-12 16:33:51
|
On 2/11/07, Nicolas Grilly <nic...@ga...> wrote: > When you are working on matplotlib, after a checkout from the SVN > repository, which method do you use to compile extensions without > re-building and re-installing everything each time? I rebuild and reinstall each time. Since only the changed files are built, this usually goes fairly quickly after a svn update. If you ware working on extension code, it can get slow if you are building the numpy, Numeric and numarray extensions each time. You can manually override the NUMERIX list in setup.py to just build one of the array extensions while developing, which should cut your compile times. JDH |
|
From: <jk...@ik...> - 2007-02-12 16:54:10
|
"Nicolas Grilly" <nic...@ga...> writes: > When you are working on matplotlib, after a checkout from the SVN > repository, which method do you use to compile extensions without > re-building and re-installing everything each time? I use "python setupegg.py develop", which I think does the same thing as build_ext --inplace. I have made a symbolic link for the mpl-data subdirectory. (Perhaps some setuptools wizard can tell us how to fix setupegg.py so that mpl-data gets installed in-place?) -- Jouni K. Seppänen http://www.iki.fi/jks |
|
From: Andrew S. <str...@as...> - 2007-02-12 18:10:53
|
Jouni K. Seppänen wrote: > "Nicolas Grilly" <nic...@ga...> writes: > > >> When you are working on matplotlib, after a checkout from the SVN >> repository, which method do you use to compile extensions without >> re-building and re-installing everything each time? >> > > I use "python setupegg.py develop", which I think does the same thing > as build_ext --inplace. I have made a symbolic link for the mpl-data > subdirectory. (Perhaps some setuptools wizard can tell us how to fix > setupegg.py so that mpl-data gets installed in-place?) IMO, the way to fix it is to move all the data into lib/matplotlib/mpl-data to start with. If we did that, all the shuffling that setup.py does would be unnecessary and using setuptools' develop mode would work out-of-the-box. I'd be happy to move these files in the svn repository if people agree this is a good idea. -Andrew |
|
From: John H. <jd...@gm...> - 2007-02-12 18:29:12
|
On 2/12/07, Andrew Straw <str...@as...> wrote: > IMO, the way to fix it is to move all the data into > lib/matplotlib/mpl-data to start with. If we did that, all the shuffling > that setup.py does would be unnecessary and using setuptools' develop > mode would work out-of-the-box. I'd be happy to move these files in the > svn repository if people agree this is a good idea. I'm +1 on that. For some reason, I thought we had already decided to do that when we moved to support setuptool in the first place. JDH |
|
From: Charlie M. <cw...@gm...> - 2007-02-12 18:35:02
|
On 2/12/07, John Hunter <jd...@gm...> wrote: > On 2/12/07, Andrew Straw <str...@as...> wrote: > > IMO, the way to fix it is to move all the data into > > lib/matplotlib/mpl-data to start with. If we did that, all the shuffling > > that setup.py does would be unnecessary and using setuptools' develop > > mode would work out-of-the-box. I'd be happy to move these files in the > > svn repository if people agree this is a good idea. > > I'm +1 on that. For some reason, I thought we had already decided to > do that when we moved to support setuptool in the first place. You're changing positions on me now, John! ;) I asked about moving everything into the module, but you didn't want to mix the fonts/images/etc, so I made the install merge the data into a folder. I would love to be able to run "setup develop" as well. - Charlie |
|
From: John H. <jd...@gm...> - 2007-02-12 18:43:36
|
On 2/12/07, Charlie Moad <cw...@gm...> wrote: > You're changing positions on me now, John! ;) It's better that way -- that way I can be right both times :-) Or is it wrong both times? hmmm... > I asked about moving everything into the module, but you didn't want > to mix the fonts/images/etc, so I made the install merge the data into > a folder. I would love to be able to run "setup develop" as well. OK, I see, so the problem was we had to flatten everything and mix all these datatypes together? eggs can't handle the existing directory structure in the mpl-data? It would clearly be better if it could, because dumping all the data files together isn't terribly elegant, so I think it would be useful for some egghead to verify this and see if there isn't a way to have the best of all worlds. I haven't used eggs so can't say anything intelligent, except that I'm happy to defer to those who know more about it and let's get it setup that that work properly with mpl-data for eggs and and regular old-fashioned mpl installs. If this means that we do have to flatten the data directory structure, so be it. That's why god gave us 'ls *.EXT ' JDH |
|
From: Andrew S. <str...@as...> - 2007-02-12 19:03:04
|
John Hunter wrote: > On 2/12/07, Charlie Moad <cw...@gm...> wrote: > >> You're changing positions on me now, John! ;) > > It's better that way -- that way I can be right both times :-) Or is > it wrong both times? hmmm... > >> I asked about moving everything into the module, but you didn't want >> to mix the fonts/images/etc, so I made the install merge the data into >> a folder. I would love to be able to run "setup develop" as well. > > OK, I see, so the problem was we had to flatten everything and mix all > these datatypes together? eggs can't handle the existing directory > structure in the mpl-data? It would clearly be better if it could, > because dumping all the data files together isn't terribly elegant, so > I think it would be useful for some egghead to verify this and see if > there isn't a way to have the best of all worlds. I haven't used eggs > so can't say anything intelligent, except that I'm happy to defer to > those who know more about it and let's get it setup that that work > properly with mpl-data for eggs and and regular old-fashioned mpl > installs. If this means that we do have to flatten the data directory > structure, so be it. That's why god gave us 'ls *.EXT ' Well, another option is to maintain a sub-directory structure when creating mpl-data in the source package. The downside is that this will introduce code changes all over the code -- for example, code that previously asked for "os.path.join( mpl.get_data_dir(), 'blah.glade' )" would have to be converted to "os.path.join( mpl.get_data_dir(), 'gui', 'blah.glade' )". So, the low energy, low probability-of-breakage approach is indeed to pile everything into mpl-data and make judicious use of "ls *.ext". This is what we install in site-packages anyway, so it can't be that bad, right? :) -Andrew |
|
From: Charlie M. <cw...@gm...> - 2007-02-12 20:42:21
|
> Well, another option is to maintain a sub-directory structure when > creating mpl-data in the source package. The downside is that this will > introduce code changes all over the code -- for example, code that > previously asked for "os.path.join( mpl.get_data_dir(), 'blah.glade' )" > would have to be converted to "os.path.join( mpl.get_data_dir(), 'gui', > 'blah.glade' )". I think we should maintain a clean tree and make the code changes. I bet it would only take one person a diligent hour or two to fix all the occurrences as mentioned above. Maybe we could split up the work? I would be willing to update the "backends" folder. - Charlie |
|
From: Eric F. <ef...@ha...> - 2007-02-12 21:03:52
|
Charlie Moad wrote: >> Well, another option is to maintain a sub-directory structure when >> creating mpl-data in the source package. The downside is that this will >> introduce code changes all over the code -- for example, code that >> previously asked for "os.path.join( mpl.get_data_dir(), 'blah.glade' )" >> would have to be converted to "os.path.join( mpl.get_data_dir(), 'gui', >> 'blah.glade' )". > > I think we should maintain a clean tree and make the code changes. I > bet it would only take one person a diligent hour or two to fix all > the occurrences as mentioned above. Maybe we could split up the work? > I would be willing to update the "backends" folder. Although my understanding of setup* is minimal, I agree; I think that keeping some organization in the data will be helpful. It looks like get_data_path() is not called in many places, so if that is essentially what has to be fixed, it should not be very difficult. It might be facilitated by making _get_data_path() accept *args, a set of subdirectories to be appended. Eric |
|
From: Charlie M. <cw...@gm...> - 2007-02-12 21:38:20
|
> Although my understanding of setup* is minimal, I agree; I think that > keeping some organization in the data will be helpful. It looks like > get_data_path() is not called in many places, so if that is essentially > what has to be fixed, it should not be very difficult. It might be > facilitated by making _get_data_path() accept *args, a set of > subdirectories to be appended. I think the way to go is what Andrew typed: os.path.join( mpl.get_data_dir(), 'gui', 'blah.glade' ) |
|
From: Andrew S. <str...@as...> - 2007-02-12 22:39:04
|
Charlie Moad wrote: >> Although my understanding of setup* is minimal, I agree; I think that >> keeping some organization in the data will be helpful. It looks like >> get_data_path() is not called in many places, so if that is essentially >> what has to be fixed, it should not be very difficult. It might be >> facilitated by making _get_data_path() accept *args, a set of >> subdirectories to be appended. >> > > I think the way to go is what Andrew typed: > > os.path.join( mpl.get_data_dir(), 'gui', 'blah.glade' ) > OK, I just implemented this nested idea and it appears to work. I tried to commit it to SVN, but SF's servers were giving me some talk about permissions... Sigh. So, I sent the patch file to John, who'll hopefully commit it if he approves. To paraphrase Robert Kern, "Working code ends all arguments". So, since this was the nested way, and the biggest reason against it was that it require code change, well, I changed the code, and perhaps the arguments are ended? Because I did change a fair amount of data_path -using code, it'd be good to exercise all your favorite font and image finding sections of code. The bits I tried worked for me, of course, but no guarantees... |
|
From: Charlie M. <cw...@gm...> - 2007-02-12 22:55:05
|
Thanks for tackling this. On 2/12/07, Andrew Straw <str...@as...> wrote: > Charlie Moad wrote: > >> Although my understanding of setup* is minimal, I agree; I think that > >> keeping some organization in the data will be helpful. It looks like > >> get_data_path() is not called in many places, so if that is essentially > >> what has to be fixed, it should not be very difficult. It might be > >> facilitated by making _get_data_path() accept *args, a set of > >> subdirectories to be appended. > >> > > > > I think the way to go is what Andrew typed: > > > > os.path.join( mpl.get_data_dir(), 'gui', 'blah.glade' ) > > > OK, I just implemented this nested idea and it appears to work. > > I tried to commit it to SVN, but SF's servers were giving me some talk > about permissions... Sigh. > > So, I sent the patch file to John, who'll hopefully commit it if he > approves. To paraphrase Robert Kern, "Working code ends all arguments". > So, since this was the nested way, and the biggest reason against it was > that it require code change, well, I changed the code, and perhaps the > arguments are ended? > > Because I did change a fair amount of data_path -using code, it'd be > good to exercise all your favorite font and image finding sections of > code. The bits I tried worked for me, of course, but no guarantees... > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier. > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > |
|
From: John H. <jd...@gm...> - 2007-02-12 23:18:20
|
On 2/12/07, Andrew Straw <str...@as...> wrote: > So, I sent the patch file to John, who'll hopefully commit it if he > approves. To paraphrase Robert Kern, "Working code ends all arguments". > So, since this was the nested way, and the biggest reason against it was > that it require code change, well, I changed the code, and perhaps the > arguments are ended? > > Because I did change a fair amount of data_path -using code, it'd be > good to exercise all your favorite font and image finding sections of > code. The bits I tried worked for me, of course, but no guarantees... OK, with a sledgehammer and a prayer, I got it in. I got the same error you did, and then had to physically remove mpl-data and it's contents from svn and commit, restore everything in the new order and commit again. It compiled, installed and ran, passing the tests I threw at it. I only ran about half of backend driver before I grew impatient, so others may want to do some additional testing. I copied your note from CHANGELOG to API_CHANGES. I suggest flushing your old mpl-data in site-packages/matplotlib as well as your ~/.matplotlib/ttffont.cache file before testing. Thanks for settling the argument! JDH |
|
From: Nicolas G. <nic...@ga...> - 2007-02-12 19:06:52
|
On 2/12/07, John Hunter <jd...@gm...> wrote: > OK, I see, so the problem was we had to flatten everything and mix all > these datatypes together? eggs can't handle the existing directory > structure in the mpl-data? It would clearly be better if it could, > because dumping all the data files together isn't terribly elegant, so > I think it would be useful for some egghead to verify this and see if > there isn't a way to have the best of all worlds. I haven't used eggs > so can't say anything intelligent, except that I'm happy to defer to > those who know more about it and let's get it setup that that work > properly with mpl-data for eggs and and regular old-fashioned mpl > installs. If this means that we do have to flatten the data directory > structure, so be it. That's why god gave us 'ls *.EXT ' I think this should work even with distutils alone, without setuptools. In order to not flatten everything, one solution is to simply create subfolders in mpl-data. We could have those folders: /lib/matplotlib/mpl-data/fonts/afm /lib/matplotlib/mpl-data/fonts/ttf /lib/matplotlib/mpl-data/images -- Nicolas |
|
From: Nicolas G. <nic...@ga...> - 2007-02-12 18:47:05
|
On 2/12/07, Andrew Straw <str...@as...> wrote: > IMO, the way to fix it is to move all the data into > lib/matplotlib/mpl-data to start with. If we did that, all the shuffling > that setup.py does would be unnecessary and using setuptools' develop > mode would work out-of-the-box. I'd be happy to move these files in the > svn repository if people agree this is a good idea. Me too, I'm +1 on that. This way, in development mode, we can run matplotlib just by adding directory "lib" to the PYTHONPATH, even if using standard distutils without setuptools. NG |
|
From: Andrew S. <str...@as...> - 2007-02-12 23:57:14
|
Great, thanks for checking that in. It looks like images/*.png didn't make it in. John Hunter wrote: > On 2/12/07, Andrew Straw <str...@as...> wrote: > > >> So, I sent the patch file to John, who'll hopefully commit it if he >> approves. To paraphrase Robert Kern, "Working code ends all arguments". >> So, since this was the nested way, and the biggest reason against it was >> that it require code change, well, I changed the code, and perhaps the >> arguments are ended? >> >> Because I did change a fair amount of data_path -using code, it'd be >> good to exercise all your favorite font and image finding sections of >> code. The bits I tried worked for me, of course, but no guarantees... >> > > OK, with a sledgehammer and a prayer, I got it in. I got the same > error you did, and then had to physically remove mpl-data and it's > contents from svn and commit, restore everything in the new order and > commit again. It compiled, installed and ran, passing the tests I > threw at it. > > I only ran about half of backend driver before I grew impatient, so > others may want to do some additional testing. I copied your note > from CHANGELOG to API_CHANGES. > > I suggest flushing your old mpl-data in site-packages/matplotlib as > well as your ~/.matplotlib/ttffont.cache file before testing. > > Thanks for settling the argument! > > JDH > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier. > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > |
|
From: Andrew S. <str...@as...> - 2007-02-13 00:03:26
|
Andrew Straw wrote: > Great, thanks for checking that in. > > It looks like images/*.png didn't make it in. > And, grr, I can't put them in, either: $ svn commit -m "added .png files that didn't make it into new mpl-data location" images Adding (bin) images/back.png svn: Commit failed (details follow): svn: COPY of back.png: 403 Forbidden (https://svn.sourceforge.net) > John Hunter wrote: > >> On 2/12/07, Andrew Straw <str...@as...> wrote: >> >> >> >>> So, I sent the patch file to John, who'll hopefully commit it if he >>> approves. To paraphrase Robert Kern, "Working code ends all arguments". >>> So, since this was the nested way, and the biggest reason against it was >>> that it require code change, well, I changed the code, and perhaps the >>> arguments are ended? >>> >>> Because I did change a fair amount of data_path -using code, it'd be >>> good to exercise all your favorite font and image finding sections of >>> code. The bits I tried worked for me, of course, but no guarantees... >>> >>> >> OK, with a sledgehammer and a prayer, I got it in. I got the same >> error you did, and then had to physically remove mpl-data and it's >> contents from svn and commit, restore everything in the new order and >> commit again. It compiled, installed and ran, passing the tests I >> threw at it. >> >> I only ran about half of backend driver before I grew impatient, so >> others may want to do some additional testing. I copied your note >> from CHANGELOG to API_CHANGES. >> >> I suggest flushing your old mpl-data in site-packages/matplotlib as >> well as your ~/.matplotlib/ttffont.cache file before testing. >> >> Thanks for settling the argument! >> >> JDH >> >> ------------------------------------------------------------------------- >> Using Tomcat but need to do more? Need to support web services, security? >> Get stuff done quickly with pre-integrated technology to make your job easier. >> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Matplotlib-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> >> > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier. > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > |
|
From: John H. <jd...@gm...> - 2007-02-13 15:19:54
|
On 2/12/07, Andrew Straw <str...@as...> wrote: > Great, thanks for checking that in. > > It looks like images/*.png didn't make it in. OK, I committed these. I'm not sure why they didn't go in the first time since they were in my mpl-data/images dir when I added it. Give it a test drive. |
|
From: Nicolas G. <nic...@ga...> - 2007-02-13 15:41:55
|
On 2/13/07, John Hunter <jd...@gm...> wrote: > On 2/12/07, Andrew Straw <str...@as...> wrote: > > Great, thanks for checking that in. > > > > It looks like images/*.png didn't make it in. > > OK, I committed these. I'm not sure why they didn't go in the first > time since they were in my mpl-data/images dir when I added it. Give > it a test drive. I've checked out this (with the PNG files !) and it works perfectly. Thanks! Nicolas |
