Menu
▾
▴
numpy-discussion
|
From: Daran L. R. <dr...@uc...> - 2006-08-10 05:45:54
|
Hello, I recently switched from a Debian Linux box to a Mac G5 PowerPC, running Mac OS X 10.4 Tiger (8.7.0). I use the Python Numeric package extensively, and have come to depend upon it. In my view, this piece of software is truly first rate, and it has greatly improved my productivity in the area of scientific analysis. Unfortunately, I am experiencing a problem that I cannot sort out. I am running Python 2.4.3 on a Debian box (V3.1), using gcc version 4.0.1, and the Apple vecLib.framework which has an optimized BLAS and LAPACK. When building Numeric 24.0, 24.1, or 24.2 everything seems to go AOK. But when I run code which makes use of the Numeric package (maksed arrays, dot product, LinearAlgebra, etc.) my code crashes hard and unpredictably. When it crashes I simply get a "Segmentation Fault". I'm sorry that I can't be more specific about what seems to happen just before the crash...I've tried to trace it but to no avail. Interestingly, I can get Numeric version 23.8 to build and run just fine, but it appears that the dotblas (BLAS optimized matrixmultiply/dot/innerproduct) does not properly get built in. Thus, all my matrix operations are -very- slow. Has anyone seen this problem, or know where I might look to solve it? Perhaps I have overlooked a crucial step in the build/install of Numeric 24.x on the Mac. I searched round the Net with google, and have sifted through the numpy/scipy/numeric Web pages, various mailing lists, user groups, etc., and can't seem to find any relevant info. Alternatively, can someone explain how to get Numeric 23.8 to compile on OS X 10.4 Tiger, with the dotblas module? Thanks very much for your help, Daran |
|
From: Daran L. R. <dr...@uc...> - 2006-08-10 14:02:32
|
Hi group, Sorry, but there was an error on my previous message, 2nd paragraph, 2nd setence. It should read: Unfortunately, I am experiencing a problem that I cannot sort out. I am running Python 2.4.3 on a Mac G5 running OS X 10.4 Tiger (8.7.0), using gcc version 4.0.1, and the Apple vecLib.framework which has an optimized BLAS and LAPACK. When building Numeric 24.0, 24.1, or 24.2 everything seems to go AOK. But when I run code which makes use of the Numeric package (maksed arrays, dot product, LinearAlgebra, etc.) my code crashes hard and unpredictably. When it crashes I simply get a "Segmentation Fault". I'm sorry that I can't be more specific about what seems to happen just before the crash... I've tried to trace it but to no avail. Thanks again for your help. Daran -- > I recently switched from a Debian Linux box to a Mac G5 > PowerPC, running Mac OS X 10.4 Tiger (8.7.0). I use the > Python Numeric package extensively, and have come to > depend upon it. In my view, this piece of software is > truly first rate, and it has greatly improved my > productivity in the area of scientific analysis. > > Unfortunately, I am experiencing a problem that I cannot sort > out. I am running Python 2.4.3 on a Debian box (V3.1), using > gcc version 4.0.1, and the Apple vecLib.framework which has > an optimized BLAS and LAPACK. When building Numeric 24.0, > 24.1, or 24.2 everything seems to go AOK. But when I run > code which makes use of the Numeric package (maksed arrays, > dot product, LinearAlgebra, etc.) my code crashes hard and > unpredictably. When it crashes I simply get a "Segmentation > Fault". I'm sorry that I can't be more specific about what > seems to happen just before the crash...I've tried to trace > it but to no avail. > > Interestingly, I can get Numeric version 23.8 to build and > run just fine, but it appears that the dotblas (BLAS > optimized matrixmultiply/dot/innerproduct) does not properly > get built in. Thus, all my matrix operations are -very- slow. > > Has anyone seen this problem, or know where I might look > to solve it? Perhaps I have overlooked a crucial step in > the build/install of Numeric 24.x on the Mac. > > I searched round the Net with google, and have sifted through > the numpy/scipy/numeric Web pages, various mailing lists, user > groups, etc., and can't seem to find any relevant info. > > Alternatively, can someone explain how to get Numeric 23.8 > to compile on OS X 10.4 Tiger, with the dotblas module? > > > Thanks very much for your help, > > > Daran > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
|
From: Hanno K. <kl...@ph...> - 2006-08-10 14:13:01
|
Daran, I had a similar behaviour when I tried to use module compield with an older f2py with a newer version of numpy. So is it maybe possible that some *.so files are used from an earlier build? Hanno "Daran L. Rife" <dr...@uc...> said: > Hi group, > > Sorry, but there was an error on my previous message, > 2nd paragraph, 2nd setence. It should read: > > Unfortunately, I am experiencing a problem that I cannot sort > out. I am running Python 2.4.3 on a Mac G5 running OS X 10.4 > Tiger (8.7.0), using gcc version 4.0.1, and the Apple > vecLib.framework which has an optimized BLAS and LAPACK. > When building Numeric 24.0, 24.1, or 24.2 everything seems > to go AOK. But when I run code which makes use of the Numeric > package (maksed arrays, dot product, LinearAlgebra, etc.) my > code crashes hard and unpredictably. When it crashes I simply > get a "Segmentation Fault". I'm sorry that I can't be more > specific about what seems to happen just before the crash... > I've tried to trace it but to no avail. > > Thanks again for your help. > > > Daran > > -- > > > I recently switched from a Debian Linux box to a Mac G5 > > PowerPC, running Mac OS X 10.4 Tiger (8.7.0). I use the > > Python Numeric package extensively, and have come to > > depend upon it. In my view, this piece of software is > > truly first rate, and it has greatly improved my > > productivity in the area of scientific analysis. > > > > Unfortunately, I am experiencing a problem that I cannot sort > > out. I am running Python 2.4.3 on a Debian box (V3.1), using > > gcc version 4.0.1, and the Apple vecLib.framework which has > > an optimized BLAS and LAPACK. When building Numeric 24.0, > > 24.1, or 24.2 everything seems to go AOK. But when I run > > code which makes use of the Numeric package (maksed arrays, > > dot product, LinearAlgebra, etc.) my code crashes hard and > > unpredictably. When it crashes I simply get a "Segmentation > > Fault". I'm sorry that I can't be more specific about what > > seems to happen just before the crash...I've tried to trace > > it but to no avail. > > > > Interestingly, I can get Numeric version 23.8 to build and > > run just fine, but it appears that the dotblas (BLAS > > optimized matrixmultiply/dot/innerproduct) does not properly > > get built in. Thus, all my matrix operations are -very- slow. > > > > Has anyone seen this problem, or know where I might look > > to solve it? Perhaps I have overlooked a crucial step in > > the build/install of Numeric 24.x on the Mac. > > > > I searched round the Net with google, and have sifted through > > the numpy/scipy/numeric Web pages, various mailing lists, user > > groups, etc., and can't seem to find any relevant info. > > > > Alternatively, can someone explain how to get Numeric 23.8 > > to compile on OS X 10.4 Tiger, with the dotblas module? > > > > > > Thanks very much for your help, > > > > > > Daran > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job > > easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Num...@li... > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- Hanno Klemm kl...@ph... |
|
From: Daran L. R. <dr...@uc...> - 2006-08-10 14:58:53
|
Hi Hanno, > I had a similar behaviour when I tried to use module compield with an > older f2py with a newer version of numpy. So is it maybe possible that > some *.so files are used from an earlier build? Many thanks for the reply. This was my first attempt to build and use numpy; I have no previous version. May I ask how you specificlly solved the problem on your machine? Thanks, Daran -- |
|
From: Hanno K. <kl...@ph...> - 2006-08-10 16:50:43
|
Hi Daran, I fortunately never had the need to run different versions in parallel, so I basically removed the earlier versions of numpy. However, as you possibly know, you can build wrapper functions for fortran code with f2py (which is now shipped with numpy). And that is where I got the segfault behaviour: I had a module compiled for numpy 0.9.6 and then tried to use it with numpy 1.0b. Therefore I thought if you have similar stuff running on your machine that might be a reason. The obvious solution is to recompile the fortran code with the newer version of f2py. But fom what you write, your problem seems to be different. Regards, Hanno "Daran L. Rife" <dr...@uc...> said: > Hi Hanno, > > > I had a similar behaviour when I tried to use module compield with an > > older f2py with a newer version of numpy. So is it maybe possible that > > some *.so files are used from an earlier build? > > > Many thanks for the reply. This was my first attempt > to build and use numpy; I have no previous version. > May I ask how you specificlly solved the problem > on your machine? > > Thanks, > > Daran > > -- > > -- Hanno Klemm kl...@ph... |
|
From: Christopher B. <Chr...@no...> - 2006-08-10 16:14:12
|
Daran L. Rife wrote: > Many thanks for the reply. This was my first attempt > to build and use numpy; "numpy" used to be a generic name for the Numerical extensions to Python. Now there are three versions: "Numeric": The original, now at version 24.2 This is probably the last version that will be produced. "numarray": This was designed to be the "next generation" array package. It has some nice additional features that Numeric does not have, but is missing some as well. It is at version 1.5.1. it may see some bug fix releases in the future, but probably won't see any more major development. "numpy": this is the "grand unification" array package. It is based on the Numeric code base, and is designed to have the best features of Numeric and numarray, plus some extra good stuff. It is now at version 1.0beta, with an expected release date for 1.0final sometime this fall. It is under active development, the API is pretty stable now, and it appears to have the consensus of the numerical python community as the "way of the future" I wrote all that out so that you can be clear which package you are having trouble with -- you've used both the term "Numeric" and "numpy" in your posts, and there is some confusion. If you are working on a project that does not need to be released for a few months (i.e. after numpy has reached 1.0 final), I'd use numpy, rather than Numeric or numarray. Also: on OS-X, there are far to many ways to build Python. When you report a problem, you need to define exactly which python build you are using, and this goes beyond python version -- fink? darwinports? built-it-from-source? Framework? Universal, etc... The MacPython community is doing it's best to standardize on the Universal Build of 2.4.3 that you can find here: http://www.pythonmac.org/packages/py24-fat/ There you will also find pre-built packages for Numeric24.2, numarray1.5.1, and numpy0.9.8 Have you tried any of those? They should be built against Apple's vectLib. There isn't a package for numpy 1.0beta there yet. I may add one soon. > Interestingly, I can get Numeric version 23.8 to build and > run just fine, but it appears that the dotblas (BLAS > optimized matrixmultiply/dot/innerproduct) does not properly > get built in. Thus, all my matrix operations are -very- slow. I'm not sure of the dates, but that is probably a version that didn't have the check for Apple's vecLib in the setup.py, so it built with the built-in lapack-lite instead. You can compare the setup.py files from that and newer versions to see how to make it build against vectLib, but I suspect if you do that, you'll see the same problems. Also, please send a small test script that crashes for you, so others can test it. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
|
From: Daran L. R. <dr...@uc...> - 2006-08-10 16:36:23
|
Hi Chris, Thanks very much for your reply. My apology for the confusion. To be clear, I am a veteran user of Numeric not numpy. I tried installing four versions of Numeric: 23.8, 24.0, 24.1, and 24.2. My Python distro is built from source, using the GCC 4.0.1 suite of compilers. I am running all of this on a Mac G5 PowerPC with Mac OS X 10.4 Tiger (8.7.0). All branches of Numeric 24.x cause a "Segmentation Fault". The scripts I was running this against are a bit complex, so it is not so easy for me to sort out when/where the failure occurs. I'll keep doing some testing and see if I can get a better idea for what seems to be the issue. I'd very like much like to move to numpy, but I have code that needs to be working -now-, so at this point I am more interested in Numeric; I am an adept user of Numeric, and I know it works well on Debian Linux boxes. I will try your suggestion of installing and running the pre-built packages at <http://www.pythonmac.org/packages/py24-fat/>. Thanks again for your patience and for your help. Daran -- > Daran L. Rife wrote: >> Many thanks for the reply. This was my first attempt >> to build and use numpy; > > "numpy" used to be a generic name for the Numerical extensions to > Python. Now there are three versions: > > "Numeric": The original, now at version 24.2 This is probably the last > version that will be produced. > > "numarray": This was designed to be the "next generation" array package. > It has some nice additional features that Numeric does not have, but is > missing some as well. It is at version 1.5.1. it may see some bug fix > releases in the future, but probably won't see any more major development. > > "numpy": this is the "grand unification" array package. It is based on > the Numeric code base, and is designed to have the best features of > Numeric and numarray, plus some extra good stuff. It is now at version > 1.0beta, with an expected release date for 1.0final sometime this fall. > It is under active development, the API is pretty stable now, and it > appears to have the consensus of the numerical python community as the > "way of the future" > > I wrote all that out so that you can be clear which package you are > having trouble with -- you've used both the term "Numeric" and "numpy" > in your posts, and there is some confusion. > > If you are working on a project that does not need to be released for a > few months (i.e. after numpy has reached 1.0 final), I'd use numpy, > rather than Numeric or numarray. > > Also: on OS-X, there are far to many ways to build Python. When you > report a problem, you need to define exactly which python build you are > using, and this goes beyond python version -- fink? darwinports? > built-it-from-source? Framework? Universal, etc... > > The MacPython community is doing it's best to standardize on the > Universal Build of 2.4.3 that you can find here: > > http://www.pythonmac.org/packages/py24-fat/ > > There you will also find pre-built packages for Numeric24.2, > numarray1.5.1, and numpy0.9.8 > > Have you tried any of those? They should be built against Apple's > vectLib. There isn't a package for numpy 1.0beta there yet. I may add > one soon. > >> Interestingly, I can get Numeric version 23.8 to build and >> run just fine, but it appears that the dotblas (BLAS >> optimized matrixmultiply/dot/innerproduct) does not properly >> get built in. Thus, all my matrix operations are -very- slow. > > I'm not sure of the dates, but that is probably a version that didn't > have the check for Apple's vecLib in the setup.py, so it built with the > built-in lapack-lite instead. You can compare the setup.py files from > that and newer versions to see how to make it build against vectLib, but > I suspect if you do that, you'll see the same problems. > > Also, please send a small test script that crashes for you, so others > can test it. > > -Chris |
|
From: Daran L. R. <dr...@uc...> - 2006-08-11 05:50:38
|
Hi Chris, I tried your suggestion of installing and running the pre-built packages at <http://www.pythonmac.org/packages/py24-fat/>. I am sorry to report that the pre-built MacPython and Numeric 24.2 package did not work. I get the same "Segmentation Fault" that I got when I built Python 2.4.3 and Numeric 24.2 from source. I tried running my code with debug prints in various places to try and pin down where the problem arises. Thus, I ran my code a number of times. Strangely, it never crashes in the same place twice. I'm not sure what to do next, but I will keep at it. As a last resort, I may build ATLAS and LAPACK from source, then build Numeric 23.8 against these, and try installing this into MacPython. I hate having to try this, but I cannot do any development without a functioning Python and Numeric. Thanks again, Daran -- > Daran L. Rife wrote: >> Many thanks for the reply. This was my first attempt >> to build and use numpy; > > "numpy" used to be a generic name for the Numerical extensions to Python. Now there are three versions: > > "Numeric": The original, now at version 24.2 This is probably the last version that will be produced. > > "numarray": This was designed to be the "next generation" array package. It has some nice additional features that Numeric does not have, but is missing some as well. It is at version 1.5.1. it may see some bug fix releases in the future, but probably won't see any more major development. > > "numpy": this is the "grand unification" array package. It is based on the Numeric code base, and is designed to have the best features of Numeric and numarray, plus some extra good stuff. It is now at version 1.0beta, with an expected release date for 1.0final sometime this fall. It is under active development, the API is pretty stable now, and it appears to have the consensus of the numerical python community as the "way of the future" > > I wrote all that out so that you can be clear which package you are having trouble with -- you've used both the term "Numeric" and "numpy" in your posts, and there is some confusion. > > If you are working on a project that does not need to be released for a few months (i.e. after numpy has reached 1.0 final), I'd use numpy, rather than Numeric or numarray. > > Also: on OS-X, there are far to many ways to build Python. When you report a problem, you need to define exactly which python build you are using, and this goes beyond python version -- fink? darwinports? built-it-from-source? Framework? Universal, etc... > > The MacPython community is doing it's best to standardize on the Universal Build of 2.4.3 that you can find here: > > http://www.pythonmac.org/packages/py24-fat/ > > There you will also find pre-built packages for Numeric24.2, > numarray1.5.1, and numpy0.9.8 > > Have you tried any of those? They should be built against Apple's vectLib. There isn't a package for numpy 1.0beta there yet. I may add one soon. > > > Interestingly, I can get Numeric version 23.8 to build and > > run just fine, but it appears that the dotblas (BLAS > > optimized matrixmultiply/dot/innerproduct) does not properly > > get built in. Thus, all my matrix operations are -very- slow. > > I'm not sure of the dates, but that is probably a version that didn't have the check for Apple's vecLib in the setup.py, so it built with the built-in lapack-lite instead. You can compare the setup.py files from that and newer versions to see how to make it build against vectLib, but I suspect if you do that, you'll see the same problems. > > Also, please send a small test script that crashes for you, so others can test it. > > -Chris > > > > > -- > Christopher Barker, Ph.D. > Oceanographer > > NOAA/OR&R/HAZMAT (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chr...@no... > |
|
From: Christopher B. <Chr...@no...> - 2006-08-14 23:39:39
|
Daran L. Rife wrote: > I tried your suggestion of installing and running the pre-built > packages at <http://www.pythonmac.org/packages/py24-fat/>. I am > sorry to report that the pre-built MacPython and Numeric 24.2 > package did not work. I get the same "Segmentation Fault" that > I got when I built Python 2.4.3 and Numeric 24.2 from source. Darn. My few simple tests all work. If you can figure out which functions are failing, and make a small sample that fails, post it here and to the python-mac list. There are some smart folks there that might be able to help. > As a last resort, I may build ATLAS and LAPACK from source, > then build Numeric 23.8 against these, and try installing > this into MacPython. I hate having to try this, but I cannot > do any development without a functioning Python and Numeric. However, it might be easier to port to numpy that do all that. And you'll definitely get more help solving any problems you have with numpy. good luck. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
|
From: Daran R. <dr...@uc...> - 2006-08-14 23:57:00
|
Hi Chris,
> Darn. My few simple tests all work. If you can figure out which
> functions are failing, and make a small sample that fails, post it here
> and to the python-mac list. There are some smart folks there that might
> be able to help.
I will try to do so, but like you, I think my time is better
spent transitioning to Numpy.
Incidentally, I am now using the MacPython distro--thanks for
pointing me toward that. I also got Numeric 23.8 to work well
with MacPython, including the optimized vecLib framework. I
got the harebrained idea to try compiling and installing
Numeric 23.8 using the setup.py and customize.py files from
Numeric 24.x, since they seem to get the Apple veclib stuff
compiled in properly, especially the optimized matrix math
libs. The one tweak I had to make was in setup.py, where I
pointed it to the new vecLib in:
/System/Library/Frameworks/Accelerate.framework
> However, it might be easier to port to numpy that do all that. And
> you'll definitely get more help solving any problems you have with numpy.
Agreed. I am looking forward to the first official release of
numpy. In the meantime, I will experiment with the Beta version.
Thanks again,
Daran
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X