Menu
▾
▴
numpy-discussion — Discussion list for all users of Numerical Python
You can subscribe to this list here.
| 2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
| 2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
| 2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
| 2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
| 2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
| 2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
|
From: Colin J. W. <cj...@sy...> - 2006-01-22 22:24:24
|
Travis Oliphant wrote:
> Colin J. Williams wrote:
>
>> In the script below, b is presented as an integer, either with str or
>> repr, but it is an array and needs further processing to treat it as
>> in integer.
>
>
> I think there is a misunderstanding here---or, at least you need to
> clarify what your concern is.
Travis,
You are right, it was an incomplete understanding on my part. Sasha
helped to explain things.
> In the code you provided, b is not an array. It is a scalar that has
> the attributes of an array (but it is also subclassed from a Python
> integer and acts like one everywhere).
Yes, it is dressed like an array but behaves as an integer.
>>
>> I much prefer the numarray treatment, particularly in a matrix context.
>
>
> What do you mean by "in a matrix context?"
numarray returns a simple scalar, without the ArrayType dressing. numpy
would appear to be operationally equivalent and thus adequate.
In exploring these matters, it seems that numpy does not broadcast
scalar operations on arrays, as numarray does. Is this intended?
Please see the script below and the result under that.
# tArray.py
from numpy.core import multiarray as _mu
import numarray.numarraycore as _na
naa= _na.array([1, 2])
print type(naa[1]),
print naa * 3
mua= _mu.array([3, 4])
print type(mua[1]),
print mua * 3 # Grief here
*** Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit
(Intel)] on win32. ***
>>>
<type 'int'> [3 6]
<type 'int32_arrtype'>
Traceback (most recent call last):
File "<string>", line 63, in run_nodebug
File "C:\Python24\Lib\site-packages\PyMatrix\Misc\Test\tArray.py",
line 11, in ?
print mua * 3 # Grief here
TypeError: unsupported operand type(s) for *: 'numpy.ndarray' and 'int'
|
|
From: Nadav H. <Na...@Vi...> - 2006-01-22 12:33:07
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=utf-8" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<br>
Paulo Jose da Silva e Silva wrote:
<blockquote cite="mid...@lo..."
type="cite">
<pre wrap="">Em Dom, 2006-01-22 às 12:59 +0200, Nadav Horesh escreveu:
</pre>
<blockquote type="cite">
<pre wrap="">Dimension: 500
Array 1.2200 4.6700 1.0700 1188.6400 15.3500 2.5000 2.8100 <--
Matrix 13.1100 4.7800 1.1000 1052.9200 15.3500 2.6000 2.9100 <--
NumArr 2.4700 7.5800 1.1900 76.7100 13.0600 1.8400 3.6600 <--
Numeri 1.3700 7.3900 1.2400 1068.0200 14.6800 1.3600 3.5200 <--
</pre>
</blockquote>
<pre wrap=""><!---->
I bet that Numeric and Numpy are not really accessing the ATLAS
libraries or the ATLAS package you are using are using in Linux are not
optimized for you architecture.
</pre>
</blockquote>
The ATLAS lib was compiled on the same machine. But it is very possible
that I did not edited site.cfg correctly to search the ATLAS libraries
before the standard Blas/Lapack.<br>
There is some difficulty with gentoo on an amd64 machine, because some
packages (lapack-atlas and blas-atlas) are masked, and g2c is not on
the expected path.<br>
I'll try to play around more with site.cfg.<br>
<blockquote cite="mid...@lo..."
type="cite">
<pre wrap="">
The same test in my system gives:
Dimension: 500
Array 1.09 8.05 1.77 151.32 1.89 3.63 3.86
Matrix 4.87 8.16 1.82 151.44 1.94 4.22 4.35
NumArr 3.12 8.19 1.80 151.66 19.83 3.94 5.30
Numeri 1.43 8.18 1.81 151.68 17.52 2.94 4.08
</pre>
</blockquote>
It is really looks like I can reach the desired speed.<br>
BTW: Which OS?<br>
<blockquote cite="mid...@lo..."
type="cite">
<pre wrap="">
And my system is only a Sempron overclocked to run at 1900Mhz (and a
400Mhz bus) with an SSE optimized ATLAS. Your machine should beat mine
by a large margin.
Best,
Paulo
</pre>
</blockquote>
Thank you,<br>
<br>
Nadav.<br>
<blockquote cite="mid...@lo..."
type="cite">
<pre wrap="">
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
<a class="moz-txt-link-freetext" href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642</a>
_______________________________________________
Numpy-discussion mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Num...@li...">Num...@li...</a>
<a class="moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/numpy-discussion">https://lists.sourceforge.net/lists/listinfo/numpy-discussion</a>
</pre>
</blockquote>
</body>
</html>
|
|
From: Paulo J. da S. e S. <pjs...@im...> - 2006-01-22 11:48:46
|
Em Dom, 2006-01-22 às 12:59 +0200, Nadav Horesh escreveu: > Dimension: 500 > Array 1.2200 4.6700 1.0700 1188.6400 15.3500 2.5000 2.8100 <-- > Matrix 13.1100 4.7800 1.1000 1052.9200 15.3500 2.6000 2.9100 <-- > NumArr 2.4700 7.5800 1.1900 76.7100 13.0600 1.8400 3.6600 <-- > Numeri 1.3700 7.3900 1.2400 1068.0200 14.6800 1.3600 3.5200 <-- > I bet that Numeric and Numpy are not really accessing the ATLAS libraries or the ATLAS package you are using are using in Linux are not optimized for you architecture. The same test in my system gives: Dimension: 500 Array 1.09 8.05 1.77 151.32 1.89 3.63 3.86 Matrix 4.87 8.16 1.82 151.44 1.94 4.22 4.35 NumArr 3.12 8.19 1.80 151.66 19.83 3.94 5.30 Numeri 1.43 8.18 1.81 151.68 17.52 2.94 4.08 And my system is only a Sempron overclocked to run at 1900Mhz (and a 400Mhz bus) with an SSE optimized ATLAS. Your machine should beat mine by a large margin. Best, Paulo |
|
From: Nadav H. <Na...@Vi...> - 2006-01-22 11:16:26
|
I ran benchmark.py from the "Numpy x Matlab: some synthetic benchmarks" thread (the first version), on a amd64 machine (dual-core athlon 4400+, 1GB RAM) running either win64 or gentoo linux (dual boot). And got a strange results in the dimension 500 matrix multiplication benchmark: On windows: Tests x.T*y x*y.T A*x A*B A.T*x half 2in2 Dimension: 5 Array 0.7547 0.2491 0.1939 0.2688 0.8351 0.6387 0.8398 Matrix 6.4227 1.3667 0.7070 0.8192 1.4211 2.2491 3.3690 NumArr 1.7175 3.0416 0.3539 3.0406 5.0468 4.3359 7.0414 Numeri 0.8756 0.2918 0.2639 0.3418 0.6445 0.5888 0.6953 Dimension: 50 Array 7.8431 1.5494 0.7105 12.9425 6.4525 2.1420 2.4210 Matrix 69.4642 2.8870 1.2589 13.6162 7.1357 3.7315 4.9553 NumArr 17.6435 4.6267 0.9372 33.3093 6.6129 4.7710 7.5562 Numeri 9.3534 1.6228 1.0502 12.9967 5.4694 0.9972 2.0816 Dimension: 500 Array 1.1935 6.7593 1.6090 129.6738 12.8805 1.6672 2.0153 <-- Matrix 12.6622 6.8386 1.6488 129.8621 12.6773 1.9591 2.1887 <-- NumArr 2.2546 7.5078 1.0158 545.9313 7.6133 0.6549 1.4395 <-- Numeri 1.3390 6.8766 1.2113 133.2397 12.7436 0.5740 1.7577 <-- On linux: Dimension: 5 Array 0.7500 0.1600 0.1600 0.1800 0.7200 0.5900 0.7900 Matrix 5.9600 1.1800 0.5900 0.6300 1.2700 2.1500 3.3200 NumArr 1.9400 0.4500 0.4300 0.4600 5.4100 4.6500 7.4600 Numeri 0.9100 0.2200 0.2100 0.2400 0.5200 0.4400 0.5600 Dimension: 50 Array 7.8700 1.5900 0.6900 25.9800 7.9200 2.3700 2.6300 Matrix 67.1700 2.8100 1.1500 26.5000 8.4700 3.9500 5.2000 NumArr 20.0700 1.6200 0.9500 10.4400 7.8400 5.3500 8.2600 Numeri 9.4300 1.7700 0.7400 26.1000 9.6500 0.7700 3.1500 Dimension: 500 Array 1.2200 4.6700 1.0700 1188.6400 15.3500 2.5000 2.8100 <-- Matrix 13.1100 4.7800 1.1000 1052.9200 15.3500 2.6000 2.9100 <-- NumArr 2.4700 7.5800 1.1900 76.7100 13.0600 1.8400 3.6600 <-- Numeri 1.3700 7.3900 1.2400 1068.0200 14.6800 1.3600 3.5200 <-- Numeric/numpy matrix multiplication is about 8 fold slower on linux, and about 7 fold fast with numarray. Configuration: On win64 I used the provided binaries for numarray1.5, numpy0.92 and scipy 4.4 (compiled for P4+sse2) On linux I used numarray 1.5.1 (from cvs) numpy0.92, and scipy0.44, all compiled from source for 64 bits, and linked with ATLAS 3.7.11 (linking with ACML provided roughly the same figures). Any idea were this huge performance factor came from? Nadav. |
|
From: Travis O. <oli...@ie...> - 2006-01-22 07:48:07
|
Due to last-minute use of PyOS_ascii_strtod(), the numpy-0.9.4 release won't build properly with Python 2.3. The fix is easy and I made it to release a windows binary for numpy-0.9.4 on Python 2.3. I'll put a patch up soon. If you need support for Python 2.3, get numpy out of SVN or use the patch. -Travis |
|
From: Matthew B. <mat...@gm...> - 2006-01-22 01:13:46
|
Hi, > Thanks very much for this useful information! I totally agree, long live = the matrices... > In the ebook you are selling, are things like that (which functions prese= rve matrix-type) > documented? Those kind of things would be a reason for me to buy it. A bit off-topic I suppose, but can I second several previous suggestions that we all buy a copy if we can? First, it's very useful, and second to support Travis in his Herculean efforts. Best, Matthew |
|
From: Sasha <nd...@ma...> - 2006-01-22 00:57:21
|
On 1/21/06, Colin J. Williams <cj...@sy...> wrote: > ... > I much prefer the numarray treatment, particularly in a matrix context. > I suggest that, if it looks like an integer, b[1] should return a Python > scalar. In NumPy b[1] IS an integer: >>> from numpy import * >>> a =3D array([1,2,3]) >>> b =3D a[1] >>> isinstance(b, int) True It IS NOT rank-0 ndarray: >>> isinstance(b, ndarray) False Surely, the whole truth is that numpy scalars are instances of classes derived from Python scalars. What is the problem that numpy is causing you? If b was a python scalar, b.dtype would just raise an AttributeError. -- sasha |
|
From: Colin J. W. <cj...@sy...> - 2006-01-22 00:11:18
|
In the script below, b is presented as an integer, either with str or repr, but it is an array and needs further processing to treat it as in integer. I much prefer the numarray treatment, particularly in a matrix context. I suggest that, if it looks like an integer, b[1] should return a Python scalar. Script: # tScalar.py Scalar vs rank 0 import numpy.core.multiarray as _mu a= _mu.array([1, 2, 4]) print 'a:', a, 'repr(a):', repr(a), 'shape:', a.shape, 'dtype:', a.dtype b= a[1] print 'b:', b, 'repr(b):', repr(b), 'shape:', b.shape, 'dtype:', b.dtype Result: a: array([1, 2, 4], 'l') repr(a): array([1, 2, 4], 'l') shape: (3,) dtype: <type 'int32_arrtype'> b: 2 repr(b): 2 shape: () dtype: <type 'int32_arrtype'> Colin W. |
|
From: Sasha <nd...@ma...> - 2006-01-21 17:53:39
|
You may be hitting a known problem in lapack's _geev functions that rely on computations not being performed with extra precision. See https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=3D138683 -- sasha On 1/21/06, Jose Borreguero <bor...@gm...> wrote: > Hi all, > I don't understand what's going on. Here's my python session: > $ python > >>> from numpy.random import rand > >>> a=3Drand(3,3) > >>> from numpy.linalg import det,eig > >>> det(a) > 0.070796819514446802 > >>> eig(a) > and the process freezes here (at least 18minutes from now). I checked wit= h > 'top' and python is using all CPU. > Any ideas, please? > jose > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://sel.as-us.falkag.net/sel?cmdlnk&kid=103432&bid#0486&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
|
From: Alan G I. <ai...@am...> - 2006-01-21 17:51:27
|
On Sat, 21 Jan 2006, Jose Borreguero apparently wrote:
>>>> eig(a)
> and the process freezes here (at least 18minutes from now). I checked with
> 'top' and python is using all CPU.
No problem here.
fwiw,
Alan Isaac
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as N
>>> a=N.rand(3,3)
>>> from numpy.linalg import det,eig
>>> det(a)
-0.1313102666029379
>>> eig(a)
(array([ 1.65540878, -0.16786818, 0.47252527]), array([[-0.55573211, -0.8712454
8, 0.25278726],
[-0.52611295, 0.37246933, -0.5095001 ],
[-0.64371343, 0.31968409, 0.82250122]]))
>>> N.__version__
'0.9.2'
>>>
|
|
From: Jose B. <bor...@gm...> - 2006-01-21 17:15:49
|
Hi all, I don't understand what's going on. Here's my python session: $ python >>> from numpy.random import rand >>> a=3Drand(3,3) >>> from numpy.linalg import det,eig >>> det(a) 0.070796819514446802 >>> eig(a) and the process freezes here (at least 18minutes from now). I checked with 'top' and python is using all CPU. Any ideas, please? jose |
|
From: Darren D. <dd...@co...> - 2006-01-21 02:40:39
|
Would be possible to add a feature to numpy's fromstring function, to make it consistent with fromfile? fromfile will return an array from a binary file or an ascii file, but fromstring will only work with binary data. I would put fromstring to good use if it dealt with ascii data. Darren |
|
From: Darren D. <dd...@co...> - 2006-01-20 23:29:29
|
I'm looking for the license file in svn numpy, but cant find it. a grep of "license" on the numpy directory turned up several references to terms of the SciPy license, and to the missing LICENSE.txt. Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's bugzilla, and raised questions about numpy's license. Darren |
|
From: Travis O. <oli...@ee...> - 2006-01-20 23:03:16
|
Russell E. Owen wrote: >We're getting numeric data from a (MySQL) database. We'd like to use >numarray or NumPy on the resulting data, but some values may be None. Is >there a fast, efficient way to replace None with NaN? I'd hate to use a >list comprehension on each data tuple before turning it into an array, >but I haven't thought of anything else. > > Current numpy SVN allows array([1.0,None,2.0],dtype=float) array([ 1. , nan, 2. ]) -Travis |
|
From: Bruce S. <bso...@gm...> - 2006-01-20 22:47:41
|
Hi, > The Numpy code itself is BSD but the included f2py is LGPL, so it seems > that the entire package is LGPL. This is not really correct but I am not a lawyer or license expert nor a Numpy developer. Just because f2py is (or rather was) LGPL it does not automatically make all the Numpy code LGPL. If a piece of code is a derivative of f2py then it must be released as LGPL or with a more restrictive license that does not violate the LGPL but you can not release it under a less restrictive license such as BSD. But any piece of code that is not a derivative of f2py (such as being completely independent of f2py) can have it own license (that must not violate other licenses). Exactly what is a derivative is a real mess so seek legal advice. Regards Bruce On 1/20/06, Darren Dale <dd...@co...> wrote: > > On Friday 20 January 2006 14:50, Ignacio Vazquez-Abrams wrote: > > On Fri, 2006-01-20 at 13:24 -0500, Darren Dale wrote: > > > I also apologize if this is a duplicate, my mail isnt getting through > > > either. > > > > > > I'm looking for the license file in svn numpy, but cant find it. a > grep > > > of "license" on the numpy directory turned up several references to > terms > > > of the SciPy license, and to the missing LICENSE.txt. > > > > > > Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's > > > bugzilla, and raised questions about numpy's license. > > > > The Numpy code itself is BSD but the included f2py is LGPL, so it seems > > that the entire package is LGPL. > > I don't want this to seem disrespectful, but could I get a second opinion > on > this? > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D103432&bid=3D230486&dat= =3D121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
|
From: Russell E. O. <ro...@ce...> - 2006-01-20 22:43:53
|
In article <d38...@ma...>, Sasha <nd...@ma...> wrote: > >>> from numpy.core.ma import masked values > >>> from numpy import nan > >>> masked values([1.0,None,2.0],None).filled(nan).astype(float) > array([ 1. , nan, 2. ]) Neat! Any idea if that's likely to keep working in future versions? It doesn't work in numarray.ma.masked_values and in general it seems like a lot of numpy and numarray raise exception when they get a list that contains None. -- Russell |
|
From: Travis O. <oli...@ee...> - 2006-01-20 22:27:46
|
Sasha wrote: >On 1/19/06, Travis Oliphant <oli...@ie...> wrote: > > >>... >>I don't think this is right. zero-rank float arrays use Python's >>floating-point str or repr function to produce the string, so whatever >>Python is doing will be used. >> >> > >Well, it looks like they use repr for str: > > > >>>>from numpy import * >>>>len(str(1/3.)) >>>> >>>> >14 > > >>>>len(str(array(1/3.))) >>>> >>>> >19 > > >>>>len(repr(1/3.)) >>>> >>>> >19 > > > O.K. That is a bug. And it is now fixed (an oversight...) -Travis |
|
From: Travis O. <oli...@ee...> - 2006-01-20 22:16:33
|
Ignacio Vazquez-Abrams wrote: >On Fri, 2006-01-20 at 13:24 -0500, Darren Dale wrote: > > >>I also apologize if this is a duplicate, my mail isnt getting through either. >> >>I'm looking for the license file in svn numpy, but cant find it. a grep of >>"license" on the numpy directory turned up several references to terms of the >>SciPy license, and to the missing LICENSE.txt. >> >>Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's >>bugzilla, and raised questions about numpy's license. >> >> > >The Numpy code itself is BSD but the included f2py is LGPL, so it seems >that the entire package is LGPL. > > Pearu has agreed to release the numpy.f2py under the same license as NumPy. Thus, there should be no problem regardless of the veracity of this implication. -Travis |
|
From: Darren D. <dd...@co...> - 2006-01-20 22:02:21
|
On Friday 20 January 2006 14:50, Ignacio Vazquez-Abrams wrote: > On Fri, 2006-01-20 at 13:24 -0500, Darren Dale wrote: > > I also apologize if this is a duplicate, my mail isnt getting through > > either. > > > > I'm looking for the license file in svn numpy, but cant find it. a grep > > of "license" on the numpy directory turned up several references to terms > > of the SciPy license, and to the missing LICENSE.txt. > > > > Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's > > bugzilla, and raised questions about numpy's license. > > The Numpy code itself is BSD but the included f2py is LGPL, so it seems > that the entire package is LGPL. I don't want this to seem disrespectful, but could I get a second opinion on this? |
|
From: Travis O. <oli...@ee...> - 2006-01-20 22:00:14
|
|
From: Ignacio Vazquez-A. <iva...@iv...> - 2006-01-20 19:50:43
|
On Fri, 2006-01-20 at 13:24 -0500, Darren Dale wrote: > I also apologize if this is a duplicate, my mail isnt getting through eit= her. >=20 > I'm looking for the license file in svn numpy, but cant find it. a grep o= f=20 > "license" on the numpy directory turned up several references to terms of= the=20 > SciPy license, and to the missing LICENSE.txt. >=20 > Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's=20 > bugzilla, and raised questions about numpy's license. The Numpy code itself is BSD but the included f2py is LGPL, so it seems that the entire package is LGPL. --=20 Ignacio Vazquez-Abrams <iva...@iv...> http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 |
|
From: Francesc A. <fa...@ca...> - 2006-01-20 19:05:37
|
A Divendres 20 Gener 2006 16:42, Colin J. Williams va escriure: > Travis Oliphant wrote: > > I'd like to make a release of NumPy over the weekend. Please submit > > bugs to the list before Saturday night (GMT -7) > > > > NumPy 0.9.2 was based on SVN version 1829 and we are over 100 checkins > > from that point, including the significant change to the .dtype > > attribute.... > > Will there be some note with the release explaining this, or do we have > to browse the discussion to get the detail? Colin told me that my previous message got truncated. I'm sending now a link to the original message from Travis: http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2979474 Hope this time it would look well. =2D-=20 >0,0< Francesc Altet =A0 =A0 http://www.carabos.com/ V V C=E1rabos Coop. V. =A0=A0Enjoy Data "-" |
|
From: Giles H. <gh...@re...> - 2006-01-20 19:00:42
|
Hi,
This simple python code snippit used to core dump on my modern AMD 64
Linux servers:
from numarray import *
a = zeros( 10, Float32 )
b = transpose(a)
dot( b,a )
This crash was caused by a corrupted "libnumarray_API". The pointer was
initially assigned the correct value, but becomes corrupted almost
immediately after its initial runtime binding. On my systems, line 725
of _dotblas.c would clobber the "libnumarray_API" pointer while
initializing the "dotFunctions" vector table. the problem is
"dotFunctions", an array of function pointers, is sized statically by
"PyArray_NTYPES", an enumerated constant that reflects the value of
"tMaxType" defined in arraybase.h like this:
typedef enum
{
tAny,
tBool,
tInt8,
tUInt8,
tInt16,
tUInt16,
tInt32,
tUInt32,
tInt64,
tUInt64,
tFloat32,
tFloat64,
tComplex32,
tComplex64,
tObject, /* placeholder... does nothing */
tDefault = tFloat64,
#if LP64
tLong = tInt64,
#else
tLong = tInt32,
#endif
tMaxType
} NumarrayType;
The problem is that "tMaxType" is not assigned 17, like we would expect,
since it's at the end of the list. On my machine, "tMaxType" is
assigned the value 9! The problem is the explicit assignment of "tLong"
and "tDefault". Explicitly assigned constants in an enumerated list
will modify the running count. Since the macro "LP64" is defined,
"tLong" is assigned value of "tInt64", which is 8, and "tMaxType" is
assigned the next value, 9. This means the vector table "dotFunctions"
is shorter then we intend, and overflowing this table can corrupt
important data. The fix is simple, just define tMaxType before the
explicitly assigned aliases:
typedef enum
{
tAny,
tBool,
tInt8,
tUInt8,
tInt16,
tUInt16,
tInt32,
tUInt32,
tInt64,
tUInt64,
tFloat32,
tFloat64,
tComplex32,
tComplex64,
tObject, /* placeholder... does nothing */
tMaxType,
tDefault = tFloat64,
#if LP64
tLong = tInt64,
#else
tLong = tInt32,
#endif
} NumarrayType;
|
|
From: Darren D. <dd...@co...> - 2006-01-20 18:24:26
|
I also apologize if this is a duplicate, my mail isnt getting through either. I'm looking for the license file in svn numpy, but cant find it. a grep of "license" on the numpy directory turned up several references to terms of the SciPy license, and to the missing LICENSE.txt. Some folks are working on a gentoo ebuild for numpy/scipy in gentoo's bugzilla, and raised questions about numpy's license. Darren |
|
From: Schofield, E. <Ed....@ft...> - 2006-01-20 18:16:18
|
(Apologies if this is a duplicate. My mail isn't getting through ...)
Travis Oliphant wrote:
>> Sven Schreiber wrote:
>>
>> I see, thanks for the quick answer. So wouldn't it be a good idea to
>> have all the specifically
>> matrix-related stuff (afaics, things in numpy/lib/twodim_base.py)
>> return matrices?
>>
>> It seems my question (or misunderstanding) has a broader scope:
>> Coming from matrix languages, I'm
>> glad about short notations like A.I or A*B representing standard
>> matrix operations. Much easier than
>> linalg.inverse(A) or matrixmultiply(A,B). However, many matrix
>> functions (decompositions etc.) seem
>> to return arrays instead of matrices, even if you feed them matrices
>> (is this assumption correct?).
>> So you have to use mat(returned-result) again and again to be able to
>> do return-result.I afterwards,
>> which seems clumsy. So what's the best strategy here?
>
> Yes, some of them do still return arrays. Matrices are longer lived
> in NumPy then the were in Numeric, for sure, but many functions still
> aren't friendly to matrices and convert all inputs to arrays before
> operation. Originally, I had the asarray(...) function not convert
> matrices by default, but this is too surprising because matrices
> change the '*' and '**' operators which could make your function not
> work.
> We should convert all the functions that don't handle matrices so they
> will. I'd like to see matrices survive longer than they do. There
> are some functions that try to do that
I agree, and I'm willing to help with this. We should also think about
handling SciPy's spmatrix objects properly. In this case we don't want
NumPy to have any dependency on a particular format for SciPy's spmatrix
objects -- but we could design and publish a simple interface for sparse
matrix objects (from SciPy or anywhere else) to conform to for basic
NumPy compatibility.
How should we go about this? Let's take linalg.solve_linear_equations()
as an example. We could use isinstance(x, matrix) for dense matrix
objects, because they're part of NumPy, then use asarray(x), process =
them
as arrays, and wrap them up with asmatrix(x) at the end. But with =
sparse
matrices this wouldn't work. What if we specify instead that objects
passed to functions like solve_linear_equations() need to provide two
members:
x.toarray(), allowing x to represent itself as an array
x.fromarray(a), creating another object of the same type as x out of
the given array a
We could choose other names, of course, maybe asarray() or whatever =
else.
But this would simplify the code for all these functions while allowing
us to support other array-like objects without needing to know any more
about them.
I'm also +1 on eye() returning a matrix :)
-- Ed
|
258 messages has been excluded from this view by a project administrator.