numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Matlab page on scipy wiki

[Stefan v. d. W. <st...@su...>]

On Thu, Feb 09, 2006 at 05:21:10PM +0900, Bill Baxter wrote:
> I added some content to the "NumPy/SciPy for Matlab users" page on the scipy
> wiki.
> 
> But my knowledge of NumPy/SciPy isn't sufficient to fill in the whole chart of
> equivalents that I laid out.

One of my colleagues also asked about the shortest way to do array
concatenation.

In Octave that would be

[1, 0, 1:4, 0, 1]

Using numpy we currently do

concatenate([[1, 0], arange(1,5), [0, 1]]) or
vstack(...)

The "+" operator now means something else, so you can't do

[1,0] + arange(1,5) + [0,1], while

[1, 0, arange(1,5), 0, 1] produces

[1, 0, array([1, 2, 3, 4]), 0, 1]

which can't be converted to an array by simply doing

array([[1, 0, array([1, 2, 3, 4]), 0, 1]])

I'll add it to the wiki once I know what the best method is.

Regards
Stéfan

Re: [Numpy-discussion] Matlab page on scipy wiki

[Sven S. <sve...@gm...>]

Fernando Perez schrieb:
> Bill Baxter wrote:
>> For what it's worth, matlab's rank function just calls svd, and
>> returns the
>> number singular values greater than a tolerance.  The implementation is a
>> whopping 5 lines long.
> 
> Yup, and it would be pretty much the same 5 lines in numpy, with the
> same semantics.
> 
> Here's a quick and dirty implementation for old-scipy (I don't have
> new-scipy on this box):
> 

Is there any reason not to use the algorithm implicit in lstsq, as in:
rk = linalg.lstsq(M, ones(p))[2]

(where M is the matrix to check, and p==M.shape[0])

thanks,
sven

[Numpy-discussion] BUG(?): array([None])==None test

[Pearu P. <pe...@sc...>]

Hi,

While converting some Numeric based code to numpy, I noticed that
in Numeric

   array([None])==None

returns array([1]) while in numpy it returns False.

Is this expected behaviour or a bug?

Pearu

[Numpy-discussion] BUG: numpy.put raises TypeError.

[Pearu P. <pe...@sc...>]

Hi,

There seems to be a bug in numpy.put function (i.e. array.put method). 
Consider the following example:

a = array([0, 0, 0, 0, 0])

a.put([1.1],[2])   # works as documented

a.put(array([1.1]),[2])   # raises the following exception:
   TypeError: array cannot be safely cast to required type

The bug seems to boil down to calling PyArray_FromArray function but I 
got a bit lost on debugging this issue..

Pearu

Re: [Numpy-discussion] Using ndarray for 2-dimensional, heterogeneous data

[Francesc A. <fa...@ca...>]

El dv 10 de 02 del 2006 a les 06:11 +0000, en/na N. Volbers va escriure:
> N. Volbers wrote:
> Sorry, I meant of course
>=20
>    b =3D numpy.array( [(1,2.3), (2, 17.2), (3, 19.1), (4, 22.2)],=20
> dtype=3D{'names':['col1','col2'], 'formats': ['i2','f4']})
>=20
> > Row operations are much easier now, because I can use numpy's=20
> > intrinsic capabilities. However column operations require to create a=20
> > new array based on the old one.

Yes, but this should be a pretty fast operation, as not data copy is
implied in doing b['col1'], for example.

> > Now I am wondering if the use of such an array has more drawbacks that=20
> > I am not aware of. E.g. is it possible to mask values in such an array?

I'm not familiar with masked arrays, but my understanding is that such
column arrays are the same than regular arrays, so I'd say yes.

> > And is it slower to get a certain column by using b['col1'] than it=20
> > would using a homogeneous array c and the notation c[:,0]?

Well, you should do some benchmarks, but I'd be surprised if there is a
big speed difference.

> > Does anyone else use such a data layout and can report on problems=20
> > with it?

I use column data *a lot* in numarray and had not problems with this.
With NumPy things should be similar in terms of stability.

> The mathematical operations I want to use will be limited to operations=20
> acting on the column e.g. creating a new column =3D b['col1'] + b['col2']=
=20
> and such. So of course I am aware of the basic difference that slicing=20
> works different if I have an heterogeneous array due to the fact that=20
> each row is considered a single item.

Exactly, these array columns are the same than regular homogeneous
arrays. The only difference is that there is a 'hole' between elements.
However, this is handled internally by NumPy through the use of the
stride.

My 2 cents,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"

Re: [Numpy-discussion] Re: NumPy Glossary Was:Matlab page on scipy wiki

[Bill B. <wb...@gm...>]

Well, I'm with Fernando.  Wikis are meant for having people muck with them.
I am much more annoyed for instance by the "NumPy Tutorial" teaser link on
the main documentation page that goes to nowhere than I would be by a half
finished page that acknowledges it's half finished.   If I'd known I was
supposed to go to some Dev Zone and put in a provisional page or whatnot I
probably wouldn't have made the NumPy for Matlab users page.  Which is stil=
l
half finished, thank-you-very-much, but nonetheless still contains useful
information.  More useful information than the "NumPy Tutorial" link at
least.  :-)

--bb

On 2/10/06, Fernando Perez <Fer...@co...> wrote:
>
> Joe Harrington wrote:
>
> >>Why? Did you read the argument I made for putting it on the main wiki?
> How are
> >>you going to get contributions on the dev wiki once anonymous edits are
> locked
> >>out (which they will hopefully be very soon, before the wiki is spammed
> out of
> >>recognition)?
> >
> >
> > Fernando, that page *is* on the main wiki (I don't deal with the
> > developers' wiki at all).  Go to scipy.org, click on Developer Zone in
> > the navigation tabs, scroll down to DOCUMENTATION: Projects.
>
> I misunderstood something: I thought you wanted it moved over to the dev
> wiki,
> which is the first link on the DeveloperZone page.  I read the
> DeveloperZone
> page (on the main wiki) and thought you wanted the glossary moved over to
> the
> pages linked there, and since the first ones are for the Trac wiki, I
> (mis)understood you wanted the glossary pushed over there.  Sorry for the
> confusion.
>
> Cheers,
>
> f
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D103432&bid=3D230486&dat=
=3D121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>



--
William V. Baxter III
OLM Digital
Kono Dens Building Rm 302
1-8-8 Wakabayashi Setagaya-ku
Tokyo, Japan  154-0023
+81 (3) 3422-3380

Re: [Numpy-discussion] Is there a known problem compiling numpy with VC7? Success

[Pearu P. <pe...@sc...>]

On Thu, 9 Feb 2006, Tim Hochberg wrote:

> Tim Hochberg wrote:
>
>> Pearu Peterson wrote:
>> 
>>> 
>>> 
>>> On Thu, 9 Feb 2006, Tim Hochberg wrote:
>>> 
>>>> I had this fantasy that default_lib_dirs would get picked up 
>>>> automagically; however that does not happen. I still ended up putting:
>>>>
>>>>           from  numpy.distutils import system_info
>>>>           library_dirs = system_info.default_lib_dirs
>>>>           result = config_cmd.try_run(tc,include_dirs=[python_include], 
>>>> library_dirs=library_dirs)
>>>> 
>>>> into setup.py. Is that acceptable? It's not very elegant.
>>> 
>>> 
>>> 
>>> No, don't use system_info.default_lib_dirs.
>>> 
>>> Use distutils.sysconfig.get_python_lib() to get the directory that 
>>> contains Python library.
>> 
>> 
>> That's the wrong library. Get_python_lib gives you the location of the 
>> python standard library, not the location of python24.lib. The former being 
>> python24/Lib (or python24/Lib/site-packages depending what options you feed 
>> get_python_lib) 'and the latter being python24/libs on my box.

Ok, but using system_info.default_lib_dirs is still wrong, this list is 
not designed for this purpose..

> To follow up on this a little bit, I investigated how distutils itself finds 
> python24.lib. It turns out that it is in build_ext.py, near line 168. The 
> relevant code is:
>
>       # also Python's library directory must be appended to library_dirs
>       if os.name == 'nt':
>           self.library_dirs.append(os.path.join(sys.exec_prefix, 'libs'))

Hmm, this should be effective also for numpy.distutils. self.library_dirs 
and other such attributes are used in distutils.command.build_ext.run() 
method while our numpy.distutils.command.build_ext.run() doesn't. So, I we 
have to do is to update numpy.distutils.command.build_ext.run() method to 
resolve this issue. This should fix also rpath issues that was reported on 
this list for certain platforms. I'll look fixing it today..

Pearu

Re: [Numpy-discussion] arange(start, stop, step) and floating point (Ticket #8)

[Travis O. <oli...@ie...>]

Sasha wrote:

>Well, my results are different.
>
>SVN r2087:
>  
>
>>python -m timeit -s "from numpy import arange" "arange(10000.0)"
>>    
>>
>10000 loops, best of 3: 21.1 usec per loop
>
>SVN r2088:
>  
>
>>python -m timeit -s "from numpy import arange" "arange(10000.0)"
>>    
>>
>10000 loops, best of 3: 25.6 usec per loop
>
>I am using gcc version 3.3.4 with the following flags: -msse2
>-mfpmath=sse -fno-strict-aliasing -DNDEBUG -g -O3.
>
>The timing is consistent with the change in the DOUBLE_fill loop:
>
>r2087:
>   1b8f0:       f2 0f 11 08             movsd  %xmm1,(%eax)
>   1b8f4:       f2 0f 58 ca             addsd  %xmm2,%xmm1
>   1b8f8:       83 c0 08                add    $0x8,%eax
>   1b8fb:       39 c8                   cmp    %ecx,%eax
>   1b8fd:       72 f1                   jb     1b8f0 <DOUBLE_fill+0x30>
>
>r2088:
>   1b9d0:       f2 0f 2a c2             cvtsi2sd %edx,%xmm0
>   1b9d4:       42                      inc    %edx
>   1b9d5:       f2 0f 59 c1             mulsd  %xmm1,%xmm0
>   1b9d9:       f2 0f 58 c2             addsd  %xmm2,%xmm0
>   1b9dd:       f2 0f 11 00             movsd  %xmm0,(%eax)
>   1b9e1:       83 c0 08                add    $0x8,%eax
>   1b9e4:       39 ca                   cmp    %ecx,%edx
>   1b9e6:       7c e8                   jl     1b9d0 <DOUBLE_fill+0x20>
>
>  
>
Nice to see some real hacking on this list :-)

>My change may be worth commiting because C code is shorter and
>arguably more understandable (at least by Fortran addicts :-). 
>Travis?
>  
>
Yes, I think it's worth submitting.  Most of the suggestions for 
pointer-arithmetic for fast C-code were developed when processors spent 
more time computing than fetching memory.  Now it seem it's all about 
fetching memory intelligently.

The buffer[i]=

style is even recommended according to the AMD-optimization book Sasha 
pointed out.

So, I say go ahead unless somebody can point out something we are missing...

-Travis

Re: [Numpy-discussion] Using ndarray for 2-dimensional, heterogeneous data

[N. V. <mit...@we...>]

N. Volbers wrote:

> Hello everyone,
>
> I am re-thinking the design of my evaluation software, but I am not 
> quite sure if I am doing the right decision, so let me state my problem:
>
> I am writing a simple evaluation program to read scientific (ASCII) 
> data and plot it both via gnuplot and matplotlib. The data is 
> typically very simple: numbers arranged in columns. Before numpy I was 
> using Numeric arrays to store this data in a list of 1-dimensional 
> arrays, e.g.:
>
> a =  [ array([1,2,3,4]), array([2.3,17.2,19.1,22.2]) ]
>
> This layout made it very easy to add, remove or rearrange columns, 
> because these were simple list operations. It also had the nice effect 
> to allow different data types for different columns. However, row 
> access was hard and I had to use my own iterator object to do so.
>
> When I read about heterogeneous arrays in numpy I started a new 
> implementation which would store the same data as above like this:
>
> b = numpy.array( [(1,2,3,4), (2.3,17.2,19.1,22.2)], 
> dtype={'names':['col1','col2'], 'formats': ['i2','f4']})
>
Sorry, I meant of course

   b = numpy.array( [(1,2.3), (2, 17.2), (3, 19.1), (4, 22.2)], 
dtype={'names':['col1','col2'], 'formats': ['i2','f4']})

> Row operations are much easier now, because I can use numpy's 
> intrinsic capabilities. However column operations require to create a 
> new array based on the old one.
>
> Now I am wondering if the use of such an array has more drawbacks that 
> I am not aware of. E.g. is it possible to mask values in such an array?
>
> And is it slower to get a certain column by using b['col1'] than it 
> would using a homogeneous array c and the notation c[:,0]?
>
> Does anyone else use such a data layout and can report on problems 
> with it?

The mathematical operations I want to use will be limited to operations 
acting on the column e.g. creating a new column = b['col1'] + b['col2'] 
and such. So of course I am aware of the basic difference that slicing 
works different if I have an heterogeneous array due to the fact that 
each row is considered a single item.

Niklas.

[Numpy-discussion] Using ndarray for 2-dimensional, heterogeneous data

[N. V. <mit...@we...>]

Hello everyone,

I am re-thinking the design of my evaluation software, but I am not 
quite sure if I am doing the right decision, so let me state my problem:

I am writing a simple evaluation program to read scientific (ASCII) data 
and plot it both via gnuplot and matplotlib. The data is typically very 
simple: numbers arranged in columns. Before numpy I was using Numeric 
arrays to store this data in a list of 1-dimensional arrays, e.g.:

 a =  [ array([1,2,3,4]), array([2.3,17.2,19.1,22.2]) ]

This layout made it very easy to add, remove or rearrange columns, 
because these were simple list operations. It also had the nice effect 
to allow different data types for different columns. However, row access 
was hard and I had to use my own iterator object to do so.

When I read about heterogeneous arrays in numpy I started a new 
implementation which would store the same data as above like this:

 b = numpy.array( [(1,2,3,4), (2.3,17.2,19.1,22.2)], 
dtype={'names':['col1','col2'], 'formats': ['i2','f4']})

Row operations are much easier now, because I can use numpy's intrinsic 
capabilities. However column operations require to create a new array 
based on the old one.

Now I am wondering if the use of such an array has more drawbacks that I 
am not aware of. E.g. is it possible to mask values in such an array?

And is it slower to get a certain column by using b['col1'] than it 
would using a homogeneous array c and the notation c[:,0]?

Does anyone else use such a data layout and can report on problems with it?

Best regards,

Niklas Volbers.

Re: [Numpy-discussion] Re: NumPy Glossary Was:Matlab page on scipy wiki

[Fernando P. <Fer...@co...>]

Joe Harrington wrote:

>>Why? Did you read the argument I made for putting it on the main wiki? How are 
>>you going to get contributions on the dev wiki once anonymous edits are locked 
>>out (which they will hopefully be very soon, before the wiki is spammed out of 
>>recognition)?
> 
> 
> Fernando, that page *is* on the main wiki (I don't deal with the
> developers' wiki at all).  Go to scipy.org, click on Developer Zone in
> the navigation tabs, scroll down to DOCUMENTATION: Projects.

I misunderstood something: I thought you wanted it moved over to the dev wiki, 
which is the first link on the DeveloperZone page.  I read the DeveloperZone 
page (on the main wiki) and thought you wanted the glossary moved over to the 
pages linked there, and since the first ones are for the Trac wiki, I 
(mis)understood you wanted the glossary pushed over there.  Sorry for the 
confusion.

Cheers,

f

Re: [Numpy-discussion] Re: NumPy Glossary Was:Matlab page on scipy wiki

[Joe H. <jh...@oo...>]

>>>http://scipy.org/NumPyGlossary
>> 
>> 
>> In general, such things should be birthed on the Developer_Zone page.
>> This is where the front page directs people to go if they are
>> interested in contributing.  We're getting a lot of new interest now,
>> so posting hidden pages on the mailing list will miss new talent.
>> Items can move to Documentation (or wherever) when they are somewhat
>> stable, and continue to grow there.

>Why? Did you read the argument I made for putting it on the main wiki? How are 
>you going to get contributions on the dev wiki once anonymous edits are locked 
>out (which they will hopefully be very soon, before the wiki is spammed out of 
>recognition)?

Fernando, that page *is* on the main wiki (I don't deal with the
developers' wiki at all).  Go to scipy.org, click on Developer Zone in
the navigation tabs, scroll down to DOCUMENTATION: Projects.

There are two reasons to put it there.  First, there are now many
people who are looking for projects to do.  This is where we can list
stuff we want to call attention to as needing work.  Once someone is
happy with it, they can link it from the Documentation page as well,
but it should also stay in Developer Zone until it's mature enough
that we'd rather people spent their time on other projects.  This is
the "work on me first" page.

Second, it might not belong on the Documentation page until it gets at
least a little review for scope, correctness, and readability.
Remember that too many stubs and apologies for being under
construction will turn people away.
 
> The less friction and committee-ness we impose on this whole thing, the better 
> of we'll all be.  Let's be _less_ bureaucratic, not more.

It just takes one happy person (you?) to link it under Documentation
(and one unhappy person to take it off, but that won't be me, in this
case).

--jh--

Re[2]: [Numpy-discussion] Matlab page on scipy wiki

[Alan G I. <ai...@am...>]

On Fri, 10 Feb 2006, Vidar Gundersen apparently wrote: 
> i guess when first used a GPL/GFDL it always has to be?  

If you own the copyright, you can license it anyway you want 
at any time.  You have already licensed it under the GFDL,
but you can license it other ways as well.

> i also considered CC, but didn't want to spend a lot of 
> time digging into legal stuff: i wanted to make the 
> reference available and reusable to anyone. 

If that is really the goal, then just include a statement 
placing it in the public domain.  E.g.,
Copyright: This document has been placed in the public domain.
If you want attribution, use an attribution license:
http://creativecommons.org/licenses/by/2.5/
(Be sure to say what you want as attribution.)

Cheers,
Alan Isaac

PS IANAL!

Re: [Numpy-discussion] Re: NumPy Glossary Was:Matlab page on scipy wiki

[Fernando P. <Fer...@co...>]

Joe Harrington wrote:
>>http://scipy.org/NumPyGlossary
> 
> 
> In general, such things should be birthed on the Developer_Zone page.
> This is where the front page directs people to go if they are
> interested in contributing.  We're getting a lot of new interest now,
> so posting hidden pages on the mailing list will miss new talent.
> Items can move to Documentation (or wherever) when they are somewhat
> stable, and continue to grow there.

Why? Did you read the argument I made for putting it on the main wiki? How are 
you going to get contributions on the dev wiki once anonymous edits are locked 
out (which they will hopefully be very soon, before the wiki is spammed out of 
recognition)?

The less friction and committee-ness we impose on this whole thing, the better 
of we'll all be.  Let's be _less_ bureaucratic, not more.

f

[Numpy-discussion] Re: NumPy Glossary Was:Matlab page on scipy wiki

[Joe H. <jh...@oo...>]

> http://scipy.org/NumPyGlossary

In general, such things should be birthed on the Developer_Zone page.
This is where the front page directs people to go if they are
interested in contributing.  We're getting a lot of new interest now,
so posting hidden pages on the mailing list will miss new talent.
Items can move to Documentation (or wherever) when they are somewhat
stable, and continue to grow there.

There's now a link for this page under the heading DOCUMENTATION:
Projects on the Developer_Zone page.

--jh--

Re: [Numpy-discussion] arange(start, stop, step) and floating point (Ticket #8)

[Sasha <nd...@ma...>]

Well, my results are different.

SVN r2087:
> python -m timeit -s "from numpy import arange" "arange(10000.0)"
10000 loops, best of 3: 21.1 usec per loop

SVN r2088:
> python -m timeit -s "from numpy import arange" "arange(10000.0)"
10000 loops, best of 3: 25.6 usec per loop

I am using gcc version 3.3.4 with the following flags: -msse2
-mfpmath=3Dsse -fno-strict-aliasing -DNDEBUG -g -O3.

The timing is consistent with the change in the DOUBLE_fill loop:

r2087:
   1b8f0:       f2 0f 11 08             movsd  %xmm1,(%eax)
   1b8f4:       f2 0f 58 ca             addsd  %xmm2,%xmm1
   1b8f8:       83 c0 08                add    $0x8,%eax
   1b8fb:       39 c8                   cmp    %ecx,%eax
   1b8fd:       72 f1                   jb     1b8f0 <DOUBLE_fill+0x30>

r2088:
   1b9d0:       f2 0f 2a c2             cvtsi2sd %edx,%xmm0
   1b9d4:       42                      inc    %edx
   1b9d5:       f2 0f 59 c1             mulsd  %xmm1,%xmm0
   1b9d9:       f2 0f 58 c2             addsd  %xmm2,%xmm0
   1b9dd:       f2 0f 11 00             movsd  %xmm0,(%eax)
   1b9e1:       83 c0 08                add    $0x8,%eax
   1b9e4:       39 ca                   cmp    %ecx,%edx
   1b9e6:       7c e8                   jl     1b9d0 <DOUBLE_fill+0x20>


The loop was 5 instructions before the change and 8 instructions
after.  It is possible that 387 FPU may do addition and multiplication
in parallel and this is why you don't see the difference.

Nevetheless, I would like to withdraw my prior objections.  I think
the code is now more numerically correct and that is worth the
slow-down on some platforms.

By the way, as I was playing with the code. I've also tried the
recommendation of using a[i] instead of *p:

--- numpy/core/src/arraytypes.inc.src   (revision 2088)
+++ numpy/core/src/arraytypes.inc.src   (working copy)
@@ -1652,9 +1652,8 @@
        @typ@ start =3D buffer[0];
        @typ@ delta =3D buffer[1];
        delta -=3D start;
-       buffer +=3D 2;
-       for (i=3D2; i<length; i++, buffer++) {
-               *buffer =3D start + i*delta;
+       for (i=3D2; i!=3Dlength; ++i) {
+               buffer[i] =3D start + i*delta;
        }
 }

The resulting optimized code for the loop was:

   1b9d0:       f2 0f 2a c0             cvtsi2sd %eax,%xmm0
   1b9d4:       f2 0f 59 c1             mulsd  %xmm1,%xmm0
   1b9d8:       f2 0f 58 c2             addsd  %xmm2,%xmm0
   1b9dc:       f2 0f 11 04 c2          movsd  %xmm0,(%edx,%eax,8)
   1b9e1:       40                      inc    %eax
   1b9e2:       39 c8                   cmp    %ecx,%eax
   1b9e4:       75 ea                   jne    1b9d0 <DOUBLE_fill+0x20>

This is one instruction less because "add    $0x8,%eax" was eliminated
and all pointer arithmetics and store (buffer[i] =3D ...) is now done in
a single instruction "movsd  %xmm0,(%edx,%eax,8)".

The timing, however did not change:

> python -m timeit -s "from numpy import arange" "arange(10000.0)"
10000 loops, best of 3: 25.6 usec per loop

My change may be worth commiting because C code is shorter and
arguably more understandable (at least by Fortran addicts :-).=20
Travis?



On 2/9/06, Tim Hochberg <tim...@co...> wrote:
> # baseline
> arange(10000.0) took 4.39404812623 seconds for 100000 reps

> # multiply instead of repeated add.
> arange(10000.0) took 4.34652784083 seconds for 100000 reps

Re: [Numpy-discussion] NumPy Glossary Was:Matlab page on scipy wiki

[Sasha <nd...@ma...>]

On 2/9/06, Fernando Perez <Fer...@co...> wrote:
> A humble suggestion: move it NOW.

Done. See <http://scipy.org/NumPyGlossary>.

Re: [Numpy-discussion] NumPy Glossary Was:Matlab page on scipy wiki

[Fernando P. <Fer...@co...>]

Sasha wrote:
> I've created a rough draft of NumPy Glossary on the developer's wiki
> <http://projects.scipy.org/scipy/numpy/wiki/NumPyGlossary>.  Please
> comment/edit.  When it is ready, we can move it to scipy.org.

A humble suggestion: move it NOW.  It will never 'be ready', and that's just 
the wiki way: put it in early, mark it at the top as a stub (so we don't 
falsely claim it to be in great shape when it isn't), and let it be improved 
in-place.

The trac wiki should be for developers to work on pure development things, and 
it requires an SVN login (well, it doesn't right now, but this should be 
changed ASAP: spammers WILL show up sooner or later, and they will destroy the 
wiki.  They did it to Enthought's and to IPython's in the past, they will also 
do it here; it's just a matter of time, and the cleanup later will be more 
work than running trac-admin now and closing wiki edit permissions for 
anonymous users).

The public wiki is where this content should be: a non-developer can do a 
perfectly good job of contributing content here, so there's no reason to keep 
this material in the dev wiki.

Cheers,

f

[Numpy-discussion] NumPy Glossary Was:Matlab page on scipy wiki

[Sasha <nd...@ma...>]

I've created a rough draft of NumPy Glossary on the developer's wiki
<http://projects.scipy.org/scipy/numpy/wiki/NumPyGlossary>.  Please
comment/edit.  When it is ready, we can move it to scipy.org.

Re[2]: [Numpy-discussion] Matlab page on scipy wiki

[Alan G I. <ai...@am...>]

On Fri, 10 Feb 2006, Bill Baxter apparently wrote:=20
> Some kind soul added 'svd' and 'inv' to the NumPy/SciPi=20
> columns, but those don't seem to be defined, at least for=20
> the versions of NumPy/SciPy that I have.

Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit=20
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy.linalg import svd, inv
>>>

hth,
Alan Isaac

Re: [Numpy-discussion] Matlab page on scipy wiki

[Fernando P. <Fer...@co...>]

Bill Baxter wrote:
> For what it's worth, matlab's rank function just calls svd, and returns the
> number singular values greater than a tolerance.  The implementation is a
> whopping 5 lines long.

Yup, and it would be pretty much the same 5 lines in numpy, with the same 
semantics.

Here's a quick and dirty implementation for old-scipy (I don't have new-scipy 
on this box):

def matrix_rank(arr,tol=1e-8):
     """Return the matrix rank of an input array."""

     arr = scipy.asarray(arr)

     if len(arr.shape) != 2:
         raise ValueError('Input must be a 2-d array or Matrix object')

     svdvals = scipy.linalg.svdvals(arr)
     return sum(scipy.where(svdvals>tol,1,0))


If you really hate readability and error-checking, it's a one-liner :)

matrix_rank = lambda arr,tol=1e-8: 
sum(scipy.where(scipy.linalg.svdvals(arr)>tol,1,0))


Looks OK (RA is RandomArray from Numeric):

In [21]: matrix_rank([[1,0],[0,0]])
Out[21]: 1

In [22]: matrix_rank(RA.random((3,3)))
Out[22]: 3

In [23]: matrix_rank([[1,0],[0,0]])
Out[23]: 1

In [24]: matrix_rank([[1,0],[1,0]])
Out[24]: 1

In [25]: matrix_rank([[1,0],[0,1]])
Out[25]: 2

In [26]: matrix_rank(RA.random((3,3)),1e-1)
Out[26]: 2

In [48]: matrix_rank([[1,0],[1,1e-8]])
Out[48]: 1

In [49]: matrix_rank([[1,0],[1,1e-4]])
Out[49]: 2

In [50]: matrix_rank([[1,0],[1,1e-8]],1e-9)
Out[50]: 2


Cheers,

f

Re: [Numpy-discussion] Matlab page on scipy wiki

[Bill B. <wb...@gm...>]

Anyone know what the terms are for redistribution of applications built wit=
h
Matlab?  I searched around their site a bit but couldn't find anything
conclusive.  One page seemed to be saying there was a per-application fee
for distributing a matlab-based application, but other pages made it sound
more like it was no extra charge.   If the former, then that's another poin=
t
that should go in the 'key differences' section.

--bb

Re: [Numpy-discussion] Matlab page on scipy wiki

[Bill B. <wb...@gm...>]

For what it's worth, matlab's rank function just calls svd, and returns the
number singular values greater than a tolerance.  The implementation is a
whopping 5 lines long.

On 2/10/06, Fernando Perez <Fer...@co...> wrote:
>
> Since numpy is a n-dimensional array package, it may be convenient to
> introduce a matrix_rank() routine which does what matlab's rank() for 2-d
> arrays and matrices, while raising an error for any other shape.  This
> would
> also make it explicit that this operation is only well-defined for 2-d
> objects.


Or put it in numpy.linalg, which also makes it pretty clear what the scope
is.

--bb

Re: [Numpy-discussion] Is there a known problem compiling numpy with VC7? Success

[Tim H. <tim...@co...>]

Tim Hochberg wrote:

> Pearu Peterson wrote:
>
>>
>>
>> On Thu, 9 Feb 2006, Tim Hochberg wrote:
>>
>>> I had this fantasy that default_lib_dirs would get picked up 
>>> automagically; however that does not happen. I still ended up putting:
>>>
>>>           from  numpy.distutils import system_info
>>>           library_dirs = system_info.default_lib_dirs
>>>           result = 
>>> config_cmd.try_run(tc,include_dirs=[python_include], 
>>> library_dirs=library_dirs)
>>>
>>> into setup.py. Is that acceptable? It's not very elegant.
>>
>>
>>
>> No, don't use system_info.default_lib_dirs.
>>
>> Use distutils.sysconfig.get_python_lib() to get the directory that 
>> contains Python library.
>
>
> That's the wrong library. Get_python_lib gives you the location of the 
> python standard library, not the location of python24.lib. The former 
> being python24/Lib (or python24/Lib/site-packages depending what 
> options you feed get_python_lib) 'and the latter being python24/libs 
> on my box.

To follow up on this a little bit, I investigated how distutils itself 
finds python24.lib. It turns out that it is in build_ext.py, near line 
168. The relevant code is:

        # also Python's library directory must be appended to library_dirs
        if os.name == 'nt':
            self.library_dirs.append(os.path.join(sys.exec_prefix, 'libs'))

Unfortunately, there's no obvious, clean way to extract the library 
information from there. You can grab it using the following magic formula:

            from distutils.core import Distribution
            from distutils.command import build_ext
            be = build_ext.build_ext(Distribution())
            be.finalize_options()
            librarys_dirs = be.library_dirs

However, that seems worse than what we're doing now. I haven't actually 
tried this in the code either -- for all I know instantiating an extra 
Distribution may have some horrible side effect that I don't know about. 
If someone can come up with a cleaner way to get to this info, that'd be 
great, otherwise I'd say we might as well just keep things as they are 
for the time being.

Regards,

-tim

Re: [Numpy-discussion] Matlab page on scipy wiki

[Fernando P. <Fer...@co...>]

[ regarding the way of describing arrays vs. matlab's matrices, and the use of 
'dimension', 'rank', 'number of axes', etc.]

Let's not introduce new terms where none are needed.  These concepts have had 
well-established names (tensor rank and matrix rank) for a long time.  It may 
be a good idea to add a local glossary page reminding anyone of what the 
definitions are, but for as long as I remember reading literature on these 
topics, the two terms have been fully unambiguous.

A numpy array with length(array.shape)==d is closest to a rank d tensor (minus 
the geometric co/contravariance information).  A d=2 array can be used to 
represent a matrix, and linear algebra operations can be performed on it; if a 
Matrix object is built out of it, a number of things (notably the * operator) 
are then performed in the linear algebra sense (and not element-wise).  The 
rank of a matrix has nothing to do with the shape attribute of the underlying 
array, but with the number of non-zero singular values (and for floating-point 
matrices, is best defined up to a given tolerance).

Since numpy is a n-dimensional array package, it may be convenient to 
introduce a matrix_rank() routine which does what matlab's rank() for 2-d 
arrays and matrices, while raising an error for any other shape.  This would 
also make it explicit that this operation is only well-defined for 2-d objects.

My 1e-2,

f