numpy-discussion

RE: [Numpy-discussion] RE: default axis for numarray

[Perry G. <pe...@st...>]

<Eric Jones writes>:
   <Konrad Hinsen writes>:

> > What needs to be improved in that area?
> 
> Comparisons of complex numbers.  But lets save that debate for later.
> 
 
No, no, let's do it now. ;-) We for one would like to know for 
numarray what should be done.

If I might be presumptious enough to anticipate what Eric would
say, it is that complex comparisons should be allowed, and that
they use all the information in the complex number (real and imaginary)
so that they lead to consistent results in sorting.

But the purist argues that comparisons for complex numbers are
meaningless. Well, yes, but there are cases in code where you 
don't which such comparisons to cause an exception. But even
more important, there is at least one case which is practical.
It isn't all that uncommon to want to eliminate duplicate values
from arrays, and one would like to be able to do that for 
complex values as well. A common technique is to sort the values
and then eliminate all identical adjacent values. A predictable
comparison rule would allow that to be easily implemented.

Eric, am I missing anything in this? It should be obvious that we
agree with his position, but I am wondering if there are any arguments
we have not heard yet that outweigh the advantages we see.

Perry

RE: [Numpy-discussion] RE: default axis for numarray

[eric j. <er...@en...>]

> From: cbarker@localhost.localdomain
[mailto:cbarker@localhost.localdomain]
> 
> eric jones wrote:
> > The default axis choice influences how people choose to lay out
their
> > data in arrays.  If the default is to sum down columns, then users
lay
> > out their data so that this is the order of computation.
> 
> This is absolutely true. I definitely choose my data layout to that
the
> various rank reducing operators do what I want. Another reason to have
> consistency. So I don't really care which way is default, so the
default
> might as well be the better performing option.
> 
> Of course, compatibility with previous versions is helpful
too...arrrgg!
> 
> What kind of a performance difference are we talking here anyway?

Guess I ought to test instead of just saying it is so...

I ran the following test of summing 200 sets of 10000 
numbers.  I expected a speed-up of about 2... I didn't get it.  They 
are pretty much the same speed on my machine.?? (more later)

C:\WINDOWS\system32>python
ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from Numeric import *
>>> import time
>>> a = ones((10000,200),Float) * arange(10000)[:,NewAxis]
>>> b = ones((200,10000),Float) * arange(10000)[NewAxis,:]
>>> t1 = time.clock();x=sum(a,axis=0);t2 = time.clock();print t2-t1
0.0772411018719
>>> t1 = time.clock();x=sum(b,axis=-1);t2 = time.clock();print t2-t1
0.079615705348

I also tried FFT, and did see a difference -- a speed up of 1.5+:

>>> q = ones((1024,1024),Float)
>>> t1 = time.clock();x = FFT.fft(q,axis=0);t2 = time.clock();print
t2-t1
0.907373143793
>>> t1 = time.clock();x= FFT.fft(q,axis=-1);t2 = time.clock();print
t2-t1
0.581641800843
>>> .907/.581
1.5611015490533564

Same in scipy

>>> from scipy import *
>>> a = ones((1024,1024),Float)
>>> import time
>>> t1 = time.clock(); q = fft(a,axis=0); t2 = time.clock();print t2-t1
0.870259488287
>>> t1 = time.clock(); q = fft(a,axis=-1); t2 = time.clock();print t2-t1
0.489512214541
>>> t1 = time.clock(); q = fft(a,axis=0); t2 = time.clock();print t2-t1
0.849266317367
>>> .849/.489
1.7361963190184049

So why is sum() the same speed for both cases? I don't know.  I wrote
a quick C program that is similar to how Numeric loops work, and I saw 
about a factor of 4 improvement by summing rows instead columns:

C:\home\eric\wrk\axis_speed>gcc -O2 axis.c
C:\home\eric\wrk\axis_speed>a
summing rows (sec): 0.040000
summing columns (sec): 0.160000
pass

These numbers are more like what I expected to see in the Numeric tests,

but they are strange when compared to the Numeric timings -- the row sum

is twice as fast as Numeric while the column sum is twice as slow.  
Because all the work is done in C and we're summing reasonably long 
arrays, the Numeric and C versions should be roughly the same speed.  
I can understand why summing rows is twice as fast in my C routine -- 
the Numeric loop code is not going to win awards for being optimal.  
What I don't understand is why the column summation is twice as slow in
my C code as in Numeric.  This should not be.  I've posted it below in 
case someone can enlighten me.

I think in general, you should see a speed up of 1.5+ when the summing
over the "faster" axis.  This holds true for fft in Python and my sum
in C.  As to why I don't in Numeric's sum(), I'm not sure.  It is 
certainly true that non-strided access makes the best use of 
cache and *usually* is faster.

eric


------------------------------------------------------------------------
--
#include <malloc.h>
#include <time.h>

int main()
{
    double *a, *sum1, *sum2;
    int i, j, si, sj, ind, I, J;
    int small=200, big=10000;
    time_t t1, t2;
    
    I = small;
    J = big;
    si = big;
    sj = 1;
    a = (double*)malloc(I*J*sizeof(double));
    sum1 = (double*)malloc(small*sizeof(double));
    sum2 = (double*)malloc(small*sizeof(double));
    
    //set memory
    for(i = 0; i < I; i++)
    {
        sum1[i] = 0;
        sum2[i] = 0;
        ind = si * i;
        for(j = 0; j < J; j++)
        {
            a[ind] = (double)j;            
            ind += sj;            
        }
        ind += si;
    }
    
    t1 = clock();
    for(i = 0; i < I; i++)
    {
        sum1[i] = 0;
        ind = si * i;
        for(j = 0; j < J; j++)
        {
            sum1[i] += a[ind];            
            ind += sj;            
        }
        ind += si;
    }
    t2 = clock();
    printf("summing rows (sec): %f\n", (t2-t1)/(float)CLOCKS_PER_SEC);

    
    I = big;
    J = small;
    sj = big;
    si = 1;
    t1 = clock();

    //set memory
    for(i = 0; i < I; i++)
    {        
        ind = si * i;
        for(j = 0; j < J; j++)
        {
            a[ind] = (double)i;            
            ind += sj;            
        }
        ind += si;
    }

    for(j = 0; j < J; j++)
    {
        sum2[j] = 0;
        ind = sj * j;
        for(i = 0; i < I; i++)
        {
            sum2[j] += a[ind];               
            ind += si;            
        }
    }
    t2 = clock();
    printf("summing columns (sec): %f\n",
(t2-t1)/(float)CLOCKS_PER_SEC);
    
    for (i=0; i < small; i++)
    {
        if(sum1[i] != sum2[i])
            printf("failure %d, %f %f\n", i, sum1[i], sum2[i]);
    }            
    printf("pass %f\n", sum1[0]);
    return 0;
}

RE: [Numpy-discussion] RE: default axis for numarray

[Paul F D. <pa...@pf...>]

One can make a case for allowing == and != for complex arrays, but >
just doesn't make sense and should not be allowed.


> -----Original Message-----
> From: num...@li... 
> [mailto:num...@li...] On 
> Behalf Of Perry Greenfield
> Sent: Tuesday, June 11, 2002 11:52 AM
> To: eric jones; 'Konrad Hinsen'
> Cc: num...@li...
> Subject: RE: [Numpy-discussion] RE: default axis for numarray
> 
> 
> <Eric Jones writes>:
>    <Konrad Hinsen writes>:
> 
> > > What needs to be improved in that area?
> > 
> > Comparisons of complex numbers.  But lets save that debate 
> for later.
> > 
>  
> No, no, let's do it now. ;-) We for one would like to know for 
> numarray what should be done.
> 
> If I might be presumptious enough to anticipate what Eric 
> would say, it is that complex comparisons should be allowed, 
> and that they use all the information in the complex number 
> (real and imaginary) so that they lead to consistent results 
> in sorting.
> 
> But the purist argues that comparisons for complex numbers 
> are meaningless. Well, yes, but there are cases in code where you 
> don't which such comparisons to cause an exception. But even 
> more important, there is at least one case which is 
> practical. It isn't all that uncommon to want to eliminate 
> duplicate values from arrays, and one would like to be able 
> to do that for 
> complex values as well. A common technique is to sort the 
> values and then eliminate all identical adjacent values. A 
> predictable comparison rule would allow that to be easily implemented.
> 
> Eric, am I missing anything in this? It should be obvious 
> that we agree with his position, but I am wondering if there 
> are any arguments we have not heard yet that outweigh the 
> advantages we see.
> 
> Perry
> 
> _______________________________________________________________
> 
> Multimillion Dollar Computer Inventory
> Live Webcast Auctions Thru Aug. 2002 - 
> http://www.cowanalexander.com/calendar
> 
> 
> 
> 
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

Re: [Numpy-discussion] RE: default axis for numarray

[Scott R. <ra...@ph...>]

On June 11, 2002 04:56 pm, you wrote:
> One can make a case for allowing == and != for complex arrays, but >
> just doesn't make sense and should not be allowed.

It depends if you think of complex numbers in phasor form or not.  In phasor 
form, the amplitude of the complex number is certainly something that you 
could compare with > or < -- and in my opinion, that seems like a reasonable 
comparison.  You _could_ do the same thing with the phases, except you run 
into the modulo 2pi thing...

Scott

> > -----Original Message-----
> > From: num...@li...
> > [mailto:num...@li...] On
> > Behalf Of Perry Greenfield
> > Sent: Tuesday, June 11, 2002 11:52 AM
> > To: eric jones; 'Konrad Hinsen'
> > Cc: num...@li...
> > Subject: RE: [Numpy-discussion] RE: default axis for numarray
> >
> >
> > <Eric Jones writes>:
> >
> >    <Konrad Hinsen writes>:
> > > > What needs to be improved in that area?
> > >
> > > Comparisons of complex numbers.  But lets save that debate
> >
> > for later.
> >
> >
> > No, no, let's do it now. ;-) We for one would like to know for
> > numarray what should be done.
> >
> > If I might be presumptious enough to anticipate what Eric
> > would say, it is that complex comparisons should be allowed,
> > and that they use all the information in the complex number
> > (real and imaginary) so that they lead to consistent results
> > in sorting.
> >
> > But the purist argues that comparisons for complex numbers
> > are meaningless. Well, yes, but there are cases in code where you
> > don't which such comparisons to cause an exception. But even
> > more important, there is at least one case which is
> > practical. It isn't all that uncommon to want to eliminate
> > duplicate values from arrays, and one would like to be able
> > to do that for
> > complex values as well. A common technique is to sort the
> > values and then eliminate all identical adjacent values. A
> > predictable comparison rule would allow that to be easily implemented.
> >
> > Eric, am I missing anything in this? It should be obvious
> > that we agree with his position, but I am wondering if there
> > are any arguments we have not heard yet that outweigh the
> > advantages we see.
> >
> > Perry
> >
> > _______________________________________________________________
> >
> > Multimillion Dollar Computer Inventory
> > Live Webcast Auctions Thru Aug. 2002 -
> > http://www.cowanalexander.com/calendar
> >
> >
> >
> >
> > _______________________________________________
> > Numpy-discussion mailing list Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
> _______________________________________________________________
>
> Multimillion Dollar Computer Inventory
> Live Webcast Auctions Thru Aug. 2002 -
> http://www.cowanalexander.com/calendar
>
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
Scott M. Ransom              Address:  McGill Univ. Physics Dept.
Phone:  (514) 398-6492                 3600 University St., Rm 338
email:  ra...@ph...       Montreal, QC  Canada H3A 2T8 
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989

Re: [Numpy-discussion] RE: default axis for numarray

[Konrad H. <hi...@cn...>]

Scott Ransom <ra...@ph...> writes:

> On June 11, 2002 04:56 pm, you wrote:
> > One can make a case for allowing == and != for complex arrays, but >
> > just doesn't make sense and should not be allowed.
> 
> It depends if you think of complex numbers in phasor form or not.  In phasor 
> form, the amplitude of the complex number is certainly something that you 
> could compare with > or < -- and in my opinion, that seems like a reasonable 

Sure, but that doesn't give a full order relation for complex numbers.
Two different numbers with equal magnitude would be neither equal nor
would one be larger than the other.

I agree with Paul that complex comparison should not be allowed. On the
other hand, Perry's argument about sorting makes sense as well. Is there
anything that prevents us from permitting arraysort() on complex arrays
but not the comparison operators?

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RE: default axis for numarray

[Scott R. <ra...@ph...>]

On Wed, Jun 12, 2002 at 10:32:12AM +0200, Konrad Hinsen wrote:
> Scott Ransom <ra...@ph...> writes:
> 
> > On June 11, 2002 04:56 pm, you wrote:
> > > One can make a case for allowing == and != for complex arrays, but >
> > > just doesn't make sense and should not be allowed.
> > 
> > It depends if you think of complex numbers in phasor form or not.  In phasor 
> > form, the amplitude of the complex number is certainly something that you 
> > could compare with > or < -- and in my opinion, that seems like a reasonable 
> 
> Sure, but that doesn't give a full order relation for complex numbers.
> Two different numbers with equal magnitude would be neither equal nor
> would one be larger than the other.

The comparison operators could be defined to operate on the
magnitudes only.  In this case you would get the kind of ugly
result that two complex numbers with the same magnitude but
different phases would be equal.

Complex comparisons of this type could be quite useful to those
(like me) who are do lots of Fourier domain signal processing.

> I agree with Paul that complex comparison should not be allowed. On the
> other hand, Perry's argument about sorting makes sense as well. Is there
> anything that prevents us from permitting arraysort() on complex arrays
> but not the comparison operators?

How do you sort an array of complex numbers if you can't compare them?

Scott

-- 
Scott M. Ransom              Address:  McGill Univ. Physics Dept.
Phone:  (514) 398-6492                 3600 University St., Rm 338
email:  ra...@ph...       Montreal, QC  Canada H3A 2T8 
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989

Re: [Numpy-discussion] RE: default axis for numarray

[Konrad H. <hi...@cn...>]

> The comparison operators could be defined to operate on the
> magnitudes only.  In this case you would get the kind of ugly
> result that two complex numbers with the same magnitude but
> different phases would be equal.

If you want to compare magnitudes, you can do that explicitly without
much effort.

> How do you sort an array of complex numbers if you can't compare them?

You could for example sort by real part first and by imaginary part
second. That would be a well-defined sort order, but not a useful
definition of comparison in the mathematical sense.

Konrad
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RE: default axis for numarray

[Pearu P. <pe...@ce...>]

On Wed, 12 Jun 2002, Konrad Hinsen wrote:

> > How do you sort an array of complex numbers if you can't compare them?
> 
> You could for example sort by real part first and by imaginary part
> second. That would be a well-defined sort order, but not a useful
> definition of comparison in the mathematical sense.

Releated discussion has been also in the scipy list. See the thread
starting in
 
http://www.scipy.org/site_content/mailman?fn=scipy-dev/2002-February/000364.html

But here I would like to draw your attention to the suggestion that
sort() function could take an optional argument that specifies the
comparison method for complex numbers (for real numbers they are all
equivalent). Here follows the releavant fragment of the message:
http://www.scipy.org/site_content/mailman?fn=scipy-dev/2002-February/000366.html

...
However, in different applications different conventions may be useful or
reasonable for ordering complex numbers. Whatever is the convention, their
mathematical correctness is irrelevant and this cannot be used as an
argument for prefering one convention to another.

I would propose providing number of efficient comparison methods for
complex (or any) numbers that users may use in sort functions as an
optional argument. For example,

scipy.sort([2,1+2j],cmpmth='abs') -> [1+2j,2]  # sorts by abs value
scipy.sort([2,1+2j],cmpmth='real') -> [2,1+2j] # sorts by real part
scipy.sort([2,1+2j],cmpmth='realimag') # sorts by real then by imag
scipy.sort([2,1+2j],cmpmth='imagreal') # sorts by imag then by real
scipy.sort([2,1+2j],cmpmth='absangle') # sorts by abs then by angle
etc.
scipy.sort([2,1+2j],cmpfunc=<user defined comparison function>)

Note that

scipy.sort([-1,1],cmpmth='absangle') -> [1,-1]

which also demonstrates the arbitrariness of sorting complex numbers.

...

Regards,
	Pearu

RE: [Numpy-discussion] RE: default axis for numarray

[Paul F D. <pa...@pf...>]

Using the term "comparison operators" is too loose and is causing a
communication problem here.

There are these comparison operators
== and != (group 1)

<, >, <=, and >= (group 2)

For complex numbers it is easy to define the operators in group 1: x ==
y iff x.real == y.real and x.imag == y.imag. And, x != y iff (not x ==
y).

I hardly think any other definition would be conceivable. The utility of
this definition is questionable, as in most instances one should be
making these comparisons with a tolerance, but there at least are cases
when it makes sense. 

For group 2, there are a variety of possible definitions. Just to name
three possible > definitions, the greater magnitude, the greater phase
mod 2pi,  or a radix-type order e.g., x >  y if x.real > y.real or
(x.real == y.real and x.imag > y.imag). 

A person can always define a function my_greater_than (c1, c2) to embody
one of these definitions, and use it as an argument to a sort routine
that takes a function argument to tell it how to sort. What you are
arguing about is whether some particular version of this comparison
should be "blessed" by attaching it to the operator ">". I do not think
one of the definitions is such a clear winner that it should be blessed
-- it would mean a casual reader could not guess what the operator
means, and ">" does not have a doc string. Therefore I oppose doing so.

[Numpy-discussion] Complex comparisions

[Travis O. <oli...@ie...>]

I'd be interested to know what IDL does?  Does it compare complex
numbers.

Matlab allows comparisons of complex numbers but just compares the real
part.  I think this is reasonable.  Often during a calculation of
limited precision one ends up with a complex number when the result is
in a "mathematically pure sense" real.  

I guess I trust the user to realize that if they are comparing numbers
they know what they mean --- (only real numbers are compared so the
complex part is ignored).  


-Travis

Re: [Numpy-discussion] Complex comparisions

[Rick W. <rl...@st...>]

On 12 Jun 2002, Travis Oliphant wrote:

> I'd be interested to know what IDL does?  Does it compare complex
> numbers.

Well, that was an interesting question with a surprising answer (at least
to me, a long-time IDL user):

(1) IDL allows comparisons of complex number using equality and
    inequality, but attempts to compare using GT, LT, etc. cause an
    illegal exception.

(2) IDL sorts complex numbers by the amplitude.  It ignores the phase.
    Numbers with the same amplitude and different phases are randomly
    ordered depending on their positions in the original array.

> Matlab allows comparisons of complex numbers but just compares the real
> part.  I think this is reasonable.  Often during a calculation of
> limited precision one ends up with a complex number when the result is
> in a "mathematically pure sense" real.

So neither IDL nor Matlab has what I consider the desirable feature
that the sort order be unique at least to the extent that equal values
wind up next to each other in the sorted array.  (Sorting by real value
and then, for equal real values, by imaginary value would accomplish
that.)  Since complex numbers can't be fully ordered there is no single
comparison function that can be plugged into a standard sort algorithm
and give that result -- it would require a special complex sort algorithm.

I guess if neither of the major array processing systems (that I know
about) have this property in their complex sorts, it must not be *that*
important.  And since I've been using IDL for 13 years without discovering
that complex greater-than comparisons are illegal, I guess that must not
be an important property either (at least to me :-).

My conclusion now is similar to Paul Dubois's suggestion -- we should
allow equality comparisons and sorting.  Beyond that I guess whatever
other people want should carry the day, since it clearly doesn't matter
to the sorts of things that I do with Numeric!
					Rick