numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] BLAS

[Alexander S. <a.s...@gm...>]

"Paul F Dubois" <pa...@pf...> writes:

> I had forgotten about this case. I think when these were done it was thought
> that it would be better if the Numeric core did not require use of
> LAPACK/BLAS. We were thinking back then of a core with other packages, and
> the blas we use by default is probably the same speed so it didn't seem
> important. I would have no problem with a patch to change this.

Great. I submitted a patch just now.

alex

Re: [Numpy-discussion] speedy remove mean of rows

[Eric J. <er...@en...>]

Hey John,

I think broadcasting is your best bet.  Here is a snippet using scipy (Numeric will be pretty much the same).

>>> from scipy import *
>>> a = stats.random((4,3))
a
array([[ 0.94058263,  0.24342623,  0.74673623],
       [ 0.53151542,  0.07523929,  0.49730805],
       [ 0.5161854 ,  0.51049614,  0.70360875],
       [ 0.09470515,  0.60604334,  0.64941102]])
>>> stats.mean(a) # axis=-1 by default in scipy
array([ 0.6435817 ,  0.36802092,  0.57676343,  0.45005317])
>>> a-stats.mean(a)[:,NewAxis]
array([[ 0.29700093, -0.40015546,  0.10315453],
       [ 0.1634945 , -0.29278163,  0.12928713],
       [-0.06057803, -0.06626729,  0.12684532],
       [-0.35534802,  0.15599017,  0.19935785]])

eric

John Hunter <jdh...@ac...> wrote ..
> 
> I have a large (40,000 x 128) Numeric array, X, with typecode Float.
> In some cases the number of rows may be approx 10x greater.
> 
> I want to create an array Y with the same dimensions as X, where each
> element of Y is the corresponding element of X with the mean of the
> row on which it occurs subtracted away.  Ie,
> 
>   Y = X - transpose(resize(mean(X,1), (X.shape[1],X.shape[0])))
> 
> I am wondering if this is the most efficient way (speed and memory).
> 
> Thanks for any suggestions,
> John Hunter
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] speedy remove mean of rows

[John H. <jdh...@ac...>]

I have a large (40,000 x 128) Numeric array, X, with typecode Float.
In some cases the number of rows may be approx 10x greater.

I want to create an array Y with the same dimensions as X, where each
element of Y is the corresponding element of X with the mean of the
row on which it occurs subtracted away.  Ie,

  Y = X - transpose(resize(mean(X,1), (X.shape[1],X.shape[0])))

I am wondering if this is the most efficient way (speed and memory).

Thanks for any suggestions,
John Hunter

[Numpy-discussion] Last call for v. 23

[Paul F. D. <du...@ll...>]

I am going to make a release of Numeric, 23.0. Fellow developers who are inspired to fix a bug are urged to do so immediately.

This will be a bug fix release. 

[Numpy-discussion] Re: filtering numeric arrays

[Gordon W. <g_...@cy...>]

Thanks to Tim and Chris.  Just what I was looking for!

I tested both along with some other tries that I had made.


For 10000 points -
G:\GPS\Python\GUI_Test\Filter>speed.py
time for <function listComp at 0x007AE4D8> is 0.022809
time for <function arraykludge at 0x0092AFF0> is 0.060303
time for <function arrayComp at 0x0092AFB0> is 0.055692
time for <function arrayTimH at 0x0092B030> is 0.003652
time for <function arrayChrisB at 0x0092B070> is 0.003561

For 100 points -
G:\GPS\Python\GUI_Test\Filter>speed.py
time for <function listComp at 0x007AE4D8> is 0.000238
time for <function arraykludge at 0x00804BD8> is 0.000784
time for <function arrayComp at 0x00804B98> is 0.000678
time for <function arrayTimH at 0x00804C18> is 0.000376
time for <function arrayChrisB at 0x00804C58> is 0.000153

They scale slightly differently between Tim's and Chris' methods.

Thanks again,

Gordon Williams

Here is the code (since someone will ask):

'''Test the speed of different methods of getting points out of a list'''

import time
import Numeric as n

size= 100
maxNum=size/10.

#data
a= n.array(n.arange(0,maxNum,.05))
a.shape= (size,2)

#list
l= a.tolist()


(xMin,yMin)= (3,2)
(xMax,yMax)= (4,6)

def listComp(seq):
    '''using list comprehension'''
    return [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]

def arrayComp(seq):
    '''using list comprehension'''
    return n.array([(x,y) for x,y in seq if xMin < x <xMax and yMin< y
<yMax])

def arraykludge(seq):
    '''using numeric stuff'''
    res= n.copy.deepcopy(seq)
    i=0 #index
    for x,y in seq:
        if xMin < x <xMax and yMin< y <yMax:
            res[i]= [x,y]
            i+=1
    return res[:i]

def arrayTimH(seq):
    '''using Tim Hochberg suggestions'''
    cond  = (xMin < seq[:,0]) & (seq[:,0] < xMax) & (yMin < seq[:,1]) &
(seq[:,1] < yMax)
    return n.compress(cond, seq, 0)

def arrayChrisB(seq):
    '''using Chris Barker suggestions'''
    valid = (seq[:,0] > xMin) & (seq[:,0] < xMax) & (seq[:,1] > yMin) &
(seq[:,1] < yMax)
    return n.take(a,n.nonzero(valid))


#Tests
tests= [(listComp,l), (arraykludge,a),(arrayComp,a), (arrayTimH,a),
(arrayChrisB,a)]

for fun,seq in tests:
    t=time.clock()
    apply(fun,(seq,))
    dt= time.clock()-t
    print "time for %s is %f" %(str(fun),dt)

----- Original Message -----
From: "Gordon Williams" <g_...@cy...>
To: <num...@li...>
Sent: Thursday, February 27, 2003 2:05 PM
Subject: filtering numeric arrays


> Hi All,
>
> I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I
would
> like to remove all the points in the array that don't meet the min/max
point
> criteria.  I will have several thousand points.  With lists I can do it
like
>
> [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]
>
> How do I get the same functionality and better speed using numeric.  I
have
> tried a bunch of things using compress and take but I am running up
against
> a brick wall.
>
>
> Any ideas?
>
> Thanks
>
> Gordon Williams
>
>
>
>

Re: [Numpy-discussion] filtering numeric arrays

[Chris B. <Chr...@no...>]

On Thursday, February 27, 2003, at 11:05 AM, Gordon Williams wrote:
> I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I 
> would
> like to remove all the points in the array that don't meet the min/max 
> point
> criteria.  I will have several thousand points.  With lists I can do 
> it like

This should do it:
 >>> a
array([[1, 3],
        [2, 4],
        [5, 6]])

 >>> valid = (a[:,0] > minX) & (a[:,0] < maxX) & (a[:,1] > minY) & 
(a[:,1] < maxY)
 >>> take(a,nonzero(valid))
array([       [2, 4]])

Note that & is a bitwise-and, not a logical and, but in this case, the 
result is the same. Unfortunately, the way Python works, overloading 
"and" is difficult.

-Chris


Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] filtering numeric arrays

[Tim H. <tim...@ie...>]

Gordon Williams wrote:

>Hi All,
>
>I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I would
>like to remove all the points in the array that don't meet the min/max point
>criteria.  I will have several thousand points.  With lists I can do it like
>
>[(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]
>
>How do I get the same functionality and better speed using numeric.  I have
>tried a bunch of things using compress and take but I am running up against
>a brick wall.
>  
>
I think you want something like this:

 >>> cond  = (xMin < a[:,0]) & (a[:,0] < xMax) & (yMin < a[:,1]) & 
(a[:,1] < yMax)
 >>> np.compress(cond, a, 0)

Where 'a' is your original Nx2 array. Unfortunately the obvious notation 
and prettier notation using (xMin < a[:,0] < xMax) fails because python 
treats that as "(xMin < a[:,0]) and (a[:,0] < xMax)" and "and" is not 
what you need here, '&' is.

-tim


>
>Any ideas?
>
>Thanks
>
>Gordon Williams
>
>
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>

[Numpy-discussion] filtering numeric arrays

[Gordon W. <g_...@cy...>]

Hi All,

I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I would
like to remove all the points in the array that don't meet the min/max point
criteria.  I will have several thousand points.  With lists I can do it like

[(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]

How do I get the same functionality and better speed using numeric.  I have
tried a bunch of things using compress and take but I am running up against
a brick wall.


Any ideas?

Thanks

Gordon Williams

Re: [Numpy-discussion] make C array accessible to python without copy

[Sebastian H. <ha...@ms...>]

Short follow up: 
1) Is it planned to support this more directly?
2) How much does it cost to create a buffer object if it 
uses my already allocated memory ?
3) Can I change the pointer so that it points to a 
different memory space WITHOUT having to recreate any 
python objects? Or would that "confuse" the buffer or 
numarray?    (We are hoping to aquire 30 images per second 
- the images should get written into a circular buffer so 
that the data can be written to disk in larger chunks - 
but the python array should always refer to the current 
image )

Thanks for all the nice toys (tools) ;-)
Sebastian Haase



On Fri, 17 Jan 2003 18:16:01 -0500
  Todd Miller <jm...@st...> wrote:
>Sebastian Haase wrote:
>
>>Hi,
>>What is the C API to make an array that got allocated,
>>let's say, by  a = new short[512*512],
>>accessible to python as numarray.
>>
>What you want to do is not currently supported well in C. 
> The way to do what you want is:
>
>1.  Create a buffer object from your C++ array.  The 
>buffer object can be built such that it refers to the 
>original copy of the data.
>
>2.  Call  back into Python (numarray.NumArray) with your 
>buffer object as the buffer parameter.
>
>You can scavenge the code in NA_newAll (Src/newarray.ch) 
>for most of the callback.
>
>>I tried NA_New - but that seems to make a copy.
>>I would need it to use the original memory space
>>so that I can "observe" the array from Python WHILE
>>the underlying C array changes (it's actually a camera 
>>image)
>>
>That sounds cool!
>
>>
>>Thanks,
>>Sebastian Haase

RE: [Numpy-discussion] BLAS

[Paul F D. <pa...@pf...>]

I had forgotten about this case. I think when these were done it was thought
that it would be better if the Numeric core did not require use of
LAPACK/BLAS. We were thinking back then of a core with other packages, and
the blas we use by default is probably the same speed so it didn't seem
important. I would have no problem with a patch to change this.

> -----Original Message-----
> From: Jens Jorgen Mortensen [mailto:je...@fy...] 
> Sent: Thursday, February 20, 2003 9:00 AM
> To: Paul F Dubois
> Subject: Re: [Numpy-discussion] BLAS
> 
> 
> On Torsdag den 20. februar 2003 17:49, Paul F Dubois wrote:
> 
> > > When doing matrix-matrix multiplications with large 
> matrices, using 
> > > the BLAS library (Basic Linear Algebra Subprograms) can speed up
> > > things a lot. I don't
> > > think Numeric takes advantage of this (is this correct?).
> >
> > No. You can configure it at installation to use the BLAS of choice.
> >
> 
> I know that the stuff in LinearAlgebra can be configured to 
> use a BLAS of 
> choice, but what about the Numeric.dot function?
> 
> Can I configure Numeric so that this:
> 
> >>> a = Numeric.dot(b, c)
> 
> will use BLAS?
> 
> Jens
> 
> 

Re: [Numpy-discussion] Non-regular lists in numarray

[Todd M. <jm...@st...>]

I logged this as a bug and I'll get to it as soon as I'm out of 
"numarray overhead reduction mode."

Thanks!

Todd

Francesc Alted wrote:

>Hi,
>
>I've found that numarray.array doesn't check enough the input for
>non-regular objects. For example:
>
>In [95]: numarray.array([3., [4, 5.2]])
>Out[95]: array([ 3.       ,  5.7096262])
>
>but,
>
>In [96]: Numeric.array([3., [4, 5.2]])
>---------------------------------------------------------------------------
>TypeError                                 Traceback (most recent call last)
>
>?
>
>TypeError: bad argument type for built-in operation
>
>
>I find Numeric behaviour more appropriate.
>
>Regards,
>
>  
>

[Numpy-discussion] Non-regular lists in numarray

[Francesc A. <fa...@op...>]

Hi,

I've found that numarray.array doesn't check enough the input for
non-regular objects. For example:

In [95]: numarray.array([3., [4, 5.2]])
Out[95]: array([ 3.       ,  5.7096262])

but,

In [96]: Numeric.array([3., [4, 5.2]])
-------------------------------------------------------------------------=
--
TypeError                                 Traceback (most recent call las=
t)

?

TypeError: bad argument type for built-in operation


I find Numeric behaviour more appropriate.

Regards,

--=20
Francesc Alted

Re: [Numpy-discussion] BLAS

[Alexander S. <a.s...@gm...>]

R.M...@ex... (R.M.Everson) writes:

> Hi,
> 
> As Paul Dubois says, some Numeric functions can be configured to use the
> BLAS library.  However, the BLAS is not used for, perhaps the most common
> and important operation: matrix/vector multiplication.
> 
> We have written a small patch to interface to replace the
> matrixproduct/dot/innerproduct functions in multiarraymodule.c with the
> appropriate BLAS calls.
> 
> The patch (against Numeric 21.1b) can be found at
> http://www.dcs.ex.ac.uk/~aschmolc/Numeric and can give a speed up of a
> factor of 40 on 1000 by 1000 matrices using the Atlas BLAS.  More details
> of the (naive!) timings can be found there too.
> 

An addendum: the new version is no longer a patch against Numeric, but a
separate module, currently called 'dotblas', which is a cleaner approach as it
doesn't require using a modified version of Numeric. To use this fast dot
instaed of Numeric's dot, you can e.g do:

import Numeric
# no errors if dotblas isn't installed
try:
   import dotblas
   Numeric.dot = dotblas.dot
except ImportError: pass

I just put a prerelease (which still handles complex arrays DIFFERENTLY from
Numeric!!!) online at:

 http://www.dcs.ex.ac.uk/~aschmolc/Numeric/dotblas.html

enjoy,

alex

Re: [Numpy-discussion] BLAS

[<R.M...@ex...>]

Hi,

As Paul Dubois says, some Numeric functions can be configured to use the
BLAS library.  However, the BLAS is not used for, perhaps the most common
and important operation: matrix/vector multiplication.

We have written a small patch to interface to replace the
matrixproduct/dot/innerproduct functions in multiarraymodule.c with the
appropriate BLAS calls.

The patch (against Numeric 21.1b) can be found at
http://www.dcs.ex.ac.uk/~aschmolc/Numeric and can give a speed up of a
factor of 40 on 1000 by 1000 matrices using the Atlas BLAS.  More details
of the (naive!) timings can be found there too.

We had planned on making a general announcement of this patch (updated to
suit Numeric 22) in a week or so.  However, we have just noticed that
Numeric.dot (=Numeric.innerproduct = Numeric.matrixmultiply) does not take
the complex conjugate of its first argument.  Taking the complex conjugate
seems to me to be the right thing for a routine named dot or innerproduct.
Indeed, until we were bitten by it not taking the conjugate, I thought it
did.

Can someone here explain the rational behind having dot, innerproduct and
matrixmultiply all do the same thing and none of them taking the conjugate?
(Matlab dot() takes the conjugate, although Matlab mtimes() (called for A*B)
does not).

I would propose that innerproduct and dot be changed to take the conjugate
and a new function that doesn't (say, mtimes) be introduced.  I suspect,
however, that this would break too much existing code.  It would be nice to
get it right in Numarray.   Alternatively, can someone suggest how both
functions can be conveniently and non-confusingly exposed?

Richard.



Paul F Dubois <Paul> writes:
  >> -----Original Message----- From:
  >> num...@li...
  >> [mailto:num...@li...] On Behalf Of
  >> Jens Jorgen Mortensen Sent: Thursday, February 20, 2003 3:59 AM To:
  >> num...@li... Subject: [Numpy-discussion]
  >> BLAS
  >>
  >>
  >> Hi,
  >>
  >> When doing matrix-matrix multiplications with large matrices, using
  >> the BLAS library (Basic Linear Algebra Subprograms) can speed up
  >> things a lot. I don't think Numeric takes advantage of this (is this
  >> correct?).

   > No. You can configure it at installation to use the BLAS of choice.

  >> Will numarray be able to do that?
  >>
  >> Jens
  >>

Re: [Numpy-discussion] byteswap() not doing anything !?

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>A Dijous 20 Febrer 2003 15:36, Todd Miller va escriure:
>  
>
>>The current "plan" is that byteswap() and _byteswap() will both behave
>>as _byteswap() does now;  i.e., they will be Numeric compatible synonyms.
>>
>>An explict (extra) call to togglebyteorder() will then produce the
>>current behavior.  The meaning of _byteorder will be unchanged.
>>
>>Please let me know if you see any snags in the plan.
>>    
>>
>
>Well, I've been doing some tests, and I think I'll be able to produce a
>version of my code that will be compatible with numarray 0.4 and future
>versions (I'm just no using byteswap() at all).
>
>However, I've detected a side effect on this change: copy() method is
>broken now in CVS:
>
>In [131]: a=numarray.array([1,2])
>
>In [132]: a.togglebyteorder()
>
>In [133]: b=a.copy()
>
>In [134]: a
>Out[134]: array([16777216, 33554432])
>
>In [135]: b
>Out[135]: array([1, 2])
>
>In [136]: a._byteorder
>Out[136]: 'big'
>
>In [137]: b._byteorder
>Out[137]: 'big'
>
>so, you don't get a well-behaved copy of original array a in b
>  
>
Doh!

>I think the next patch should cure it:
>
>--- numarray.py Tue Feb 18 16:35:16 2003
>+++ /usr/local/lib/python2.2/site-packages/numarray/numarray.py Thu Feb 20 
>19:36:07 2003
>@@ -609,6 +609,7 @@
>         c._type = self._type
>         if self.isbyteswapped():
>             c.byteswap()
>+           c.togglebyteorder()
>         return c
>  
>
Thanks!

Todd

Re: [Numpy-discussion] byteswap() not doing anything !?

[Francesc A. <fa...@op...>]

A Dijous 20 Febrer 2003 15:36, Todd Miller va escriure:
>
> The current "plan" is that byteswap() and _byteswap() will both behave
> as _byteswap() does now;  i.e., they will be Numeric compatible synonym=
s.
>
> An explict (extra) call to togglebyteorder() will then produce the
> current behavior.  The meaning of _byteorder will be unchanged.
>
> Please let me know if you see any snags in the plan.

Well, I've been doing some tests, and I think I'll be able to produce a
version of my code that will be compatible with numarray 0.4 and future
versions (I'm just no using byteswap() at all).

However, I've detected a side effect on this change: copy() method is
broken now in CVS:

In [131]: a=3Dnumarray.array([1,2])

In [132]: a.togglebyteorder()

In [133]: b=3Da.copy()

In [134]: a
Out[134]: array([16777216, 33554432])

In [135]: b
Out[135]: array([1, 2])

In [136]: a._byteorder
Out[136]: 'big'

In [137]: b._byteorder
Out[137]: 'big'

so, you don't get a well-behaved copy of original array a in b

I think the next patch should cure it:

--- numarray.py Tue Feb 18 16:35:16 2003
+++ /usr/local/lib/python2.2/site-packages/numarray/numarray.py Thu Feb 2=
0=20
19:36:07 2003
@@ -609,6 +609,7 @@
         c._type =3D self._type
         if self.isbyteswapped():
             c.byteswap()
+           c.togglebyteorder()
         return c


--=20
Francesc Alted

RE: [Numpy-discussion] BLAS

[Paul F D. <pa...@pf...>]

> -----Original Message-----
> From: num...@li... 
> [mailto:num...@li...] On 
> Behalf Of Jens Jorgen Mortensen
> Sent: Thursday, February 20, 2003 3:59 AM
> To: num...@li...
> Subject: [Numpy-discussion] BLAS
> 
> 
> Hi,
> 
> When doing matrix-matrix multiplications with large matrices, 
> using the BLAS 
> library (Basic Linear Algebra Subprograms) can speed up 
> things a lot. I don't 
> think Numeric takes advantage of this (is this correct?). 

No. You can configure it at installation to use the BLAS of choice.

> Will numarray be 
> able to do that?
> 
> Jens
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SlickEdit Inc. Develop an 
> edge. The most comprehensive and flexible code editor you can 
> use. Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 
> 30-Day Trial. www.slickedit.com/sourceforge 
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

Re: [Numpy-discussion] byteswap() not doing anything !?

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>A Dimecres 19 Febrer 2003 21:42, Todd Miller va escriure:
>  
>
>>>  >>> print a
>>>
>>>  [1 2 3]
>>>
>>>  >>> a.byteswap()
>>>  >>> print a
>>>
>>>  [16777216 33554432 50331648]
>>>      
>>>
>>This is a known bug/incompatability.  The behavior will be changed for
>>the next release of numarray.  Right now,  _byteswap() does what you want.
>>    
>>
>
>This is already decided?. Because I like the present behaviour.
>
It's already in CVS.  Let me know what you think about the stuff below.

>
>At first, I've found this behaviour a bit strange, but after get used to it,
>I admit that it is elegant because you can always see a sane representation
>of the data in array independently of which architecture you have written
>the array.
>  
>
I think byteswap() came to be the way it is in numarray-0.4 as a result 
of my experiences with cross-platform pickling.  It made sense to me at 
the time.  However,  it is definitely a new point of confusion, and not 
backwards compatible with Numeric,  so I think the numarray-0.4 
byteswap() behavior was a mistake.

>If you byteswap() an array, the _byteorder property is also changed, so you
>can check if your array is bytswapped or not just by writing:
>
>if a._byteorder <> sys.byteorder:
>    print "a is byteswapped"
>else:
>    print "a is not byteswapped"
>
>And, as you said before, you can always call _byteswap() if you *really*
>want to *only* byteswap the array.
>
>PyTables already makes use of byteswap() as it is now, and that's nice
>because an array can be restored from disk safely by just looking at the
>byte order on disk and then setting properly the ._byteorder attribute.
>That's all! This allows also to work seamlessly with objects coming from a
>mixture of big-endian and low-endian machines.
>
>But, anyway, if you plan to do the change, may you please tell us what would
>be the expected behaviour of the future .byteswap(), ._byteswap() and
>._byteorder?
>  
>
The current "plan" is that byteswap() and _byteswap() will both behave 
as _byteswap() does now;  i.e., they will be Numeric compatible synonyms.  

An explict (extra) call to togglebyteorder() will then produce the 
current behavior.  The meaning of _byteorder will be unchanged.

Please let me know if you see any snags in the plan.

>Thanks,
>
>  
>

Re: [Numpy-discussion] byteswap() not doing anything !?

[Francesc A. <fa...@op...>]

A Dimecres 19 Febrer 2003 21:42, Todd Miller va escriure:
> >   >>> print a
> >
> >   [1 2 3]
> >
> >   >>> a.byteswap()
> >   >>> print a
> >
> >   [16777216 33554432 50331648]
>
> This is a known bug/incompatability.  The behavior will be changed for
> the next release of numarray.  Right now,  _byteswap() does what you wa=
nt.

This is already decided?. Because I like the present behaviour.

At first, I've found this behaviour a bit strange, but after get used to =
it,
I admit that it is elegant because you can always see a sane representati=
on
of the data in array independently of which architecture you have written
the array.

If you byteswap() an array, the _byteorder property is also changed, so y=
ou
can check if your array is bytswapped or not just by writing:

if a._byteorder <> sys.byteorder:
    print "a is byteswapped"
else:
    print "a is not byteswapped"

And, as you said before, you can always call _byteswap() if you *really*
want to *only* byteswap the array.

PyTables already makes use of byteswap() as it is now, and that's nice
because an array can be restored from disk safely by just looking at the
byte order on disk and then setting properly the ._byteorder attribute.
That's all! This allows also to work seamlessly with objects coming from =
a
mixture of big-endian and low-endian machines.

But, anyway, if you plan to do the change, may you please tell us what wo=
uld
be the expected behaviour of the future .byteswap(), ._byteswap() and
=2E_byteorder?

Thanks,

--=20
Francesc Alted

[Numpy-discussion] BLAS

[Jens J. M. <je...@fy...>]

Hi,

When doing matrix-matrix multiplications with large matrices, using the BLAS 
library (Basic Linear Algebra Subprograms) can speed up things a lot. I don't 
think Numeric takes advantage of this (is this correct?). Will numarray be 
able to do that?

Jens

Re: [Numpy-discussion] range check: feature request for numarray

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>Hi,
>
>  
>
Hi Francesc,

I'm sorry about the slow response on this.  I looked into what it would 
take to do this,  and while I agree with you in principle,  right now my 
hands are full trying to beat down numarray overhead.

>I think it would be useful to provide some range checking in numarray. For
>example, right now, you can do:
>
>In [24]: a=numarray.array([1,2],numarray.Int8)
>
>In [25]: a[1] = 256
>
>In [26]: a
>Out[26]: array([1, 0], type=Int8)
>
>and nothing happens. But I'm proposing to raise an OverflowWarning so that
>people can be aware of such range overflows.
>
That sounds reasonable.  If you'd care to do a patch,  I think we would 
want it.  If you don't have time,  it may be a little while before we do.

>Maybe it is desirable that the default would be to not issue the warning,
>except when the user wanted to know about that.
>  
>
I think I'd rather see the warning on by default, even though it might 
"break" some existing code.

>So, my proposal is that the actual behaviour should be mantained, but when
>you want to be aware of all the warnings something like this could happen:
>
>In [28]: warnings.resetwarnings()
>
>In [29]: a=numarray.array([1,2],numarray.Int8)
>
>In [30]: a[1] = 256
>OverflowWarning: value assignment not in the type range
>
>In [31]: a
>Out[31]: array([1, 0], type=Int8)
>
>But perhaps this feature might slow a bit the performance of assignments.
>
Yes, but probably not too much.

>Regards,
>
>  
>
Todd

Re: [Numpy-discussion] byteswap() not doing anything !?

[Todd M. <jm...@st...>]

Sebastian Haase wrote:

>>>>xx = na.array((1,2,3))
>>>>xx
>>>>        
>>>>
>array([1, 2, 3])
>  
>
>>>>xx.byteswap()
>>>>xx
>>>>        
>>>>
>array([1, 2, 3])
>  
>
>>>>xx.type()
>>>>        
>>>>
>Int32
>
>Hi all,
>I was reading the documentation for array 0.4
>but I get the about results.
>How do I get the bytes swaped like it says in the manual:
>
>numbyteswap()
>   The byteswap method performs a byte swapping 
>   operation on all the elements in the array, 
>   working inplace (i.e. it returns None). 
>   >>> print a
>   [1 2 3]
>   >>> a.byteswap()
>   >>> print a
>   [16777216 33554432 50331648]
>
This is a known bug/incompatability.  The behavior will be changed for 
the next release of numarray.  Right now,  _byteswap() does what you want.

>
>
>Thanks,
>Sebastian
>
>  
>
Todd

[Numpy-discussion] byteswap() not doing anything !?

[Sebastian H. <ha...@ms...>]

>>> xx = na.array((1,2,3))
>>> xx
array([1, 2, 3])
>>> xx.byteswap()
>>> xx
array([1, 2, 3])
>>> xx.type()
Int32

Hi all,
I was reading the documentation for array 0.4
but I get the about results.
How do I get the bytes swaped like it says in the manual:

numbyteswap()
   The byteswap method performs a byte swapping 
   operation on all the elements in the array, 
   working inplace (i.e. it returns None). 
   >>> print a
   [1 2 3]
   >>> a.byteswap()
   >>> print a
   [16777216 33554432 50331648]


Thanks,
Sebastian

[Numpy-discussion] range check: feature request for numarray

[Francesc A. <fa...@op...>]

Hi,

I think it would be useful to provide some range checking in numarray. Fo=
r
example, right now, you can do:

In [24]: a=3Dnumarray.array([1,2],numarray.Int8)

In [25]: a[1] =3D 256

In [26]: a
Out[26]: array([1, 0], type=3DInt8)

and nothing happens. But I'm proposing to raise an OverflowWarning so tha=
t
people can be aware of such range overflows.

Maybe it is desirable that the default would be to not issue the warning,
except when the user wanted to know about that.

So, my proposal is that the actual behaviour should be mantained, but whe=
n
you want to be aware of all the warnings something like this could happen=
:

In [28]: warnings.resetwarnings()

In [29]: a=3Dnumarray.array([1,2],numarray.Int8)

In [30]: a[1] =3D 256
OverflowWarning: value assignment not in the type range

In [31]: a
Out[31]: array([1, 0], type=3DInt8)

But perhaps this feature might slow a bit the performance of assignments.

Regards,

--=20
Francesc Alted

`` We are shaped by our thoughts, we become what we think.
 When the mind is pure, joy follows like a shadow that never
 leaves. '' -- Buddha, The Dhammapada

Re: [Numpy-discussion] rank-0 chararrays?

[Francesc A. <fa...@op...>]

A Dimarts 18 Febrer 2003 13:44, Todd Miller va escriure:
>  You are giving me interface angst...  :)

Well, I don't know exactly what do you mean with that, but I hope it woul=
d
be something not too bad ;)

>
> This was not always so,  be we made it work when we thought rank-0 had
> something to offer.  After some discussion on numpy-discussion-list,
>  rank-0 went out of vogue.

Mmmm, do you mean that rank-0 is being deprecated in numarray?

> Why exactly do you need rank-0?

Appart from supporting chararrays in PyTables, I'm using them as a buffer=
 to
save homogeneous character standard lists and tuples, because it is very
easy to obtain a contiguous C buffer from it. However, if I have no
possibility to distinguish between "qqq" and ["qqq"] objects directly fro=
m
chararray instances obtained from them, I can't materialize them properly
when reading the objects from the persitent storage.

Perhaps using more metadata could solve the situation (for example, savin=
g
the original shape of the object), but I wouldn't like to clutter
unnecessarily PyTables metadata space.

>
> >What can be done to achieve this?
>
> 1. Add a little special casing to chararray._charArrayToStringList() to
> handle rank-0.  I did this already in CVS.

Ok. For the moment I'll be using numarray CVS, although I don't know if n=
ext
version of numarray will be out before next PyTables release (planned in =
a
couple of weeks).

> 2. Debate whether or not to change chararray.array() to work as you've
> shown above.  Proceed from there.

Well, the fact is that I needed rank-0 only for the reason stated before.
But I'm not sure if this is reason enough to open such a debate.

Thanks!,

--=20
Francesc Alted