numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Experience with Visit?

[Robert C. <cim...@nt...>]

Travis Oliphant wrote:
> Has anybody had any experience with the 3-D visualization software 
> VISIT?   It has Python bindings and seems to be pretty sophisticated.  
> I'm wondering why I haven't heard more about it.
> 
> http://www.llnl.gov/visit/

No reaction up to now, so...

I have just tried the 'getting started' part and was quite impressed, 
thanks for posting the link! Up to now I have used ParaView and was very 
satisfied, but the Python bindings of VisIt are a great lure.

r.

Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailing unused bytes

[Martin W. <mar...@gm...>]

On Friday 15 September 2006 16:13, Martin Wiechert wrote:
> Hi list,
>
> I'm using PyArray_DescrConverter with a dict object to create a
> "struct-like" dtype from C.
> As the struct contains different data types I run into "unaligned access"
> problems.

on IA64 I should have mentioned

> Is there a way to force alignment or to get trailing unused bytes in the
> dtpye?
>
> Thanks, Martin
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] PyArray_DescrConverter - alignment / trailing unused bytes

[Martin W. <mar...@gm...>]

Hi list,

I'm using PyArray_DescrConverter with a dict object to create a "struct-like" 
dtype from C.
As the struct contains different data types I run into "unaligned access" 
problems.
Is there a way to force alignment or to get trailing unused bytes in the 
dtpye?

Thanks, Martin

Re: [Numpy-discussion] recarray

[Francesc A. <fa...@ca...>]

A Divendres 15 Setembre 2006 15:57, Robert Kern va escriure:
> Lionel Roubeyrie wrote:
> > Hi all,
> > I try to use recarray with rec.fromrecords on time-series, datas come
> > from a file where they are stored in csv format, with after each data
> > colum there is one column meanning the state of the data, and the first
> > column is for dates. Then, is it possible to directly transform column =
of
> > strings to a integer one (or datetime one), and to remove a not used
> > column?
>
> When I import CSV files into record arrays, I usually read in all of the
> data and transpose the list of rows to get a list of columns. Then I can
> remove columns and transform them _en masse_, usually with map().

Another possibility is to play with columns directly from the initial=20
recarray. The next is an example:

In [101]: ra=3Dnumpy.rec.array("1"*36, dtype=3D"a4,i4,f4", shape=3D3)
In [102]: ra
Out[102]:
recarray([('1111', 825307441, 2.5784852031307537e-09),
       ('1111', 825307441, 2.5784852031307537e-09),
       ('1111', 825307441, 2.5784852031307537e-09)],
      dtype=3D[('f0', '|S4'), ('f1', '<i4'), ('f2', '<f4')])
In [103]: rb=3Dnumpy.rec.fromarrays([numpy.array(ra['f0'], 'i4'),ra['f2']],=
=20
names=3D'f0,f1')
In [104]: rb
Out[104]:
recarray([(1111, 2.5784852031307537e-09), (1111, 2.5784852031307537e-09),
       (1111, 2.5784852031307537e-09)],
      dtype=3D[('f0', '<i4'), ('f1', '<f4')])

where ra is the original recarray and rb is a derived one where its first=20
column is the original from ra, but converted to integers ('i4'), and the=20
second it's the third column from ra (so the second column from ra has been=
=20
stripped out from rb).

HTH,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] recarray

[Robert K. <rob...@gm...>]

Lionel Roubeyrie wrote:
> Hi all,
> I try to use recarray with rec.fromrecords on time-series, datas come from a 
> file where they are stored in csv format, with after each data colum there is 
> one column meanning the state of the data, and the first column is for dates.
> Then, is it possible to directly transform column of strings to a integer one 
> (or datetime one), and to remove a not used column?

When I import CSV files into record arrays, I usually read in all of the data 
and transpose the list of rows to get a list of columns. Then I can remove 
columns and transform them _en masse_, usually with map().

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] degree matrix construction

[Charles R H. <cha...@gm...>]

On 9/15/06, Satya Upadhya <sat...@ya...> wrote:
>
> Dear Friends,
> my question is the following:
>
> Suppose i have the following code:
>
> >>> from LinearAlgebra import *
>
> >>> from Numeric import *
> >>> A = [1,2,1,3,1,3,4,1,2]
> >>> B = reshape(A,(3,3))
> >>> C = sum(B,1)
> >>> C
> array([4, 7, 7])
> >>>
>
> Now, my problem is to construct a degree matrix D which is a 3 * 3 matrix
> with diagonal elements 4,7,7 (obtained from the elements of C) and all
> off-diagonal elements equal to 0.
>

Is this what you want to do?

In [2]: a = array([4, 7, 7])

In [3]: diagflat(a)
Out[3]:
array([[4, 0, 0],
       [0, 7, 0],
       [0, 0, 7]])

Chuck

Re: [Numpy-discussion] degree matrix construction

[Joris De R. <jo...@st...>]

Forgot the link to the NEL:  http://www.scipy.org/Numpy_Example_List

J.

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

Re: [Numpy-discussion] degree matrix construction

[Joris De R. <jo...@st...>]

Hi,

   [SU]: Now, my problem is to construct a degree matrix D which is a 3 * 3 matrix with diagonal elements 4,7,7 

You might have a look at the Numpy Example List, at the function diag().

Ciao,
Joris


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

[Numpy-discussion] degree matrix construction

[Satya U. <sat...@ya...>]

Dear Friends,
my question is the following:

Suppose i have the following code:

>>> from LinearAlgebra import *

>>> from Numeric import *
>>> A = [1,2,1,3,1,3,4,1,2]
>>> B = reshape(A,(3,3))
>>> C = sum(B,1)
>>> C
array([4, 7, 7])
>>> 

Now, my problem is to construct a degree matrix D which is a 3 * 3 matrix with diagonal elements 4,7,7 (obtained from the elements of C) and all off-diagonal elements equal to 0.
Could some kind soul kindly tell me how to do this.
I've looked at the help for the diagonal function and i am unable to do what i wish to. Furthermore i dont understand the meaning of axis1 and axis2:
>>> help (diagonal)
Help on function diagonal in module Numeric:

diagonal(a, offset=0, axis1=0, axis2=1)
    diagonal(a, offset=0, axis1=0, axis2=1) returns all offset diagonals
    defined by the given dimensions of the array.

>>>

Thanking you,
Satya




 				
---------------------------------
 Find out what India is talking about on  - Yahoo! Answers India 
 Send FREE SMS to your friend's mobile from Yahoo! Messenger Version 8. Get it NOW

[Numpy-discussion] recarray

[Lionel R. <lro...@li...>]

Hi all,
I try to use recarray with rec.fromrecords on time-series, datas come from a 
file where they are stored in csv format, with after each data colum there is 
one column meanning the state of the data, and the first column is for dates.
Then, is it possible to directly transform column of strings to a integer one 
(or datetime one), and to remove a not used column?
thanks

--
Lionel

Re: [Numpy-discussion] how to get info about internals of an arrayobject ?

[Albert S. <fu...@gm...>]

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Sebastian Haase
> Sent: 15 September 2006 03:21
> To: numpy-discussion
> Subject: [Numpy-discussion] how to get info about internals of an
> arrayobject ?
> 
> Hi,
> what I'm asking is if numpy has an equivalent to numarray's info()
> function:

> <snip>

> This was always helpful to me when debugging C binding code.
> 
> Especially I'm asking if there is any way to get the memory address of an
> array - for debugging purposes only - of course ;-)

numpy.array([]).__array_interface__['data'][0]

Cheers,

Albert

Re: [Numpy-discussion] how to get info about internals of an array object ?

[Francesc A. <fa...@ca...>]

El dj 14 de 09 del 2006 a les 18:20 -0700, en/na Sebastian Haase va
escriure:

> Especially I'm asking if there is any way to get the memory address of an=
=20
> array - for debugging purposes only - of course ;-)

For this, you can print the data buffer:

In [1]:import numpy
In [2]:a=3Dnumpy.array([1])
In [3]:a.data
Out[3]:<read-write buffer for 0x82a9970, ptr 0x821f630, size 4 at
0xb6dd3300>

although I'm not sure which number is the memory address I'd say it's
the last one.

Cheers,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=C3=A1rabos Coop. V.   Enjoy Data
 "-"

Re: [Numpy-discussion] Axis Iterator?

[Brendan S. <bre...@ya...>]

Oh that's cool.  For some reason I thought that the built in iterator  
(for i in array) iterated over cells, not the first axis.  I also  
didn't think about swapaxes.    Is there any desire to add a  
convenience function or method as follows?

def axisIter(selfOrArr, i):
	return iter(selfOrArr.swapAxes(0,i))

Thanks for everyone who helped out.   I've got something that works  
now.  Cheers.
    Brendan

On 14-Sep-06, at 10:05 PM, numpy-discussion- 
re...@li... wrote:

> Date: Fri, 15 Sep 2006 11:05:13 +0900
> From: "Bill Baxter" <wb...@gm...>
> Subject: Re: [Numpy-discussion] Axis Iterator?
> To: "Discussion of Numerical Python"
> 	<num...@li...>
> Message-ID:
> 	<e86...@ma...>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 9/15/06, Tim Hochberg <tim...@ie...> wrote:
>> Isn't swapaxis appropriate for this? In other words:
>>
>
> You're right.  Just didn't think of that.  Never used swapaxes before.
>
> def axisiter(arr, i):
>     return arr.swapaxes(0,i)
>
> --bb

Re: [Numpy-discussion] Axis Iterator?

[Bill B. <wb...@gm...>]

On 9/15/06, Tim Hochberg <tim...@ie...> wrote:
> Isn't swapaxis appropriate for this? In other words:
>

You're right.  Just didn't think of that.  Never used swapaxes before.

def axisiter(arr, i):
    return arr.swapaxes(0,i)

--bb

[Numpy-discussion] how to get info about internals of an array object ?

[Sebastian H. <ha...@ms...>]

Hi,
what I'm asking is if numpy has an equivalent to numarray's info() function:
>>> na.arange(10).info()
class: <class 'numarray.numarraycore.NumArray'>
shape: (10,)
strides: (4,)
byteoffset: 0
bytestride: 4
itemsize: 4
aligned: 1
contiguous: 1
buffer: <memory at 0x085b7ec8 with size:0x00000028 held by object 0x4306e5e0 
aliasing object 0x00000000>
data pointer: 0x085b7ec8 (DEBUG ONLY)
byteorder: 'little'
byteswap: 0
type: Int32

This was always helpful to me when debugging C binding code.

Especially I'm asking if there is any way to get the memory address of an 
array - for debugging purposes only - of course ;-)

Thanks,
Sebastian Haase

Re: [Numpy-discussion] Axis Iterator?

[Tim H. <tim...@ie...>]

Bill Baxter wrote:
> Iteration over axis 0 is built-in, so you can already do
>     (vectorFunc(row) for row in array)
> And you can use transpose() to make it so the axis you want to iterate
> over is axis 0.
>     (vectorFunc(col) for col in array.transpose(1,0))
> Or just use the .T attribute
>     (vectorFunc(col) for col in array.T)
>
> So it seems kind of a toss-up whether it's worth adding a specific API
> to do that.  The implementation would probably just return the
> transpose with the given axis in the zero slot.   Something like:
>
> def axisiter(arr, i):
>     ax = [i] + range(arr.ndim)
>     del ax[i+1]
>     return arr.transpose(ax)
>
> --bb
>   
Isn't swapaxis appropriate for this? In other words:

for x in arr.swapaxis(0, i):
    #...

should be more or less the same as:

for x in axister(arr, i):
    #....

and it already exists. I'm in a hurry, so I haven't checked the details 
here, but the basic idea seems sound.

-tim

> On 9/15/06, Brendan Simons <bre...@ya...> wrote:
>   
>> Hi all,
>>
>> Just wondering if there was an arbitrary axis iterator in numpy, or
>> if not, if there's demand for one.  What I'm looking for is something
>> which would allow me to do  something like (vectorFunc(column) for
>> column in array.axisIter(1) )  without a bunch of for loops and slicing.
>>
>> Thoughts?
>>     Brendan
>>     
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   

Re: [Numpy-discussion] Avoiding array-scalar arithmetic?

[Ryan G. <rn...@co...>]

Travis,

Thanks for the quick response. My application is back up to its old 
speed.

Thanks also for spearheading the numpy/scipy projects. It's certainly 
made my work much, much more productive.

Cheers,
Ryan

On Sep 14, 2006, at 7:40 PM, Travis Oliphant wrote:
> Ryan Gutenkunst wrote:
>> Thanks for the quick response, but either there's a bug, or I'm using
>> things wrong. It appears to work to work in the N-D case, but not 
>> 1-D. I
>> looked at the change you made, but my grasp of the C-API is too weak 
>> to
>> isolate the problem.
>>
>>
> It's a silly bug.  Please check out the latest SVN.
>
> -Travis

-- 
Ryan Gutenkunst               |
Cornell Dept. of Physics      |       "It is not the mountain
                               |        we conquer but ourselves."
Clark 535 / (607)255-6068     |        -- Sir Edmund Hillary
AIM: JepettoRNG               |
          http://www.physics.cornell.edu/~rgutenkunst/

Re: [Numpy-discussion] Axis Iterator?

[Bill B. <wb...@gm...>]

Iteration over axis 0 is built-in, so you can already do
    (vectorFunc(row) for row in array)
And you can use transpose() to make it so the axis you want to iterate
over is axis 0.
    (vectorFunc(col) for col in array.transpose(1,0))
Or just use the .T attribute
    (vectorFunc(col) for col in array.T)

So it seems kind of a toss-up whether it's worth adding a specific API
to do that.  The implementation would probably just return the
transpose with the given axis in the zero slot.   Something like:

def axisiter(arr, i):
    ax = [i] + range(arr.ndim)
    del ax[i+1]
    return arr.transpose(ax)

--bb

On 9/15/06, Brendan Simons <bre...@ya...> wrote:
> Hi all,
>
> Just wondering if there was an arbitrary axis iterator in numpy, or
> if not, if there's demand for one.  What I'm looking for is something
> which would allow me to do  something like (vectorFunc(column) for
> column in array.axisIter(1) )  without a bunch of for loops and slicing.
>
> Thoughts?
>     Brendan

Re: [Numpy-discussion] Axis Iterator?

[Travis O. <oli...@ee...>]

Brendan Simons wrote:

>Hi all,
>
>Just wondering if there was an arbitrary axis iterator in numpy, or  
>if not, if there's demand for one.  What I'm looking for is something  
>which would allow me to do  something like (vectorFunc(column) for  
>column in array.axisIter(1) )  without a bunch of for loops and slicing.
>  
>

Hmm... I can't think of something directly in Python, but it would be 
pretty easy to add.  You could probably also do something clever by 
creating your own ufunc with frompyfunc and object arrays. 

In C, this would be easy.  In the C-API there is a function 
PyArray_IterAllButAxis which provides an iterator that iterates over all 
axes but one.   Then, you would call the vectorFunc for each element in 
the loop.

The ufuncs use this functionality to call the underlying 1-d loops. 

-Travis

[Numpy-discussion] Axis Iterator?

[Brendan S. <bre...@ya...>]

Hi all,

Just wondering if there was an arbitrary axis iterator in numpy, or  
if not, if there's demand for one.  What I'm looking for is something  
which would allow me to do  something like (vectorFunc(column) for  
column in array.axisIter(1) )  without a bunch of for loops and slicing.

Thoughts?
    Brendan
__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[Numpy-discussion] Experience with Visit?

[Travis O. <oli...@ee...>]

Has anybody had any experience with the 3-D visualization software 
VISIT?   It has Python bindings and seems to be pretty sophisticated.  
I'm wondering why I haven't heard more about it.

http://www.llnl.gov/visit/

-Travis

Re: [Numpy-discussion] Avoiding array-scalar arithmetic?

[Travis O. <oli...@ee...>]

Ryan Gutenkunst wrote:

>Hi Travis,
>
>Travis Oliphant wrote:
>  
>
>>Ryan Gutenkunst wrote:
>>    
>>
>>>I notice that numpy_array.item() will give me the first element as a 
>>>normal scalar. Would it be possible for numpy_array.item(N) to return 
>>>the Nth element of the array as a normal scalar?
>>>  
>>>      
>>>
>>Now this is an interesting idea.  It would allow you to by-pass the 
>>slow-indexing as well as the array scalar computation should you desire 
>>it.  I like it and am going to add it unless there are convincing 
>>objections.
>>
>>-Travis
>>    
>>
>
>Thanks for the quick response, but either there's a bug, or I'm using 
>things wrong. It appears to work to work in the N-D case, but not 1-D. I 
>looked at the change you made, but my grasp of the C-API is too weak to 
>isolate the problem.
>  
>
It's a silly bug.  Please check out the latest SVN.

-Travis

Re: [Numpy-discussion] Avoiding array-scalar arithmetic?

[Ryan G. <rn...@co...>]

Hi Travis,

Travis Oliphant wrote:
> Ryan Gutenkunst wrote:
>> I notice that numpy_array.item() will give me the first element as a 
>> normal scalar. Would it be possible for numpy_array.item(N) to return 
>> the Nth element of the array as a normal scalar?
>>   
> Now this is an interesting idea.  It would allow you to by-pass the 
> slow-indexing as well as the array scalar computation should you desire 
> it.  I like it and am going to add it unless there are convincing 
> objections.
> 
> -Travis

Thanks for the quick response, but either there's a bug, or I'm using 
things wrong. It appears to work to work in the N-D case, but not 1-D. I 
looked at the change you made, but my grasp of the C-API is too weak to 
isolate the problem.

 >>> import numpy
 >>> numpy.__version__
'1.0rc1.dev3154'
 >>> a = numpy.array([[1.0, 2.0], [3.0, 4.0]])
 >>> a.item(1,1)
4.0
 >>> a = numpy.array([1.0, 2.0])
 >>> a.item(0)
1.0
 >>> a.item(1)
1.7765824089018436e-307
 >>> a.item((1,))
1.7765824089018436e-307


Thanks for your help,
Ryan

-- 
Ryan Gutenkunst               |
Cornell LASSP                 |       "It is not the mountain
                               |        we conquer but ourselves."
Clark 535 / (607)227-7914     |        -- Sir Edmund Hillary
AIM: JepettoRNG               |
          http://www.physics.cornell.edu/~rgutenkunst/

Re: [Numpy-discussion] Problem with concatenate and object arrays

[Christopher B. <Chr...@no...>]

Charles R Harris wrote:
>> > Why not simply
>> > write a wrapper function in python that does Numeric-style guesswork,
>> > and put it in the compatibility modules? 

>> Can I encourage any more comments? 

+1

> The main problem in constructing arrays
> of objects is more information needs to be supplied because the user's
> intention can't be reliably deduced from the current syntax.

I wrote about this a bit early in this conversation, and as I thought 
about it. I'm not sure it's possible _- you could specify a rank, or a 
shape, but in general, there wouldn't be a unique way to translate an 
given hierarchy of sequences into a particular shape: imagine four 
levels of nested lists, asked to turn into a rank-3 array.

This is why it may be best to simply recommend that people create an 
empty array of the shape they need, then put the objects into it - it's 
the only way to construct what you need reliably.

However, an object array constructor that take a rank as an argument 
might well work for most cases, as long as there is a clearly documented 
and consistent way to handle extra levels of sequences: perhaps specify 
that any extra levels of nesting always go to the last dimension (or the 
first). That being said, it's still dangerous -- what levels of nesting 
are allowed would depend on which sequences *happen* to be the same 
size. Also the code would be a pain to write!

I wonder how often people need to use objects arrays when they don't 
know when writing the code what shape they need?


this is making me think that maybe all we really need is a little 
syntactic sugar for creating empty object arrays:

numpy.ObjectArray(shape)

Not much different than:

numpy.empty(shape, dtype=numpy.object)

but a little cleaner an more obvious to new users that are primarily 
interested in object arrays -- analogous to ones() and zeros()

 > That said, I
> have no idea how widespread the use of object arrays is and so don't know
> how much it really matters. 

If we ever get nd-arrays into the standard lib (or want to see wider use 
of them in any case), I think that object arrays are critical. Right 
now, people think they don't have a use for numpy if they aren't doing 
serious number crunching -- it's seen mostly as a way to speed up 
computations on lots of numbers. However, I think nd-arrays have LOTS of 
  other applications, for anything where the data fits well in to a 
"rectangular" data structure. n-d slicing is a wonderful thing! As numpy 
gets wider use -- object arrays will be a very big draw.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] NumPy 1.0 release-candidate 1.0 this weekend

[Charles R H. <cha...@gm...>]

On 9/14/06, Victoria G. Laidler <la...@st...> wrote:
>
> Francesc Altet wrote:
>
> >El dj 14 de 09 del 2006 a les 02:11 -0700, en/na Andrew Straw va
> >escriure:
> >
> >
> >>>>My main focus is on the fact that you might read '<i4' as
> >>>>"less" than 4-bytes int, which is very confusing !
> >>>>
> >>>>
> >>>>
> >>>>
> >>>I can agree it's confusing at first, but it's the same syntax the
> struct
> >>>module uses which is the Python precedent for this.
> >>>
> >>>
> >>>
> >>I'm happy with seeing the repr() value since I know what it means, but I
> >>can see Sebastian's point. Perhaps there's a middle ground -- the str()
> >>representation for simple dtypes could contain both the repr() value and
> >>an English description. For example, something along the lines of
> >>"dtype('<i4') (4 byte integer, little endian)". For more complex dtypes,
> >>the repr() string could be given without any kind of English
> translation.
> >>
> >>
> >
> >+1
> >
> >I was very used (and happy) to the numarray string representation for
> >types ('Int32', 'Complex64'...) and looking at how NumPy represents it
> >now, I'd say that this is a backwards step in readability. Something
> >like '<i4' would look good for a low-level library, but not for a
> >high-level one like NumPy, IMO.
> >
> >
> I agree entirely.
> The first type I got '<i4' instead of 'Int32', my reaction was "What the
> hell is that?"
> It looked disturbingly like line-noise corrupted text to me! (Blast from
> the past...)
>
> +1 from me as well.


Just to balance the voting, I think things are fine as they are. As Travis
says, the '<' is already used in the Python structure module and the rest
doesn't take much time getting used to. However, the docstring for the dtype
class is a bit lacking. It shouldn't be too much work to fix that up.

Chuck