Thread: [Numpy-discussion] bug ? in Records arrays in numarray

A package for scientific computing with Python

Brought to you by: charris208, jarrodmillman, kern, rgommers, teoliphant

numpy-discussion

[Numpy-discussion] bug ? in Records arrays in numarray

From: Sebastian H. <ha...@ms...> - 2004-06-29 00:00:51

Hi,
I have two record arrays. I was trying to assign one item from one recarray to 
the other:
>>> omx.zext[0] = main.ring4ext[0,0]
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "X:/PrWin\numarray\records.py", line 744, in _setitem
    self.field(self._names[i])[row] = value.field(self._names[i])
  File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619, in 
__tonumtype__
ValueError: Can't use non-rank-0 array as a scalar.
>>> `omx.zext[0]`
'<numarray.records.Record instance at 0x042AFC78>'
>>> `main.ring4ext[0,0]`
'<numarray.records.Record instance at 0x042AFC78>'
>>> q = omx.zext[0]
>>> w = main.ring4ext[0,0]
>>> q
(2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
+018, 1.2771574333702815e-040)
>>> w
(array([428]), array([75]), array([124]), array([  1.08846451e+09], 
type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605], 
type=Float32))
>>> omx.zext._formats
['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>>> main.ring4ext._formats
['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']


I can't track down why one (q) contains scalars and the other (w) constains 
arrays (is array([428]) a 'rank-0 array'? ) ! 
This is how I generate them:
main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
  shape=(ts,nc), names=("num","min","max","time","mean","darkVal"), aligned=1)
omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
  names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),aligned=1 )

[[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get 
NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462   -- 
usually type(shape) == types.ListType should be OK, right ?)
]]


Any ideas?

Thanks,
Sebastian Haase

Re: [Numpy-discussion] bug ? in Records arrays in numarray

From: Sebastian H. <ha...@ms...> - 2004-06-29 21:52:26

OK,
I'm still trying to get a handle on these record arrays - because I think they 
are pretty cool, if I could get them to work...
Following the code from yesterday (see that posting below) I discovered this:
main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
is this intended ??

>>> main.ring4ext[0][0]
(2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
>>> main.ring4ext[0,0]
(array([2308, 2309]), array([76, 76]), array([272, 269]), array([  1.08848115e
+09,   1.08848115e+09], type=Float32), array([ 104.18000031,  104.45999908], 
type=Float32), array([ 1994.94995117,  1994.95996094], type=Float32))
>>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
(20, 2)

Any comments are appreciated,
Thanks
Sebastian




On Monday 28 June 2004 05:00 pm, Sebastian Haase wrote:
> Hi,
> I have two record arrays. I was trying to assign one item from one recarray
> to
>
> the other:
> >>> omx.zext[0] = main.ring4ext[0,0]
>
> Traceback (most recent call last):
>   File "<input>", line 1, in ?
>   File "X:/PrWin\numarray\records.py", line 744, in _setitem
>     self.field(self._names[i])[row] = value.field(self._names[i])
>   File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619,
> in __tonumtype__
> ValueError: Can't use non-rank-0 array as a scalar.
>
> >>> `omx.zext[0]`
>
> '<numarray.records.Record instance at 0x042AFC78>'
>
> >>> `main.ring4ext[0,0]`
>
> '<numarray.records.Record instance at 0x042AFC78>'
>
> >>> q = omx.zext[0]
> >>> w = main.ring4ext[0,0]
> >>> q
>
> (2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
> +018, 1.2771574333702815e-040)
>
> >>> w
>
> (array([428]), array([75]), array([124]), array([  1.08846451e+09],
> type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605],
> type=Float32))
>
> >>> omx.zext._formats
>
> ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>
> >>> main.ring4ext._formats
>
> ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>
>
> I can't track down why one (q) contains scalars and the other (w) constains
> arrays (is array([428]) a 'rank-0 array'? ) !
> This is how I generate them:
> main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
>   shape=(ts,nc), names=("num","min","max","time","mean","darkVal"),
> aligned=1) omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
>  
> names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),align
>ed=1 )
>
> [[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get
> NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462  
> -- usually type(shape) == types.ListType should be OK, right ?)
> ]]
>
>
> Any ideas?
>
> Thanks,
> Sebastian Haase
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> digital self defense, top technical experts, no vendor pitches,
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] bug ? in Records arrays in numarray

From: Todd M. <jm...@st...> - 2004-06-29 23:50:29

On Tue, 2004-06-29 at 17:52, Sebastian Haase wrote:
> OK,
> I'm still trying to get a handle on these record arrays - because I think they 
> are pretty cool, if I could get them to work...
> Following the code from yesterday (see that posting below) I discovered this:
> main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
> is this intended ??
> 
> >>> main.ring4ext[0][0]
> (2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
> >>> main.ring4ext[0,0]
> (array([2308, 2309]), array([76, 76]), array([272, 269]), array([  1.08848115e
> +09,   1.08848115e+09], type=Float32), array([ 104.18000031,  104.45999908], 
> type=Float32), array([ 1994.94995117,  1994.95996094], type=Float32))
> >>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
> (20, 2)
> 
> Any comments are appreciated,

I talked to JC Hsu, the numarray.records author, and he explained that
we're probably looking at a limitation of numarray.records: it doesn't
yet handle multi-dimensional arrays of records.  JC indicated he had
replied to Sebastian,  but for the benefit of everyone else, that's the
deal.

Regards,
Todd

> Thanks
> Sebastian
> 
> 
> 
> 
> On Monday 28 June 2004 05:00 pm, Sebastian Haase wrote:
> > Hi,
> > I have two record arrays. I was trying to assign one item from one recarray
> > to
> >
> > the other:
> > >>> omx.zext[0] = main.ring4ext[0,0]
> >
> > Traceback (most recent call last):
> >   File "<input>", line 1, in ?
> >   File "X:/PrWin\numarray\records.py", line 744, in _setitem
> >     self.field(self._names[i])[row] = value.field(self._names[i])
> >   File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619,
> > in __tonumtype__
> > ValueError: Can't use non-rank-0 array as a scalar.
> >
> > >>> `omx.zext[0]`
> >
> > '<numarray.records.Record instance at 0x042AFC78>'
> >
> > >>> `main.ring4ext[0,0]`
> >
> > '<numarray.records.Record instance at 0x042AFC78>'
> >
> > >>> q = omx.zext[0]
> > >>> w = main.ring4ext[0,0]
> > >>> q
> >
> > (2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
> > +018, 1.2771574333702815e-040)
> >
> > >>> w
> >
> > (array([428]), array([75]), array([124]), array([  1.08846451e+09],
> > type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605],
> > type=Float32))
> >
> > >>> omx.zext._formats
> >
> > ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
> >
> > >>> main.ring4ext._formats
> >
> > ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
> >
> >
> > I can't track down why one (q) contains scalars and the other (w) constains
> > arrays (is array([428]) a 'rank-0 array'? ) !
> > This is how I generate them:
> > main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
> >   shape=(ts,nc), names=("num","min","max","time","mean","darkVal"),
> > aligned=1) omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
> >  
> > names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),align
> >ed=1 )
> >
> > [[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get
> > NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462  
> > -- usually type(shape) == types.ListType should be OK, right ?)
> > ]]
> >
> >
> > Any ideas?
> >
> > Thanks,
> > Sebastian Haase
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email sponsored by Black Hat Briefings & Training.
> > Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> > digital self defense, top technical experts, no vendor pitches,
> > unmatched networking opportunities. Visit www.blackhat.com
> > _______________________________________________
> > Numpy-discussion mailing list
> > Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
> digital self defense, top technical experts, no vendor pitches, 
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
--

Re: [Numpy-discussion] bug ? in Records arrays in numarray

From: Sebastian H. <ha...@ms...> - 2004-06-30 00:00:22

On Tuesday 29 June 2004 04:49 pm, Todd Miller wrote:
> On Tue, 2004-06-29 at 17:52, Sebastian Haase wrote:
> > OK,
> > I'm still trying to get a handle on these record arrays - because I think
> > they are pretty cool, if I could get them to work...
> > Following the code from yesterday (see that posting below) I discovered
> > this: main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
> > is this intended ??
> >
> > >>> main.ring4ext[0][0]
> >
> > (2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
> >
> > >>> main.ring4ext[0,0]
> >
> > (array([2308, 2309]), array([76, 76]), array([272, 269]), array([ 
> > 1.08848115e +09,   1.08848115e+09], type=Float32), array([ 104.18000031, 
> > 104.45999908], type=Float32), array([ 1994.94995117,  1994.95996094],
> > type=Float32))
> >
> > >>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
> >
> > (20, 2)
> >
> > Any comments are appreciated,
>
> I talked to JC Hsu, the numarray.records author, and he explained that
> we're probably looking at a limitation of numarray.records: it doesn't
> yet handle multi-dimensional arrays of records.  JC indicated he had
> replied to Sebastian,  but for the benefit of everyone else, that's the
> deal.
>
> Regards,
> Todd
>

Thanks,
but it actually seems to work well now, as long as I use myRecArray[i][j] 
instead of myRecArray[i,j].
So, it looks like there is not missing much.
I'll keep your "official statement" in mind when I explore this further...

Thanks again,
Sebastian Haase

Re: [Numpy-discussion] bug ? in Records arrays in numarray

From: Russell E O. <rowen@u.washington.edu> - 2004-06-30 15:55:59

At 7:49 PM -0400 2004-06-29, Todd Miller wrote:
>On Tue, 2004-06-29 at 17:52, Sebastian Haase wrote:
>>  OK,
>>  I'm still trying to get a handle on these record arrays - because 
>>I think they
>>  are pretty cool, if I could get them to work...
>>  Following the code from yesterday (see that posting below) I 
>>discovered this:
>>  main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
>>  is this intended ??
>>
>>  >>> main.ring4ext[0][0]
>>  (2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
>>  >>> main.ring4ext[0,0]
>>  (array([2308, 2309]), array([76, 76]), array([272, 269]), array([ 
>>1.08848115e
>>  +09,   1.08848115e+09], type=Float32), array([ 104.18000031,  104.45999908],
>>  type=Float32), array([ 1994.94995117,  1994.95996094], type=Float32))
>>  >>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
>>  (20, 2)
>>
>>  Any comments are appreciated,
>
>I talked to JC Hsu, the numarray.records author, and he explained that
>we're probably looking at a limitation of numarray.records: it doesn't
>yet handle multi-dimensional arrays of records.  JC indicated he had
>replied to Sebastian,  but for the benefit of everyone else, that's the
>deal.

I agree. I have gotten numarray.records to handle multi-dimensional 
arrays, but it's a terrible pain to create them, str(arry) fails and 
setting elements of records arrays is painful. I hope at some point 
they get a major redesign, as they don't actually seem to have been 
designed to fit in with numarray. The resulting code was so ugly that 
I gave up and used multiple identically shaped arrays instead.

-- Russell

Re: [Numpy-discussion] bug ? in Records arrays in numarray

From: Perry G. <pe...@st...> - 2004-06-30 16:07:32

On Jun 30, 2004, at 11:55 AM, Russell E Owen wrote:

> I agree. I have gotten numarray.records to handle multi-dimensional 
> arrays, but it's a terrible pain to create them, str(arry) fails and 
> setting elements of records arrays is painful. I hope at some point 
> they get a major redesign, as they don't actually seem to have been 
> designed to fit in with numarray. The resulting code was so ugly that 
> I gave up and used multiple identically shaped arrays instead.
>
I think we will be taking a look at that soon. I agree that they could 
be generalized
to work better with numarray. Hopefully we will be soliciting comments 
in the next
few weeks about the best way to do that.

Perry