numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] real_fft2d

[Warren F. <fo...@SL...>]

On Tue, 17 Feb 2004, Scott Rifkin wrote:

> I'd like to take the 2d fourier transform of an image, remove some of the
> lower frequency signal, take the inverse 2d fourier transform, and then
> reconstruct the image.  I'm trying to use the python imaging library to do
> the image part and FFT for the fourier transform part.
>
> When I take the real_fft2d of a matrix (integer valued in this
> case-UInt16), I get a new matrix of the same size with complex entries.

That's surprising.  Input of shape (n, m) to real_fft2d should produce
output of shape (n, m/2+1).  fft2d should produce output of the same shape
as the input.

> In this implementation of the fourier transform, how is this resulting
> matrix arranged?  There seem to be a few different conventions out there
> for what, say, entry [0,0] represents.  Which one does FFT use?

Entry [0,0] of the output contains the DC term.

If your input is 4x4, your output should be 4x3, with frequencies arrnged
thus:

[[[ 0,0], [ 0,1], [ 0,2],
  [ 1,0], [ 1,1], [ 1,2],
  [ 2,0], [ 2,1], [ 2,2],
  [-1,0], [-1,1], [-1,2]]]

so if you want to zero out frequencies 0 and 1, multiply  the transform by
a mask like:

[[0, 0, 1],
 [0, 0, 1],
 [1, 1, 1],
 [0, 0, 1]]

> If I want to remove signal from wavelengths longer than a certain interval
> (say anything over 200 pixels), do I need to process the result from
> real_fft2d more or can I do the removal directly on that matrix and then
> take the inverse directly?

That should work.

> Is there a good reference for the FFT package or the FFTPACK library out
> there?

The doc strings may be on the terse side, and you have to read them for
several functions to find the answers to all of the questions you ask
here, but they are in there.

Docs for FFTPACK are at http://www.netlib.org/fftpack/doc

Warren Focke

[Numpy-discussion] real_fft2d

[Scott R. <sco...@ya...>]

I'd like to take the 2d fourier transform of an image, remove some of the 
lower frequency signal, take the inverse 2d fourier transform, and then 
reconstruct the image.  I'm trying to use the python imaging library to do 
the image part and FFT for the fourier transform part.

When I take the real_fft2d of a matrix (integer valued in this
case-UInt16), I get a new matrix of the same size with complex entries.  
In this implementation of the fourier transform, how is this resulting 
matrix arranged?  There seem to be a few different conventions out there 
for what, say, entry [0,0] represents.  Which one does FFT use?

If I want to remove signal from wavelengths longer than a certain interval 
(say anything over 200 pixels), do I need to process the result from 
real_fft2d more or can I do the removal directly on that matrix and then 
take the inverse directly?

Is there a good reference for the FFT package or the FFTPACK library out 
there?

Thanks much,
Scott Rikin

Re: [Numpy-discussion] numarrays of objects

[Todd M. <jm...@st...>]

On Tue, 2004-02-17 at 14:04, Jon Peirce wrote:
> Numeric used to handle arrays of arbitrary python objects. It was handy 
> because you can manipulate the shape of a numeric array much more easily 
> than a list-lof-lists.
> 
> Has that functionality been lost in numarray?
> Jon
Try numarray.objects and let me know how it goes.  I think
numarray.objects has seen very little usage so feedback is welcome.


Regards,
Todd
-- 
Todd Miller <jm...@st...>

[Numpy-discussion] numarrays of objects

[Jon P. <Jon...@no...>]

Numeric used to handle arrays of arbitrary python objects. It was handy 
because you can manipulate the shape of a numeric array much more easily 
than a list-lof-lists.

Has that functionality been lost in numarray?
Jon

-- 
Jon Peirce
Nottingham University
+44 (0)115 8467176 (tel)
+44 (0)115 9515324 (fax)

http://www.psychology.nottingham.ac.uk/staff/jwp/

[Numpy-discussion] Bachelor's Diploma, Master's, or PhD - No Classes Necessary, descriptive gateway blurt canary

[Sergio L. <DSN...@li...>]

sarsaparilla tetragonal bateau musicale distaff bayonet bombastic philistine 
lao thine ascetic trademark berwick agreed longitude cart indecent honda 
prove carl deformation dunlap algerian buckaroo armour beckman 

Re: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?)

[Todd M. <jm...@st...>]

On Wed, 2004-02-11 at 18:19, Tim Hochberg wrote:
> An update:
> 
> A little more tuning resulted in determinant and inverse being about 80x 
> faster than the original numarray code and about 5 times faster than 
> using NumPy for the same test cases I was using before (1000x4x4 
> matrices). If anyone is interested, let me know and I'll send you the code.
> 
> -tim

I think we should use the work you've done here in numarray... so we're
interested.  Unless you object,  I'll gleefully include your code as
drop-in replacements for the existing routines.  

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?)

[Tim H. <tim...@co...>]

An update:

A little more tuning resulted in determinant and inverse being about 80x 
faster than the original numarray code and about 5 times faster than 
using NumPy for the same test cases I was using before (1000x4x4 
matrices). If anyone is interested, let me know and I'll send you the code.

-tim

Tim Hochberg wrote:

>
> I discovered that some (all?) of the functions in 
> numarray.linear_algebra are very slow when operating on small 
> matrices. In particular, determinant and inverse are both more than 15 
> times slower than their NumPy counterparts when operating on 4x4 
> matrices. I assume that this is simply a result of numarray's higher 
> overhead.
>
> Normally the overhead of numarray is not much of a problem since when 
> I'm operating on lots of small data chunks I can usually agregate them 
> into larger chunks and operate on the big chunks. This is, of course, 
> the standard way to get decent performance in either numarray or 
> NumPy. However, because the functions in linear_algebra take only 
> rank-2 (or 1 in some cases) arrays, their is no way to aggregate the 
> small operations and thus things run quite slow.
>
> In order to address this I rewrote some of the functions in 
> linear_algebra to allow an additional, optional, dimension on the 
> input arrays. Rank-3 arrays are treated as being a set of matrices 
> that are indexed along the first axis of A. Thus determinant(A) is 
> essentially equivalent to array(map(determinant, A)) when A is rank-3. 
> See the attached file for more detail.
>
> By this trick and by some relentless tuning, I got the numarray 
> functions to run at about the same speed as their NumPy counterparts 
> when computing the determinants and inverses of 1000 4x4 matrices. 
> That's a humungous speedup.
>
> Is this approach worth pursuing for linear_algebra in general? I'll be 
> using these myself since I need the speed, although I may back out 
> some of the more aggresive  tuning so I don't get bit if numarray's 
> internals change. I'll gladly donate this code to numarray if it's 
> wanted, and I'm willing to help convert the rest, although it probaly 
> wouldn't happen as fast as this stuff since I don't need it myself 
> presently.
>
> -tim
>
> [Use this with caution at this point -- I just got finished with a 
> tuning spree and there may well be some bugs]
>
>------------------------------------------------------------------------
>

Re: [Numpy-discussion] why numarray.inputarray?

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 17:03, ve...@em... wrote:
> > > 2. Redefine asarray() as calling array() with copy=1.  Tim's right.  
> > > It's a better name than inputarray() and we want it for backward 
> > > compatibility anyway. 
>  
> In that case a copy is always made, right? You only want a copy when a 
> conversion is done, otherwise a reference should be returned. That is the 
> current behaviour with 0.8 and so does Numeric, if I recall well. 

I misspoke here.  It's correct in CVS.

Thanks,
Todd

>  
> Peter 
>  
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] _contiguous A trap for the unwary

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 16:43, Colin J. Williams wrote:
> The assignment of a value to self._shape has an unexpected side-effect.
> 
> self.iscontiguous() changes from True to False     -  desirable, but 
> unexpected
> self._contiguous remains with the value 2.
> 
> I had expected _contiguous to be in step with iscontiguous().
> 
> How does the function iscontiguous() determine contiguity?
> Is the variable now redundant?

As I recall, the attribute is used to override the function for
testing.  It's private, so ignore it. 

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] why numarray.inputarray?

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 16:20, Sebastian Haase wrote:
> Hi,
> I'm using memory mapped array and it sounds quite natural to say
> 'buffer=memmapSoAndSo'
> Are you actually saying that 'sequence' is a synonym for that and the
> preferred keyword ?

Yes.  The decision was made last year to support the sequence keyword
for the numarray.array() rather than buffer.  buffer has been deprecated
for almost a year so I think it's time to clean up the code.  I
personally like buffer better as well,  but I can't stand the code
clutter any longer... it should be one or the other not both.

Regards,
Todd

> - Sebastian Haase
> UCSF
> 
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] A bug in repeat function?

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 03:21, Nadav Horesh wrote:
> The repeat appears to accpt only short boolean array as repat counters:
> 
> >>> A
> array([1, 2, 3])
> >>> repeat(A, [1,1,1])
> array([1, 2, 3])
> >>> U = array([1,1,1], type=Bool)
> >>> U
> array([1, 1, 1], type=Bool)
> >>> repeat(A,U)    # OK
> array([1, 2, 3])
> >>> A = arange(100000)
> >>> U = ones(100000, type=Bool)
> >>> AA = repeat(A,U)   # U is too long --- an error is generated
> 
> Traceback (most recent call last):
>   File "<pyshell#90>", line 1, in -toplevel-
>     AA = repeat(A,U)
>   File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat
>     return _repeat(_nc.array(array), repeats)
>   File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat
>     newarray = array.__class__(shape=newshape, type=array._type)
> ValueError: new_memory: invalid region size: -384.
> >>> U = ones(100000)   # It is OK if U type is an integer
> >>> AA = repeat(A,U)
> >>> 
> 
>   Nadav.

This turned out to be a problem with summing the bool array to get an
element count for the result array from the repeat.  This is fixed now
in CVS.

Regards,
Todd
> 
-- 
Todd Miller <jm...@st...>

[Numpy-discussion] New version of numarray manual

[Todd M. <jm...@st...>]

JC Hsu did some work revising the numarray manual.  There's a new
version of the numarray manual for 0.8 at Source Forge here:

http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367&release_id=203420

Enjoy!

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] why numarray.inputarray?

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 16:17, Colin J. Williams wrote:
> Todd Miller wrote:
> 
> >On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote:
> >  
> >
> >>Tim Hochberg wrote:
> >>
> >>    
> >>
> >>>I just noticed that the name asarray is considered obsolete and is to 
> >>>be replaced with inputarray. Why is that? The name asarray has a 
> >>>pretty clear connection with it's behaviour (return it's argument as 
> >>>an array), while I can come up with no such connection for inputarray. 
> >>>The best I can come up with is that it's often used on the inputs of 
> >>>functions to coerce them to arrays, but that seems a pretty tenuous 
> >>>connection. Is there a better reason? Am I just being obtuse?
> >>>
> >>>-tim
> >>>      
> >>>
> >>I agree that the term asarray better describes the functionality. 
> >>
> >>One possible use would be to convert an instance of a sub-class of 
> >>NumArray into an instance of the class NumArray.  However, it doesn't do 
> >>that.
> >>
> >>It would help if there were a clearer distinction between the functions 
> >>array and inputarray.  The former would appear to cast a wider net and, 
> >>among other things, include files.
> >>    
> >>
> >
> >This is a mess, I have to agree.  The most important distinction is that
> >array() makes a copy of an array by default, while
> >inputarray()/asarray() does not.
> >
> >  
> >
> >>I wonder whether there is a need for both array and input array, 
> >>wouldn't it be best to focus on the function with the more general utility?
> >>    
> >>
> >Based on your input, and Tim Hochberg's earlier comments,  I think we
> >should:
> >
> >1. Clean up the array function by ditching the buffer/sequence keyword
> >deprecation code.  buffer has been deprecated long enough.
> >
> Good!  Although it means some changes in my not fully tested code.  I 
> believe that sequence can also be a buffer, as long as type and shape 
> are also provided.  I haven't yet tried this.
> 
> >
> >2. Redefine asarray() as calling array() with copy=1.  Tim's right. 
> >It's a better name than inputarray() and we want it for backward
> >compatibility anyway.
> >
> Good!  Although it would be good to make provision for this to convert a 
> sub-class instance into an instance of  NumArray,  something like:
>             if instance(sequence, NumArray):
>                if sequence.__class__ !=  NumArray.__class__:
>                    sequence.__class__ = NumArray.__class__
>                     copy= True

I'd like to hear more opinions on this.  

> >
> >3. Redefine inputarray() as a non-recommended synonym for asarray(). 
> >I'd say deprecate it,  but that's a PITA for anyone already using it.
> >  
> >
> It wouldn't be that much of a pain if the deprecation were in the 
> documentation only and put into the code after say a year.

Consider it done and get back to me in a year.

> 
> >4. Change the manual to document asarray() as primary and document
> >inputarray() as a non-recommended synonym.
> >
> Yes.  Incidentally, what is the current status of the documentation?  
> The last time I looked, it was for version 0.7.
> 

JC Hsu did a lot of work revising the manual.  I just posted the new
version on Source Forge as part of the 0.8 file release.

Todd
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] Garbage collection fails after numarray exception

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote:
> Hi all,
> 
> I'm not sure if I have found a bug, or simply a type of behaviour
> that should not have surprised me -- but I most definitely did
> not expect the following numarray 0.8 behaviour:
> 
> 
>    >>> import numarray
>    >>> a = numarray.zeros(100000000)
>    >>> del a
>    >>> a = numarray.zeros(100000000)
>    >>> a + ''
>    Traceback (most recent call last):
>      File "<stdin>", line 1, in ?
>      File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__
>        return ufunc.add(self, operand)
>      File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2
>        (in1, in2), insig, scalar = _inputcheck(n1, n2)
>      File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck    raise TypeError(
>    TypeError: UFunc arguments must be numarray, scalars or numeric sequences
>    >>> del a
>    >>>
> 
> 
> If I execute these statements alongside 'top' or another memory
> monitor, I of course see a big memory increase after each call to
> 'zeros', as well as a big decrease after the first 'del a' -- but
> no change whatsoever any more after the second 'del a', not even
> if I explicitly call the garbage collector via the gc module.
> 
> As far as I can tell, the occurrence of the exception somehow
> causes a permanent increase in a's refcount, with a big memory
> leak as a result.
> 
> (In case anyone's curious about the context for this, the 
> """ a + '' """" construct was taken straight from the Python
> Cookbook, as a way of determining whether an object exhibits
> string-like behaviour (useful for printing methods, etc.))
> 
> I understand that I can easily work around this problem so that
> numarray objects will not be fed to this construct to begin with,
> but I do wonder if it is really true (and intended) that numarray
> exceptions have the side effect of making certain potentially
> huge objects indestructible.
> 
> Can anyone here shed some light on this?

It looks like a bug.  I logged it on Source Forge and I'm working on the
solution.  

> Many thanks in advance,

Thanks for the report,
Todd

-- 
Todd Miller <jm...@st...>

[Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?)

[Tim H. <tim...@co...>]

I discovered that some (all?) of the functions in 
numarray.linear_algebra are very slow when operating on small matrices. 
In particular, determinant and inverse are both more than 15 times 
slower than their NumPy counterparts when operating on 4x4 matrices. I 
assume that this is simply a result of numarray's higher overhead.

Normally the overhead of numarray is not much of a problem since when 
I'm operating on lots of small data chunks I can usually agregate them 
into larger chunks and operate on the big chunks. This is, of course, 
the standard way to get decent performance in either numarray or NumPy. 
However, because the functions in linear_algebra take only rank-2 (or 1 
in some cases) arrays, their is no way to aggregate the small operations 
and thus things run quite slow.

In order to address this I rewrote some of the functions in 
linear_algebra to allow an additional, optional, dimension on the input 
arrays. Rank-3 arrays are treated as being a set of matrices that are 
indexed along the first axis of A. Thus determinant(A) is essentially 
equivalent to array(map(determinant, A)) when A is rank-3. See the 
attached file for more detail.

By this trick and by some relentless tuning, I got the numarray 
functions to run at about the same speed as their NumPy counterparts 
when computing the determinants and inverses of 1000 4x4 matrices. 
That's a humungous speedup.

Is this approach worth pursuing for linear_algebra in general? I'll be 
using these myself since I need the speed, although I may back out some 
of the more aggresive  tuning so I don't get bit if numarray's internals 
change. I'll gladly donate this code to numarray if it's wanted, and I'm 
willing to help convert the rest, although it probaly wouldn't happen as 
fast as this stuff since I don't need it myself presently.

-tim

[Use this with caution at this point -- I just got finished with a 
tuning spree and there may well be some bugs]

Re: [Numpy-discussion] why numarray.inputarray?

[<ve...@em...>]

> > 2. Redefine asarray() as calling array() with copy=1.  Tim's right.  
> > It's a better name than inputarray() and we want it for backward 
> > compatibility anyway. 
 
In that case a copy is always made, right? You only want a copy when a 
conversion is done, otherwise a reference should be returned. That is the 
current behaviour with 0.8 and so does Numeric, if I recall well. 
 
Peter 
 

[Numpy-discussion] _contiguous A trap for the unwary

[Colin J. W. <cj...@sy...>]

The assignment of a value to self._shape has an unexpected side-effect.

self.iscontiguous() changes from True to False     -  desirable, but 
unexpected
self._contiguous remains with the value 2.

I had expected _contiguous to be in step with iscontiguous().

How does the function iscontiguous() determine contiguity?
Is the variable now redundant?

Colin W.

Re: [Numpy-discussion] why numarray.inputarray?

[Sebastian H. <ha...@ms...>]

> Todd Miller wrote:
>
> [SNIP]
>
> >Based on your [Colin Williams] input, and Tim Hochberg's earlier
comments,  I think we
> >should:
> >
> >1. Clean up the array function by ditching the buffer/sequence keyword
> >deprecation code.  buffer has been deprecated long enough.
> >
> >2. Redefine asarray() as calling array() with copy=1.  Tim's right.
> >It's a better name than inputarray() and we want it for backward
> >compatibility anyway.
> >
> >3. Redefine inputarray() as a non-recommended synonym for asarray().
> >I'd say deprecate it,  but that's a PITA for anyone already using it.
> >
> >4. Change the manual to document asarray() as primary and document
> >inputarray() as a non-recommended synonym.
> >
> >Comments?
> >
> >
> >
> Sounds good to me. I've never used the buffer/sequence stuff, so I'm not
> really qualified to opine on that.
>
Hi,
I'm using memory mapped array and it sounds quite natural to say
'buffer=memmapSoAndSo'
Are you actually saying that 'sequence' is a synonym for that and the
preferred keyword ?

- Sebastian Haase
UCSF

Re: [Numpy-discussion] why numarray.inputarray?

[Colin J. W. <cj...@sy...>]

Todd Miller wrote:

>On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote:
>  
>
>>Tim Hochberg wrote:
>>
>>    
>>
>>>I just noticed that the name asarray is considered obsolete and is to 
>>>be replaced with inputarray. Why is that? The name asarray has a 
>>>pretty clear connection with it's behaviour (return it's argument as 
>>>an array), while I can come up with no such connection for inputarray. 
>>>The best I can come up with is that it's often used on the inputs of 
>>>functions to coerce them to arrays, but that seems a pretty tenuous 
>>>connection. Is there a better reason? Am I just being obtuse?
>>>
>>>-tim
>>>      
>>>
>>I agree that the term asarray better describes the functionality. 
>>
>>One possible use would be to convert an instance of a sub-class of 
>>NumArray into an instance of the class NumArray.  However, it doesn't do 
>>that.
>>
>>It would help if there were a clearer distinction between the functions 
>>array and inputarray.  The former would appear to cast a wider net and, 
>>among other things, include files.
>>    
>>
>
>This is a mess, I have to agree.  The most important distinction is that
>array() makes a copy of an array by default, while
>inputarray()/asarray() does not.
>
>  
>
>>I wonder whether there is a need for both array and input array, 
>>wouldn't it be best to focus on the function with the more general utility?
>>    
>>
>Based on your input, and Tim Hochberg's earlier comments,  I think we
>should:
>
>1. Clean up the array function by ditching the buffer/sequence keyword
>deprecation code.  buffer has been deprecated long enough.
>
Good!  Although it means some changes in my not fully tested code.  I 
believe that sequence can also be a buffer, as long as type and shape 
are also provided.  I haven't yet tried this.

>
>2. Redefine asarray() as calling array() with copy=1.  Tim's right. 
>It's a better name than inputarray() and we want it for backward
>compatibility anyway.
>
Good!  Although it would be good to make provision for this to convert a 
sub-class instance into an instance of  NumArray,  something like:
            if instance(sequence, NumArray):
               if sequence.__class__ !=  NumArray.__class__:
                   sequence.__class__ = NumArray.__class__
                    copy= True

>
>3. Redefine inputarray() as a non-recommended synonym for asarray(). 
>I'd say deprecate it,  but that's a PITA for anyone already using it.
>  
>
It wouldn't be that much of a pain if the deprecation were in the 
documentation only and put into the code after say a year.

>4. Change the manual to document asarray() as primary and document
>inputarray() as a non-recommended synonym.
>
Yes.  Incidentally, what is the current status of the documentation?  
The last time I looked, it was for version 0.7.

Colin W.

>
>Comments?
>
>Todd
>
>  
>
>>Colin W.
>>
>>    
>>
>>>
>>>-------------------------------------------------------
>>>The SF.Net email is sponsored by EclipseCon 2004
>>>Premiere Conference on Open Tools Development and Integration
>>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
>>>http://www.eclipsecon.org/osdn
>>>_______________________________________________
>>>Numpy-discussion mailing list
>>>Num...@li...
>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>>      
>>>
>>
>>-------------------------------------------------------
>>The SF.Net email is sponsored by EclipseCon 2004
>>Premiere Conference on Open Tools Development and Integration
>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
>>http://www.eclipsecon.org/osdn
>>_______________________________________________
>>Numpy-discussion mailing list
>>Num...@li...
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>

Re: [Numpy-discussion] why numarray.inputarray?

[Tim H. <tim...@co...>]

Todd Miller wrote:

[SNIP]

>Based on your [Colin Williams] input, and Tim Hochberg's earlier comments,  I think we
>should:
>
>1. Clean up the array function by ditching the buffer/sequence keyword
>deprecation code.  buffer has been deprecated long enough.
>
>2. Redefine asarray() as calling array() with copy=1.  Tim's right. 
>It's a better name than inputarray() and we want it for backward
>compatibility anyway.
>
>3. Redefine inputarray() as a non-recommended synonym for asarray(). 
>I'd say deprecate it,  but that's a PITA for anyone already using it.
>
>4. Change the manual to document asarray() as primary and document
>inputarray() as a non-recommended synonym.
>
>Comments?
>
>  
>
Sounds good to me. I've never used the buffer/sequence stuff, so I'm not 
really qualified to opine on that.

-tim

[Numpy-discussion] Garbage collection fails after numarray exception

[Leo B. <le...@ls...>]

Hi all,

I'm not sure if I have found a bug, or simply a type of behaviour
that should not have surprised me -- but I most definitely did
not expect the following numarray 0.8 behaviour:


   >>> import numarray
   >>> a = numarray.zeros(100000000)
   >>> del a
   >>> a = numarray.zeros(100000000)
   >>> a + ''
   Traceback (most recent call last):
     File "<stdin>", line 1, in ?
     File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__
       return ufunc.add(self, operand)
     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2
       (in1, in2), insig, scalar = _inputcheck(n1, n2)
     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck    raise TypeError(
   TypeError: UFunc arguments must be numarray, scalars or numeric sequences
   >>> del a
   >>>


If I execute these statements alongside 'top' or another memory
monitor, I of course see a big memory increase after each call to
'zeros', as well as a big decrease after the first 'del a' -- but
no change whatsoever any more after the second 'del a', not even
if I explicitly call the garbage collector via the gc module.

As far as I can tell, the occurrence of the exception somehow
causes a permanent increase in a's refcount, with a big memory
leak as a result.

(In case anyone's curious about the context for this, the 
""" a + '' """" construct was taken straight from the Python
Cookbook, as a way of determining whether an object exhibits
string-like behaviour (useful for printing methods, etc.))

I understand that I can easily work around this problem so that
numarray objects will not be fed to this construct to begin with,
but I do wonder if it is really true (and intended) that numarray
exceptions have the side effect of making certain potentially
huge objects indestructible.

Can anyone here shed some light on this?

Many thanks in advance,

-- 
Leo Breebaart  <le...@ls...>

Re: [Numpy-discussion] why numarray.inputarray?

[Todd M. <jm...@st...>]

On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote:
> Tim Hochberg wrote:
> 
> >
> > I just noticed that the name asarray is considered obsolete and is to 
> > be replaced with inputarray. Why is that? The name asarray has a 
> > pretty clear connection with it's behaviour (return it's argument as 
> > an array), while I can come up with no such connection for inputarray. 
> > The best I can come up with is that it's often used on the inputs of 
> > functions to coerce them to arrays, but that seems a pretty tenuous 
> > connection. Is there a better reason? Am I just being obtuse?
> >
> > -tim
> 
> I agree that the term asarray better describes the functionality. 
> 
> One possible use would be to convert an instance of a sub-class of 
> NumArray into an instance of the class NumArray.  However, it doesn't do 
> that.
> 
> It would help if there were a clearer distinction between the functions 
> array and inputarray.  The former would appear to cast a wider net and, 
> among other things, include files.

This is a mess, I have to agree.  The most important distinction is that
array() makes a copy of an array by default, while
inputarray()/asarray() does not.

> I wonder whether there is a need for both array and input array, 
> wouldn't it be best to focus on the function with the more general utility?
Based on your input, and Tim Hochberg's earlier comments,  I think we
should:

1. Clean up the array function by ditching the buffer/sequence keyword
deprecation code.  buffer has been deprecated long enough.

2. Redefine asarray() as calling array() with copy=1.  Tim's right. 
It's a better name than inputarray() and we want it for backward
compatibility anyway.

3. Redefine inputarray() as a non-recommended synonym for asarray(). 
I'd say deprecate it,  but that's a PITA for anyone already using it.

4. Change the manual to document asarray() as primary and document
inputarray() as a non-recommended synonym.

Comments?

Todd

> 
> Colin W.
> 
> >
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > Numpy-discussion mailing list
> > Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] why numarray.inputarray?

[Tim H. <tim...@co...>]

Todd Miller wrote:

>On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote:
>  
>
>>I just noticed that the name asarray is considered obsolete and is to be 
>>replaced with inputarray. Why is that? 
>>    
>>
>
>I'm not sure what gave you that impression;  asarray() isn't obsolete.
>  
>
It was this:

*asarray*( 	seq, type=None, typecode=None)

An alias for inputarray. This is deprecated and only provided for 
compatibility with Numeric.

from http://stsdas.stsci.edu/numarray/Doc/node21.html#l2h-6. Perhaps 
these docs should be tweaked a little not to freak out sensitive people 
like me <0.5 wink>.

Regards,

-tim




>inputarray() was implemented "independently" and then I discovered
>afterward that it should have been called asarray().  At the time,  I
>was hesitant to rename public functions so I just added the asarray()
>alias rather than renaming inputarray().
>
>  
>
>>The name asarray has a pretty 
>>clear connection with it's behaviour (return it's argument as an array), 
>>while I can come up with no such connection for inputarray. The best I 
>>can come up with is that it's often used on the inputs of functions to 
>>coerce them to arrays, but that seems a pretty tenuous connection. Is 
>>there a better reason? 
>>    
>>
>
>Nope.
>
>  
>
>>Am I just being obtuse?
>>
>>-tim
>>
>>
>>
>>-------------------------------------------------------
>>The SF.Net email is sponsored by EclipseCon 2004
>>Premiere Conference on Open Tools Development and Integration
>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
>>http://www.eclipsecon.org/osdn
>>_______________________________________________
>>Numpy-discussion mailing list
>>Num...@li...
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>

Re: [Numpy-discussion] why numarray.inputarray?

[Colin J. W. <cj...@sy...>]

Tim Hochberg wrote:

>
> I just noticed that the name asarray is considered obsolete and is to 
> be replaced with inputarray. Why is that? The name asarray has a 
> pretty clear connection with it's behaviour (return it's argument as 
> an array), while I can come up with no such connection for inputarray. 
> The best I can come up with is that it's often used on the inputs of 
> functions to coerce them to arrays, but that seems a pretty tenuous 
> connection. Is there a better reason? Am I just being obtuse?
>
> -tim

I agree that the term asarray better describes the functionality. 

One possible use would be to convert an instance of a sub-class of 
NumArray into an instance of the class NumArray.  However, it doesn't do 
that.

It would help if there were a clearer distinction between the functions 
array and inputarray.  The former would appear to cast a wider net and, 
among other things, include files.

I wonder whether there is a need for both array and input array, 
wouldn't it be best to focus on the function with the more general utility?

Colin W.

>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

Re: [Numpy-discussion] why numarray.inputarray?

[Todd M. <jm...@st...>]

On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote:
> I just noticed that the name asarray is considered obsolete and is to be 
> replaced with inputarray. Why is that? 

I'm not sure what gave you that impression;  asarray() isn't obsolete.

inputarray() was implemented "independently" and then I discovered
afterward that it should have been called asarray().  At the time,  I
was hesitant to rename public functions so I just added the asarray()
alias rather than renaming inputarray().

> The name asarray has a pretty 
> clear connection with it's behaviour (return it's argument as an array), 
> while I can come up with no such connection for inputarray. The best I 
> can come up with is that it's often used on the inputs of functions to 
> coerce them to arrays, but that seems a pretty tenuous connection. Is 
> there a better reason? 

Nope.

> Am I just being obtuse?
> 
> -tim
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>