numpy-discussion

[Numpy-discussion] Deleting a row from a matrix

[Keith G. <kwg...@gm...>]

How do I delete a row (or list of rows) from a matrix object?

To remove the n'th row in octave I use x(n,:) = []. Or n could be a
vector of rows to remove.

In numpy 0.9.9.2813 x[[1,2],:] = [] changes the values of all the
elements of x without changing the size of x.

In numpy do I have to turn it around and construct a list of the rows
I want to keep?

[Numpy-discussion] Deleting a row from a matrix

[Nick F. <nv...@MI...>]

On Aug 26, 2006, at 7:05 AM, Keith Goodman wrote:
> On 8/26/06, Bill Baxter <wb...@gm...> wrote:
>> On 8/26/06, Travis Oliphant <oli...@ie...> wrote:
>>
>>>
>>> I've come up with adding the functions (not methods at this point)
>>>
>>> deletefrom
>>> insertinto
>>
>>
>> "delete" and "insert" really would be better.   The current "insert"
>> function seems inaptly named.  What it does sounds more like  
>> "overlay" or
>> "set_masked".
>
> I prefer delete and insert too. I guess it is OK that del and delete
> are similar (?)

How about "deleted" and "inserted" to parallel "sorted"?  "delete"  
and "insert" sound very imperative and side-effects-ish.

Nick

Re: [Numpy-discussion] Deleting a row from a matrix

[Travis O. <oli...@ie...>]

Keith Goodman wrote:
> How do I delete a row (or list of rows) from a matrix object?
>
> To remove the n'th row in octave I use x(n,:) = []. Or n could be a
> vector of rows to remove.
>
> In numpy 0.9.9.2813 x[[1,2],:] = [] changes the values of all the
> elements of x without changing the size of x.
>
> In numpy do I have to turn it around and construct a list of the rows
> I want to keep?
>   
Basically, that is true for now.  

I think it would be worth implementing some kind of function for making 
this easier.

One might think of using:

del a[obj]

But, the problem with both of those approaches is that once you start 
removing arbitrary rows (or n-1 dimensional sub-spaces) from an array 
you very likely will no longer have a chunk of memory that can be 
described using the n-dimensional array memory model.

So, you would have to make memory copies.  This could be done, of 
course, and the data area of "a" altered appropriately.  But, such 
alteration of the memory would break any other objects that have a 
"view" of the memory area of "a."  Right now, there is no way to track 
which objects have such "views", and therefore no good way to tell 
(other than the very conservative reference count) if it is safe to 
re-organize the memory of "a" in this way.

So, "in-place" deletion of array objects would not be particularly 
useful, because it would only work for arrays with no additional 
reference counts (i.e. simple b=a assignment would increase the 
reference count and make it impossible to say del a[obj]).

However, a function call that returned a new array object with the 
appropriate rows deleted (implemented by constructing a new array with 
the remaining rows) would seem to be a good idea. 

I'll place a prototype (named delete) to that effect into SVN soon.

-Travis

Re: [Numpy-discussion] Deleting a row from a matrix

[Sebastian H. <ha...@ms...>]

On Friday 25 August 2006 07:01, Travis Oliphant wrote:
> Keith Goodman wrote:
> > How do I delete a row (or list of rows) from a matrix object?
> >
> > To remove the n'th row in octave I use x(n,:) = []. Or n could be a
> > vector of rows to remove.
> >
> > In numpy 0.9.9.2813 x[[1,2],:] = [] changes the values of all the
> > elements of x without changing the size of x.
> >
> > In numpy do I have to turn it around and construct a list of the rows
> > I want to keep?
>
> Basically, that is true for now.
>
> I think it would be worth implementing some kind of function for making
> this easier.
>
> One might think of using:
>
> del a[obj]
>
> But, the problem with both of those approaches is that once you start
> removing arbitrary rows (or n-1 dimensional sub-spaces) from an array
> you very likely will no longer have a chunk of memory that can be
> described using the n-dimensional array memory model.
>
> So, you would have to make memory copies.  This could be done, of
> course, and the data area of "a" altered appropriately.  But, such
> alteration of the memory would break any other objects that have a
> "view" of the memory area of "a."  Right now, there is no way to track
> which objects have such "views", and therefore no good way to tell
> (other than the very conservative reference count) if it is safe to
> re-organize the memory of "a" in this way.
>
> So, "in-place" deletion of array objects would not be particularly
> useful, because it would only work for arrays with no additional
> reference counts (i.e. simple b=a assignment would increase the
> reference count and make it impossible to say del a[obj]).
>
> However, a function call that returned a new array object with the
> appropriate rows deleted (implemented by constructing a new array with
> the remaining rows) would seem to be a good idea.
>
> I'll place a prototype (named delete) to that effect into SVN soon.
>
> -Travis
>
Now of course: I often needed to "insert"  a column, row or section, ... ?
I made a quick and dirty implementation for that myself:
def insert(arr, i, entry, axis=0):
    """returns new array with new element inserted at index i along axis
    if arr.ndim>1 and if entry is scalar it gets filled in (ref. broadcasting)

    note: (original) arr does not get affected
    """
    if i > arr.shape[axis]:
        raise IndexError, "index i larger than arr size"
    shape = list(arr.shape)
    shape[axis] += 1
    a= N.empty(dtype=arr.dtype, shape=shape)
    aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
    aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
    aa[:i] = aarr[:i]
    aa[i+1:] = aarr[i:]
    aa[i] = entry
    return a

but maybe there is a way to put it it numpy directly.

- Sebastian

Re: [Numpy-discussion] Deleting a row from a matrix

[Travis O. <oli...@ie...>]

Sebastian Haase wrote:
> On Friday 25 August 2006 07:01, Travis Oliphant wrote:
>   
>> Keith Goodman wrote:
>>     
>>> How do I delete a row (or list of rows) from a matrix object?
>>>
>>> To remove the n'th row in octave I use x(n,:) = []. Or n could be a
>>> vector of rows to remove.
>>>
>>> In numpy 0.9.9.2813 x[[1,2],:] = [] changes the values of all the
>>> elements of x without changing the size of x.
>>>
>>> In numpy do I have to turn it around and construct a list of the rows
>>> I want to keep?
>>>       
>> Basically, that is true for now.
>>
>> I think it would be worth implementing some kind of function for making
>> this easier.
>>
>> One might think of using:
>>
>> del a[obj]
>>
>> But, the problem with both of those approaches is that once you start
>> removing arbitrary rows (or n-1 dimensional sub-spaces) from an array
>> you very likely will no longer have a chunk of memory that can be
>> described using the n-dimensional array memory model.
>>
>> So, you would have to make memory copies.  This could be done, of
>> course, and the data area of "a" altered appropriately.  But, such
>> alteration of the memory would break any other objects that have a
>> "view" of the memory area of "a."  Right now, there is no way to track
>> which objects have such "views", and therefore no good way to tell
>> (other than the very conservative reference count) if it is safe to
>> re-organize the memory of "a" in this way.
>>
>> So, "in-place" deletion of array objects would not be particularly
>> useful, because it would only work for arrays with no additional
>> reference counts (i.e. simple b=a assignment would increase the
>> reference count and make it impossible to say del a[obj]).
>>
>> However, a function call that returned a new array object with the
>> appropriate rows deleted (implemented by constructing a new array with
>> the remaining rows) would seem to be a good idea.
>>
>> I'll place a prototype (named delete) to that effect into SVN soon.
>>
>> -Travis
>>
>>     
> Now of course: I often needed to "insert"  a column, row or section, ... ?
> I made a quick and dirty implementation for that myself:
> def insert(arr, i, entry, axis=0):
>     """returns new array with new element inserted at index i along axis
>     if arr.ndim>1 and if entry is scalar it gets filled in (ref. broadcasting)
>
>     note: (original) arr does not get affected
>     """
>     if i > arr.shape[axis]:
>         raise IndexError, "index i larger than arr size"
>     shape = list(arr.shape)
>     shape[axis] += 1
>     a= N.empty(dtype=arr.dtype, shape=shape)
>     aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
>     aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
>     aa[:i] = aarr[:i]
>     aa[i+1:] = aarr[i:]
>     aa[i] = entry
>     return a
>   

Sure, it makes sense to parallel the delete function.

-Travis

Re: [Numpy-discussion] Deleting a row from a matrix

[Travis O. <oli...@ie...>]

Travis Oliphant wrote:
>> Now of course: I often needed to "insert"  a column, row or section, ... ?
>> I made a quick and dirty implementation for that myself:
>> def insert(arr, i, entry, axis=0):
>>     """returns new array with new element inserted at index i along axis
>>     if arr.ndim>1 and if entry is scalar it gets filled in (ref. broadcasting)
>>
>>     note: (original) arr does not get affected
>>     """
>>     if i > arr.shape[axis]:
>>         raise IndexError, "index i larger than arr size"
>>     shape = list(arr.shape)
>>     shape[axis] += 1
>>     a= N.empty(dtype=arr.dtype, shape=shape)
>>     aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
>>     aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
>>     aa[:i] = aarr[:i]
>>     aa[i+1:] = aarr[i:]
>>     aa[i] = entry
>>     return a
>>   
>>     
>
> Sure, it makes sense to parallel the delete function.
>   
Although there is already and insert function present in numpy....

-Travis

Re: [Numpy-discussion] Deleting a row from a matrix

[Sebastian H. <ha...@ms...>]

On Friday 25 August 2006 08:01, Travis Oliphant wrote:
> Travis Oliphant wrote:
> >> Now of course: I often needed to "insert"  a column, row or section, ...
> >> ? I made a quick and dirty implementation for that myself:
> >> def insert(arr, i, entry, axis=0):
> >>     """returns new array with new element inserted at index i along axis
> >>     if arr.ndim>1 and if entry is scalar it gets filled in (ref.
> >> broadcasting)
> >>
> >>     note: (original) arr does not get affected
> >>     """
> >>     if i > arr.shape[axis]:
> >>         raise IndexError, "index i larger than arr size"
> >>     shape = list(arr.shape)
> >>     shape[axis] += 1
> >>     a= N.empty(dtype=arr.dtype, shape=shape)
> >>     aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
> >>     aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
> >>     aa[:i] = aarr[:i]
> >>     aa[i+1:] = aarr[i:]
> >>     aa[i] = entry
> >>     return a
> >
> > Sure, it makes sense to parallel the delete function.
>
> Although there is already and insert function present in numpy....
>
> -Travis

Yeah - I saw that ...
maybe one could introduce consistent namings like
arr.copy_insert()
arr.copy_delete()
arr.copy_append()

for the new ones.  This emphasis the fact that a copy is created ... 
(Append is also a function often asked for when people expect "list 
capabilities" - did I miss others ?)

-Sebastian

Re: [Numpy-discussion] Deleting a row from a matrix

[Keith G. <kwg...@gm...>]

On 8/25/06, Travis Oliphant <oli...@ie...> wrote:

> I've come up with adding the functions (not methods at this point)
>
> deletefrom
> insertinto
>
> appendto  (syntatic sugar for concatenate but with a separate argument
> for the array and the extra stuff) --- is this needed?
>
> These functions will operate along a particular axis (default is axis=0
> to match concatenate).

It is probably obvious to everyone except me: what is the syntax?

If x is 5x5 and I want to delete rows 2 and 4 is it deletfrom(x, [1,3], axis=0)?

Re: [Numpy-discussion] Deleting a row from a matrix

[Travis O. <oli...@ie...>]

Keith Goodman wrote:
> On 8/25/06, Travis Oliphant <oli...@ie...> wrote:
>
>   
>> I've come up with adding the functions (not methods at this point)
>>
>> deletefrom
>> insertinto
>>
>> appendto  (syntatic sugar for concatenate but with a separate argument
>> for the array and the extra stuff) --- is this needed?
>>
>> These functions will operate along a particular axis (default is axis=0
>> to match concatenate).
>>     
>
> It is probably obvious to everyone except me: what is the syntax?
>   
No, I'm sure it isn't obvious to anyone. 

Here's what I'm implementing (I'm using the default axis=None now which 
I like because it's consistent with everything else and it forces you to 
pick an axis for >1d arrays --- this also gives some purpose for the 
appendonto function)

deletefrom(arr, obj, axis=None)  where obj is either an integer, a slice 
object, or a sequence of integers indicating the rows to delete:
> If x is 5x5 and I want to delete rows 2 and 4 is it deletfrom(x, [1,3], axis=0)?
>   
Yes, if you are counting from 1.

-Travis

Re: [Numpy-discussion] Deleting a row from a matrix

[Travis O. <oli...@ie...>]

Sebastian Haase wrote:
> On Friday 25 August 2006 08:01, Travis Oliphant wrote:
>   
>> Travis Oliphant wrote:
>>     
>>>> Now of course: I often needed to "insert"  a column, row or section, ...
>>>> ? I made a quick and dirty implementation for that myself:
>>>> def insert(arr, i, entry, axis=0):
>>>>     """returns new array with new element inserted at index i along axis
>>>>     if arr.ndim>1 and if entry is scalar it gets filled in (ref.
>>>> broadcasting)
>>>>
>>>>     note: (original) arr does not get affected
>>>>     """
>>>>     if i > arr.shape[axis]:
>>>>         raise IndexError, "index i larger than arr size"
>>>>     shape = list(arr.shape)
>>>>     shape[axis] += 1
>>>>     a= N.empty(dtype=arr.dtype, shape=shape)
>>>>     aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
>>>>     aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
>>>>     aa[:i] = aarr[:i]
>>>>     aa[i+1:] = aarr[i:]
>>>>     aa[i] = entry
>>>>     return a
>>>>         
>>> Sure, it makes sense to parallel the delete function.
>>>       
>> Although there is already and insert function present in numpy....
>>
>> -Travis
>>     
>
> Yeah - I saw that ...
> maybe one could introduce consistent namings like
> arr.copy_insert()
> arr.copy_delete()
> arr.copy_append()
>   

I've come up with adding the functions (not methods at this point)

deletefrom
insertinto

appendto  (syntatic sugar for concatenate but with a separate argument 
for the array and the extra stuff) --- is this needed?

These functions will operate along a particular axis (default is axis=0 
to match concatenate).

Comments?

-Travis

Re: [Numpy-discussion] Deleting a row from a matrix

[Sebastian H. <ha...@ms...>]

On Friday 25 August 2006 16:16, Travis Oliphant wrote:
> Sebastian Haase wrote:
> > On Friday 25 August 2006 08:01, Travis Oliphant wrote:
> >> Travis Oliphant wrote:
> >>>> Now of course: I often needed to "insert"  a column, row or section,
> >>>> ... ? I made a quick and dirty implementation for that myself:
> >>>> def insert(arr, i, entry, axis=0):
> >>>>     """returns new array with new element inserted at index i along
> >>>> axis if arr.ndim>1 and if entry is scalar it gets filled in (ref.
> >>>> broadcasting)
> >>>>
> >>>>     note: (original) arr does not get affected
> >>>>     """
> >>>>     if i > arr.shape[axis]:
> >>>>         raise IndexError, "index i larger than arr size"
> >>>>     shape = list(arr.shape)
> >>>>     shape[axis] += 1
> >>>>     a= N.empty(dtype=arr.dtype, shape=shape)
> >>>>     aa=N.transpose(a, [axis]+range(axis)+range(axis+1,a.ndim))
> >>>>     aarr=N.transpose(arr, [axis]+range(axis)+range(axis+1,arr.ndim))
> >>>>     aa[:i] = aarr[:i]
> >>>>     aa[i+1:] = aarr[i:]
> >>>>     aa[i] = entry
> >>>>     return a
> >>>
> >>> Sure, it makes sense to parallel the delete function.
> >>
> >> Although there is already and insert function present in numpy....
> >>
> >> -Travis
> >
> > Yeah - I saw that ...
> > maybe one could introduce consistent namings like
> > arr.copy_insert()
> > arr.copy_delete()
> > arr.copy_append()
>
> I've come up with adding the functions (not methods at this point)
>
> deletefrom
> insertinto
>
> appendto  (syntatic sugar for concatenate but with a separate argument
> for the array and the extra stuff) --- is this needed?

not for me.

-S.

Re: [Numpy-discussion] Deleting a row from a matrix

[Robert K. <rob...@gm...>]

Travis Oliphant wrote:

> I've come up with adding the functions (not methods at this point)
> 
> deletefrom
> insertinto
> 
> appendto  (syntatic sugar for concatenate but with a separate argument 
> for the array and the extra stuff) --- is this needed?
> 
> These functions will operate along a particular axis (default is axis=0 
> to match concatenate).
> 
> Comments?

I would drop appendto(). I also recommend leaving them as functions and not 
making methods from them. This will help prevent people from thinking that these 
modify the arrays in-place.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] Deleting a row from a matrix

[Francesc A. <fa...@ca...>]

Hi,

A Dissabte 26 Agost 2006 01:55, Robert Kern va escriure:
> Travis Oliphant wrote:
> > I've come up with adding the functions (not methods at this point)
> >
> > deletefrom
> > insertinto
> >
> > appendto  (syntatic sugar for concatenate but with a separate argument
> > for the array and the extra stuff) --- is this needed?
> >
> > These functions will operate along a particular axis (default is axis=
=3D0
> > to match concatenate).
> >
> > Comments?
>
> I would drop appendto(). I also recommend leaving them as functions and n=
ot
> making methods from them. This will help prevent people from thinking that
> these modify the arrays in-place.

But there are already quite a few methods in NumPy that doesn't modify the=
=20
array in-place (swapaxes, flatten, ravel or squeeze, but I guess many more).
I'm personally an addict to encapsulate as much functionality as possible i=
n=20
methods (but perhaps I'm biased by an insane use of TAB in ipython console).

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Deleting a row from a matrix

[Bill B. <wb...@gm...>]

On 8/26/06, Francesc Altet <fa...@ca...> wrote:
>
> I'm personally an addict to encapsulate as much functionality as possible
> in
> methods (but perhaps I'm biased by an insane use of TAB in ipython
> console).


You can still get tab completion for functions:  numpy.<TAB>
Even if it's your custom to "from numpy import *" you can still also do an
"import numpy" or "import numpy as N".

--bb

Re: [Numpy-discussion] Deleting a row from a matrix

[Francesc A. <fa...@ca...>]

A Dissabte 26 Agost 2006 13:42, Bill Baxter va escriure:
> On 8/26/06, Francesc Altet <fa...@ca...> wrote:
> > I'm personally an addict to encapsulate as much functionality as possib=
le
> > in
> > methods (but perhaps I'm biased by an insane use of TAB in ipython
> > console).
>
> You can still get tab completion for functions:  numpy.<TAB>
> Even if it's your custom to "from numpy import *" you can still also do an
> "import numpy" or "import numpy as N".

Yep, you are right. It is just that I tend to do that on the objects that I=
=20
manipulate and not with first-level functions in packages.

Anyway, I think that I see now that these routines should not be methods=20
because they modify the *actual* data on ndarrays.

Sorry for the disgression,

=2D-=20
>0,0<   Francesc Altet =C2=A0 =C2=A0 http://www.carabos.com/
V   V   C=C3=A1rabos Coop. V. =C2=A0=C2=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Deleting a row from a matrix

[Bill B. <wb...@gm...>]

On 8/26/06, Travis Oliphant <oli...@ie...> wrote:
>
>
> I've come up with adding the functions (not methods at this point)
>
> deletefrom
> insertinto


"delete" and "insert" really would be better.   The current "insert"
function seems inaptly named.  What it does sounds more like "overlay" or
"set_masked".

... or the existing "putmask" which I see does a similar thing.

Actually there seems to be a little doc-bug there or something.
numpy.insert claims it differs from putmask in that it only accepts a vector
of values with same number of vals as the # of non-zero entries in the mask,
but a quick test revals it's quite happy with a different number and cycles
through them.

In [31]: a = numpy.zeros((3,3))
In [32]: numpy.insert(a, [[0,1,0],[1,0,0],[1,0,0]], [4,5])
In [33]: a
Out[33]:
array([[ 0.,  4.,  0.],
       [ 5.,  0.,  0.],
       [ 4.,  0.,  0.]])

Anyway, in the end nothing has really been inserted, existing entries have
just been replaced.

So "insert" seems like a much better name for a function that actually puts
in a new row or column.

--bb

Re: [Numpy-discussion] Deleting a row from a matrix

[Keith G. <kwg...@gm...>]

On 8/26/06, Bill Baxter <wb...@gm...> wrote:
> On 8/26/06, Travis Oliphant <oli...@ie...> wrote:
>
> >
> > I've come up with adding the functions (not methods at this point)
> >
> > deletefrom
> > insertinto
>
>
> "delete" and "insert" really would be better.   The current "insert"
> function seems inaptly named.  What it does sounds more like "overlay" or
> "set_masked".

I prefer delete and insert too. I guess it is OK that del and delete
are similar (?)

Re: [Numpy-discussion] Deleting a row from a matrix

[Charles R H. <cha...@gm...>]

Hi,

On 8/26/06, Keith Goodman <kwg...@gm...> wrote:
>
> On 8/26/06, Bill Baxter <wb...@gm...> wrote:
> > On 8/26/06, Travis Oliphant <oli...@ie...> wrote:
> >
> > >
> > > I've come up with adding the functions (not methods at this point)
> > >
> > > deletefrom
> > > insertinto
> >
> >
> > "delete" and "insert" really would be better.   The current "insert"
> > function seems inaptly named.  What it does sounds more like "overlay"
> or
> > "set_masked".
>
> I prefer delete and insert too. I guess it is OK that del and delete
> are similar (?)


Me too, although remove could be used instead of delete. Is there a problem
besides compatibility with removing or changing the old insert?

Chuck