numpy-discussion

[Numpy-discussion] repmat

[Bill B. <wb...@gm...>]

[There seem to have been some gmail delivery problems that prevented
my previous mail on this subject from being delivered]

I've proposed that we fix repmat handle arbitrary dimensions before 1.0.

   http://projects.scipy.org/scipy/numpy/ticket/292

I don't think this is particularly controversial, just I'm guessing
no-one's found the time to look at my proposed fixes.  And
gmail/sourceforge issues haven't helped either.

In short, repmat(A, M,N) is an oddball.  It only deals with 2D arrays.

We should do some combination of:
1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability
to it, while maintaining backwards compatibility.
2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability
to it, and bring it into line with the "numpy standard" of shapes
being passed as one tuple arg.
3) add a new method like reparray(A, shape) that has the multidim
ability, and matches numpy conventions.
4)   if 3), move repmat out of the top-level namespace to indicate deprecation

The tracker item has all the code necessary to implement these fixes,
it's just a matter of deciding which one to go with.

My personal preference would be a combination of 1 and 3.  Add
reparray(A,shape) , and change repmat to repmat(A, *shape).  Then just
implement repmat as a call to reparray.

If someone will decide which route to go, I'd be happy to provide an
actual patch against the latest SVN to implement that decision.

If this is somehow controversial for some reason then let's discuss
it.  But so far the only response has been "copying data is a bad
idea", which is really a separate issue.

--bb

Re: [Numpy-discussion] repmat

[Tim H. <tim...@ie...>]

Bill Baxter wrote:
> [There seem to have been some gmail delivery problems that prevented
> my previous mail on this subject from being delivered]
>
> I've proposed that we fix repmat handle arbitrary dimensions before 1.0.
>
>    http://projects.scipy.org/scipy/numpy/ticket/292
>
> I don't think this is particularly controversial, just I'm guessing
> no-one's found the time to look at my proposed fixes.  And
> gmail/sourceforge issues haven't helped either.
>
> In short, repmat(A, M,N) is an oddball.  It only deals with 2D arrays.
>
> We should do some combination of:
> 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability
> to it, while maintaining backwards compatibility.
>   
Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible 
though.

> 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability
> to it, and bring it into line with the "numpy standard" of shapes
> being passed as one tuple arg.
>   
I presume shape actually means number of repetitions, and not shape. If 
so, shape is the wrong name for the second argument. Anyway as long as 
your breaking backward compatibility you might as well improve the name.

> 3) add a new method like reparray(A, shape) that has the multidim
> ability, and matches numpy conventions.
>   
Same comment on the name shape.


> 4)   if 3), move repmat out of the top-level namespace to indicate deprecation
>   
I'm not completely sold on the whole reparray thing: does repmat have 
any use outside of the linear algebra convention of squashing a 3D array 
into a 2D matrix (or however that goes)? If not, perhaps it should just 
get left alone and exciled to matrix specific functions namespace. So:

+1 on deprecating, moving or otherwise getting rid of repmat.
Neutral on whether to replace it with something else, but if it is 
replaced #3 looks like the correct route, but not with a different 
argument name than 'shape'. Perhaps 'reps'.

-tim

> The tracker item has all the code necessary to implement these fixes,
> it's just a matter of deciding which one to go with.
>
> My personal preference would be a combination of 1 and 3.  Add
> reparray(A,shape) , and change repmat to repmat(A, *shape).  Then just
> implement repmat as a call to reparray.
>
> If someone will decide which route to go, I'd be happy to provide an
> actual patch against the latest SVN to implement that decision.
>
> If this is somehow controversial for some reason then let's discuss
> it.  But so far the only response has been "copying data is a bad
> idea", which is really a separate issue.
>
> --bb
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   

Re: [Numpy-discussion] repmat

[Bill B. <wb...@gm...>]

On 10/6/06, Tim Hochberg <tim...@ie...> wrote:
> Bill Baxter wrote:
> > In short, repmat(A, M,N) is an oddball.  It only deals with 2D arrays.
> >
> > We should do some combination of:
> > 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability
> > to it, while maintaining backwards compatibility.
> >
> Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible
> though.

It does kind of imply that (which is why I think something like
'reparray' would be a better name) but even matlab's repmat() function
doesn't restrict you to 2D arrays, and I'm pretty sure numpy.repmat
was intended to be a clone of that.

> > 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability
> > to it, and bring it into line with the "numpy standard" of shapes
> > being passed as one tuple arg.
> >
> I presume shape actually means number of repetitions, and not shape. If
> so, shape is the wrong name for the second argument. Anyway as long as
> your breaking backward compatibility you might as well improve the name.

Yeh, that did occur to me.  Maybe 'reps' would be a better name for
the argument.

> > 3) add a new method like reparray(A, shape) that has the multidim
> > ability, and matches numpy conventions.
> >
> Same comment on the name shape.
>
>
> > 4)   if 3), move repmat out of the top-level namespace to indicate deprecation
> >
> I'm not completely sold on the whole reparray thing: does repmat have
> any use outside of the linear algebra convention of squashing a 3D array
> into a 2D matrix (or however that goes)? If not, perhaps it should just
> get left alone and exciled to matrix specific functions namespace. So:

Well, I'm not sure about N-d, but I've definitely had a need for 1-d
version of it.  That's what prompted me to look into this.  Also I
think I read a comment in the other thread about pyEM that repmat()
turned out to be one of the faster ways to implement
something-or-other (?).  At the very least you could think of tiling
3D image data as one application, in fact 'tile()' is probably a
better name for the function than 'reparray'.

I was initially suprised that 'repeat()' featured so prominently in
numpy.  I initially assumed it meant "repmat", and was surprised when
it didn't do that.  I thought "why on earth would you want to do
that?".  But it does turn out to have lots of uses (like implementing
"repmat" ;-))

> +1 on deprecating, moving or otherwise getting rid of repmat.
> Neutral on whether to replace it with something else, but if it is
> replaced #3 looks like the correct route, but not with a different
> argument name than 'shape'. Perhaps 'reps'.

Hey!  Looks like we agree on the name of the argument, apparently.  :-)
I do think there is a strong connection with "shape" though, that
'reps' hides. Maybe 'repshape' or 'tileshape'?  I'd be happy with
reps, though.  Agreed that 'shape' is bad.  Makes it sound like you
should specify the actual .shape of the output.

--bb

Re: [Numpy-discussion] repmat

[David C. <da...@ar...>]

Bill Baxter wrote:
>
>> I'm not completely sold on the whole reparray thing: does repmat have
>> any use outside of the linear algebra convention of squashing a 3D array
>> into a 2D matrix (or however that goes)? If not, perhaps it should just
>> get left alone and exciled to matrix specific functions namespace. So:
>>     
>
> Well, I'm not sure about N-d, but I've definitely had a need for 1-d
> version of it.  That's what prompted me to look into this.  Also I
> think I read a comment in the other thread about pyEM that repmat()
> turned out to be one of the faster ways to implement
> something-or-other (?). 
It was in a very specific case, and it could have been replaced by an 
other function in this case without any loss of performance. The 
following is my experience with matlab/repmat: generally, repmat was 
necessary in many cases in matlab either for efficiency (big scalar 
matrix were slow to generate using ones and zeros, for example) and 
because matlab had almost no broadcasting rules, repmat was almost 
mandatory for matlab. In numpy, the syntax at least does not require 
repmat (at least, I never *had* to use it in numpy, and I used it a lot 
with matlab).

Moreoever, the slowness of some broadcasting seem solved in recent 
subversion (see my answer on the specific thread),

David
>  At the very least you could think of tiling
> 3D image data as one application, in fact 'tile()' is probably a
> better name for the function than 'reparray'.
>
> I was initially suprised that 'repeat()' featured so prominently in
> numpy.  I initially assumed it meant "repmat", and was surprised when
> it didn't do that.  I thought "why on earth would you want to do
> that?".  But it does turn out to have lots of uses (like implementing
> "repmat" ;-))
>
>   
>> +1 on deprecating, moving or otherwise getting rid of repmat.
>> Neutral on whether to replace it with something else, but if it is
>> replaced #3 looks like the correct route, but not with a different
>> argument name than 'shape'. Perhaps 'reps'.
>>     
>
> Hey!  Looks like we agree on the name of the argument, apparently.  :-)
> I do think there is a strong connection with "shape" though, that
> 'reps' hides. Maybe 'repshape' or 'tileshape'?  I'd be happy with
> reps, though.  Agreed that 'shape' is bad.  Makes it sound like you
> should specify the actual .shape of the output.
>
> --bb
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   

Re: [Numpy-discussion] repmat

[Stefan v. d. W. <st...@su...>]

On Fri, Oct 06, 2006 at 10:30:43AM +0900, Bill Baxter wrote:
> If this is somehow controversial for some reason then let's discuss
> it.  But so far the only response has been "copying data is a bad
> idea", which is really a separate issue.

An interesting issue, though.  I've often wondered about an array
view where data is visible in more than one place.  For example, you
have an array

x =3D N.arange((100))

on which you'd like to apply a filter, which, say, finds the moving
variance of the 5 closest elements.  Here, a new representation of x
would be useful:

In [23]: indices =3D N.arange(5) + N.arange(96).reshape((96,1))

In [24]: indices
Out[24]:=20
array([[ 0,  1,  2,  3,  4],
       [ 1,  2,  3,  4,  5],
       ...
       [95, 96, 97, 98, 99]])


In [25]: y =3D x.index_view(indices) # returns array with same values as
   	                           # y =3D x[indices]
                                   # except that no values are copied

In [29]: y
Out[29]:=20
array([[ 0,  1,  2,  3,  4],
       [ 1,  2,  3,  4,  5],
       [ 2,  3,  4,  5,  6],
       ...

after which you would be able to do

In [30]: y.var(axis=3D1)
Out[30]:=20
array([ 2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,
        2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,

While there would be a lot of jumping around in indices, at the loops
can be done in C, which should speed up the process.  Of course, with
small arrays I can simply copy the data, but it becomes troublesome
with images and other such large signals.

I am not sure if providing the indices is the best way to initialise
such an array (otherwise maybe a rule for index calculation?) or how
feasible it would be to implement, so I'd be glad to hear anyone's
thoughts on the topic.

Regards
St=E9fan

Re: [Numpy-discussion] repmat

[Travis O. <oli...@ee...>]

Bill Baxter wrote:

>[There seem to have been some gmail delivery problems that prevented
>my previous mail on this subject from being delivered]
>
>I've proposed that we fix repmat handle arbitrary dimensions before 1.0.
>
>   http://projects.scipy.org/scipy/numpy/ticket/292
>
>I don't think this is particularly controversial, just I'm guessing
>no-one's found the time to look at my proposed fixes.  And
>gmail/sourceforge issues haven't helped either.
>  
>
Thanks for reminding us again. 

I don't think this is too bad of a deal.  I propose to move repmat(A, 
M,N) to matlib.py and replace it in numpy with a function named

tile(a, reps)

That is more general-purpose.   If this is not acceptable, please speak up.

-Travis