Menu
▾
▴
numpy-discussion — Discussion list for all users of Numerical Python
You can subscribe to this list here.
| 2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
| 2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
| 2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
| 2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
| 2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
| 2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
|
From: Tim H. <tim...@ie...> - 2003-09-26 17:13:11
|
Todd Miller wrote:
>I tried out DeferredToByNumArray yesterday morning and was able to get
>it to work as planned with numarray and MA. So yay!
>
>
Excellent.
>Yesterday afternoon, I stopped by Perry's office for a final check, and
>not surprisingly, there are some aspects of this solution that we've
>overlooked. So not yay!
>
>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>apart as our inheritance hierarchy becomes richer:
>
>
Not so excellent.
>1) Imagine not one, but two independent subclasses of NumArray, each of
>which want NumArray to defer to them. Now imagine them trying to
>inter-operate with each other. In this case, the meaning of the
>expression is determined by the order of the operands, so A+B and B+A
>will either return an A or a B depending on the expression order.
>
>
I think that this is YAGNI right now, but may well be an issue is
"proper" subclassing get's implemented. So we may as well think about it
a bit. I call it YAGNI now because these subclasses either use the
default __op__ methods, which presently have the same behaviour
regardless of the subclasses (they always return NumArrays) or they
define there own __op__ methods in which case they can worry about it ;>
If NumArray starts to become more friendly to subclassing, this will
probably change though.
>2) Imagine again two subclasses of numarray, but this time imagine B as
>a subclass of A. Here again, both might want to defer to numarray, and
>again, A+B and B+A return different types driven by the type order of
>the expression.
>
>
I'd make the same comment for this case as for the above case except
that all else being equal, subclasses should prevail over superclasses.
>I don't have a solution yet, but am hopeful that more free candy will
>fall from the sky... or YAGNI. Perry pointed out that similar problems
>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>should forget about this until it matters. If someone sees an easy
>fix, now would be better than later.
>
>
Here's some quick thoughts. In order for this to work out sensibly you
want the order between classes to be transistive and you probably also
always want to be able to insert a class into the order between any two
classes This is equivalent to deciding the op priority based on a
comparison based on some real number associated with each class. This
leads to something like:
class NumArrayFamily:
op_priority = 0.0
# This makes it compatible with the current proposal , but isn't
strictly necessary.
class DeferredToByNumarray(NumArrayFamily):
op_priority = 1000.0
class NumArrary(..., NumArrayFamily):
# ...
def __add__(self, object):
if isinstance(object, NumArrayFamily):
if ((object.op_priority > self.object.op_priority) or
(object.op_priority == self.object.op_priority) and
issubclass(object, self.__class__)):
return other.__radd__(self)
return ufunc.add(self, other)
class MySubclassOfNumarray(NumArray):
op_priority = 10.0 # deffered to by NumArray, but defers to
DeferredToByNumarray classes.
Of course the user still has to be careful to make sure the priority
order makes sense and you could have big problems when combining
packages with disprate ideas about priority levels, but I'm not sure how
you can get away from that.
-tim
>Todd
>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>
>
>>Todd Miller wrote:
>>
>>
>>
>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>>
>>>
>>>
>>>
>>>>Hi Todd,
>>>>
>>>>There are three ways to spell "defer to me" on the table (the precise
>>>>details of
>>>>each spelling are, of course, still open for debate):
>>>>
>>>> 1. numarray.defer_to(my_class)
>>>>
>>>> 2. class ArrayLike(numarray.DeferTo):
>>>> # ...
>>>>
>>>> 3. class ArrayLike:
>>>> _numarray_defer_to = True
>>>> # ...
>>>>
>>>>I'd prefer a non-registration solution since those are both
>>>>aesthetically displeasing and leave you open to the situation where a
>>>>class in module A gets registered by module B, but module C expects it
>>>>not to be registered and everything breaks. Not all that likely, I
>>>>admit, but let's avoid the registration version if we can.
>>>>
>>>>
>>>>
>>>>
>>>I was picturing this as module A registering it's own classes only.
>>>Nevertheless, inverting the problem and distributing the registration as
>>>you suggested is better.
>>>
>>>
>>>
>>>
>>The case you describe probably will describe the majority of actual use
>>cases, and in fact describes mine. I'm trying to think ahead a bit to
>>cases may encounter as start using NumArray more extensively. Let's hope
>>this solution still looks good in six months!
>>
>>
>>
>>>>The other two solutions are almost equivalent. The one case where 3 has
>>>>an edge over 2 is if I have an object (not a class), I could potentially
>>>>set a _numarray_defer_to on the object before passing it to numarray
>>>>without having to mess with the class
>>>>of the object at all. YAGNI, though.
>>>>
>>>>
>>>>
>>>>
>>>I was more concerned about the potential impact of lots of multiple
>>>inheritance, but that's probably just my own personal blend of FUD.
>>>
>>>
>>>
>>>
>>>
>>>>The advantage of 2 in my view is that it *does* force you to subclass.
>>>>With 3, there will be the temptation to poke into some other module and
>>>>set _numarray_defer_to on some poor unsuspecting class. This has the
>>>>same disadvantage as 1, that it could confuse some other poor
>>>>unsuspecting module. The correct way to do get a deferred class from a
>>>>third party module is to import and subclass. This works with either 2
>>>>or 3::
>>>>
>>>> import A
>>>>
>>>> class Klass2(a.Klass, numarray.DeferTo): #2
>>>> #...
>>>>
>>>> class Klass3(a.Klass): #3 the good way
>>>> _numarray_defer_to = True
>>>> # ...
>>>>
>>>> A.Klass._numarray_defer_to = True #3 the evil way.
>>>>
>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd
>>>>prefer that solution.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>Good enough for me. If no one else has any comments, then
>>>numarray.DeferTo is where I'll start implementing. Tomorrow.
>>>
>>>
>>>
>>>
>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in
>>class name. It was perfect for the registration function name, but I'm
>>not sure it's so clear whether the class or numarray is being deferred
>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but
>>seems sort of slangy. DeferredTo is better than DeferTo, but still not
>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as
>>numarray is included but is a disaster if you read it on your own.
>>Below I've put down all the ideas I could come up with
>>
>>class CustomArray(numarray.DeferTo)
>>class CustomArray(numarray.DefersTo)
>>class CustomArray(numarray.DeferredTo)
>>class CustomArray(numarray.DeferToMe)
>>class CustomArray(numarray.DeferredToByNumarray)
>>class CustomArray(DeferTo)
>>class CustomArray(DefersTo)
>>class CustomArray(DeferredTo)
>>class CustomArray(DeferToMe)
>>class CustomArray(DeferredToByNumarray)
>>
>>For me it's a toss up between DefferedTo, DeferToMe and
>>DeferredToByNumarray. The first is a little lacking in descriptive
>>power, the second is slangy and the third is wordy.
>>
>>-tim
>>
>>[not that this matters much....]
>>
>>
>>
>>
>>>>regards,
>>>>
>>>>-tim
>>>>
>>>>
>>>>
>>>>
>>>Thanks again,
>>>Todd
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Numpy-discussion mailing list
>>Num...@li...
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>>
|
|
From: Todd M. <jm...@st...> - 2003-09-26 13:40:34
|
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay! Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay! Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer: 1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order. 2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression. I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later. Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote: > Todd Miller wrote: > > >On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote: > > > > > >>Hi Todd, > >> > >>There are three ways to spell "defer to me" on the table (the precise > >>details of > >>each spelling are, of course, still open for debate): > >> > >> 1. numarray.defer_to(my_class) > >> > >> 2. class ArrayLike(numarray.DeferTo): > >> # ... > >> > >> 3. class ArrayLike: > >> _numarray_defer_to = True > >> # ... > >> > >>I'd prefer a non-registration solution since those are both > >>aesthetically displeasing and leave you open to the situation where a > >>class in module A gets registered by module B, but module C expects it > >>not to be registered and everything breaks. Not all that likely, I > >>admit, but let's avoid the registration version if we can. > >> > >> > > > >I was picturing this as module A registering it's own classes only. > >Nevertheless, inverting the problem and distributing the registration as > >you suggested is better. > > > > > > The case you describe probably will describe the majority of actual use > cases, and in fact describes mine. I'm trying to think ahead a bit to > cases may encounter as start using NumArray more extensively. Let's hope > this solution still looks good in six months! > > >>The other two solutions are almost equivalent. The one case where 3 has > >>an edge over 2 is if I have an object (not a class), I could potentially > >>set a _numarray_defer_to on the object before passing it to numarray > >>without having to mess with the class > >>of the object at all. YAGNI, though. > >> > >> > > > >I was more concerned about the potential impact of lots of multiple > >inheritance, but that's probably just my own personal blend of FUD. > > > > > > > >>The advantage of 2 in my view is that it *does* force you to subclass. > >>With 3, there will be the temptation to poke into some other module and > >>set _numarray_defer_to on some poor unsuspecting class. This has the > >>same disadvantage as 1, that it could confuse some other poor > >>unsuspecting module. The correct way to do get a deferred class from a > >>third party module is to import and subclass. This works with either 2 > >>or 3:: > >> > >> import A > >> > >> class Klass2(a.Klass, numarray.DeferTo): #2 > >> #... > >> > >> class Klass3(a.Klass): #3 the good way > >> _numarray_defer_to = True > >> # ... > >> > >> A.Klass._numarray_defer_to = True #3 the evil way. > >> > >> Version 2 is cleaner and encourages you to do the right thing, so I'd > >>prefer that solution. > >> > >> > >> > > > >Good enough for me. If no one else has any comments, then > >numarray.DeferTo is where I'll start implementing. Tomorrow. > > > > > > One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in > class name. It was perfect for the registration function name, but I'm > not sure it's so clear whether the class or numarray is being deferred > to when you say numarray.DeferTo. DeferToMe is more descriptive, but > seems sort of slangy. DeferredTo is better than DeferTo, but still not > as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as > numarray is included but is a disaster if you read it on your own. > Below I've put down all the ideas I could come up with > > class CustomArray(numarray.DeferTo) > class CustomArray(numarray.DefersTo) > class CustomArray(numarray.DeferredTo) > class CustomArray(numarray.DeferToMe) > class CustomArray(numarray.DeferredToByNumarray) > class CustomArray(DeferTo) > class CustomArray(DefersTo) > class CustomArray(DeferredTo) > class CustomArray(DeferToMe) > class CustomArray(DeferredToByNumarray) > > For me it's a toss up between DefferedTo, DeferToMe and > DeferredToByNumarray. The first is a little lacking in descriptive > power, the second is slangy and the third is wordy. > > -tim > > [not that this matters much....] > > > >>regards, > >> > >>-tim > >> > >> > > > >Thanks again, > >Todd > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: Todd M. <jm...@st...> - 2003-09-25 20:19:52
|
On Thu, 2003-09-25 at 12:14, Colin J. Williams wrote: > Todd, > > Thanks for the benchmark data. Thanks for the references. They suggest that the classmethod approach we've already tried for folding in the numarray factory functions is reasonable OOP. > I intend to post the matrix stuff. Performance is not a problem for me > at this stage. > > Currently, I'm plodding along, making things work. Later, I'll post it > with some rudimentary documentation. > > Regarding the "new-classes standard", I can't find much in either the > Python docs or Alex Martelli's "Python in a Nutshell". One of the new > features is a __new__ constructor. Well, we've certainly got those (__new__ methods). They don't match the __init__ signature though, nor was I aware that they should. > The Python array module doesn't formally have a class and thus it can't > be sub-classed. It uses a factory function to create instance objects. > > Martelli, in PIAN p74, writes of factory functions being useful when one > wishes to create instances of different classes depending on some > condition or for the reuse of an existing instance. > > In the Python Cookbook p173, he proposes the use of a factory function > to get around the limitations of Python 2.1 and earlier. On page 179, > he refers to the use of a factory method, again to work around > limitations before Python 2.2. > > My rough and ready definition would include the ability to create an > instance and initialize > the data in one step. > One thing to keep in mind is that NumArray was designed to run on Python-2.0. As Python-2.2 became widely available and stable, we realized we could exploit it for better C-API compatibility with Numeric, and for better speed. I think what you and later Nadav have pointed out is that we can exploit Python-2.2 for better sub-classing as well. We just haven't done it yet. Concrete proposals are welcome. Regards, Todd |
|
From: Colin J. W. <cj...@sy...> - 2003-09-25 19:08:06
|
C:\Program Files\Python23\Lib\site-packages\numarray>python Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import testall The 21K file created by Windows to contain information about the crash is not readable. Windows XP and numarray 0.7 used. Colin W. |
|
From: Todd M. <jm...@st...> - 2003-09-25 19:05:03
|
On Thu, 2003-09-25 at 12:28, Paul Dubois wrote: > I have not followed this thread in any detail but I continue to be > puzzled by the idea of subclassing numarray arrays. Any subclass worth > its salt has an unpredictable constructor argument list, so that > virtually every ufunc and every array method have to be overridden in > order to return the subclass object rather than a numarray. I think there is a need to effectively subclass numarray where the parameter lists don't change. Doing a Matrix subclass sounds reasonable to me and we should explore the OOP necessary to make it convenient. STSCI hasn't asked for it, but a couple different numarray contributors have. > I gather > that the defers-to-me stuff allows the subclass to get control of a > mixed operation but that was possible with the radd, etc. operators > already as in the old MA. radd does not work for numarray + MA. Instead, the expression is resolved as numarray.__add__(MA). Were MA a numarray subclass, it would work, given the facilities provided by Python alone. The defers-to-me-stuff is a simple work around to avoid having to reverse engineer this one Numeric behavior. > This reasoning led me to believe that subclassing had virtually no value > and to the design of MA that has-a numeric array rather than is-a. Since we want to port MA as-is to numarray, we're not changing that. > Is there some big breakthrough here I missed? No, but there are two issues: (1) effectively supporting subclasses (2) effectively supporting interoperability with other classes. (1) needs more discussion. (2) is already solved. -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: Colin J. W. <cj...@sy...> - 2003-09-25 16:14:34
|
Todd, Thanks for the benchmark data. I intend to post the matrix stuff. Performance is not a problem for me at this stage. Currently, I'm plodding along, making things work. Later, I'll post it with some rudimentary documentation. Regarding the "new-classes standard", I can't find much in either the Python docs or Alex Martelli's "Python in a Nutshell". One of the new features is a __new__ constructor. The Python array module doesn't formally have a class and thus it can't be sub-classed. It uses a factory function to create instance objects. Martelli, in PIAN p74, writes of factory functions being useful when one wishes to create instances of different classes depending on some condition or for the reuse of an existing instance. In the Python Cookbook p173, he proposes the use of a factory function to get around the limitations of Python 2.1 and earlier. On page 179, he refers to the use of a factory method, again to work around limitations before Python 2.2. My rough and ready definition would include the ability to create an instance and initialize the data in one step. Perhaps it would be good to post your exchange with Nadev to the newsgroup comp.lang.python. There are many contributors to the group who could give a much more complete answer. Colin W. Todd Miller wrote: >On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote: > > >>Nadav Horesh wrote: >>[snip] >> >> >> >>>As I recall, one of the targets of the numarray project is to be >>>included as a standard python library --- shouldn't it eventually comply >>>with the new-classes standard? >>> >>> > >Can someone please explain what the "new-classes standard" is? >(Obviously meeting it is not one of our current goals) > > > >>I hope that more consideration will be given to this as the numarray >>package evolves. >> >> > >Of course. We don't need sub-classing internally, however, so it's >difficult to predict when we'll do better. > > > >> From time to time, I am developing a matrix sub-class, it's a bit more >>convoluted than I would like it to be. >> >> > >Posting it here would be good. Sending things directly to me tends to >dumb things down... to me. > > > >>It would also be helpful if a current assessment of the speed of >>numarray could be published. >> >> > >You asked... and... (sorry if this is less than succinct) > >The numarray source distribution has a module called bench.py in the >Examples directory. Without any analysis, I've attached the output >from bench.py on my 1.7 GHz Pentium IV running under i386 linux. Each >row shows the timing numbers for numarray and Numeric for a particular >function call and parameter set; the important part is the ratio ( < 1 >means numarray is faster. In general, numarray is slower; this is due >to the fact that numarray supports in-place operation on memory mapped >non-native data formats). > >Suspect numbers are... suspect. Please investigate! > >Caveats: > >1) The benchmarks were run on a multi-tasking kernel with no specific >load. The benchmarks also vary from platform to platform, processor to >processor, compiler to compiler, etc. These are gcc version 3.2 >20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM. > >2) The absolute measured units (microseconds) haven't really been >validated and shouldn't be taken too seriously. The relative time >comparisons should be valid. > >3) a,c,d are 1D arrays. b is a 2D array. > >4) the 10**x column shows the logarithmic array size. > 2 --> 100 elements, 6 --> 1 million elements > >5) The version of bench.py which made these measurements is in CVS, not >the numarray-0.7 tar-ball. > >Wish-list: > >1) The right way to eliminate the effects of host load on the >benchmarks. > >2) A way to create a regression test which includes performance. > > > >>I gather that some enhancements arrived with version 0.7. >> >> > >I don't recall any performance enhancements. numarray-0.7 was mostly >about object array support and bug fixes. > >bench.py results from numarray-0.7 09-25-03: > >benchmark 10**x numarray (usec) Numeric (usec) numarray:Numeric >take(a,c) 2 1477 25 56.8 > 3 1592 108 14.7 > 4 2920 989 2.9 > 5 15568 9814 1.6 > 6 136960 96253 1.4 >put(a,c,d) 2 1673 64 26.1 > 3 1798 195 9.2 > 4 3270 1587 2.1 > 5 16742 15334 1.1 > 6 147768 151932 1.0 >transpose(b) 2 154 22 6.8 > 3 157 24 6.3 > 4 190 39 4.8 > 5 262 74 3.5 > 6 257 67 3.8 >putmask(a,c%2,d) 2 1415 139 10.1 > 3 1504 271 5.6 > 4 2439 1735 1.4 > 5 11036 17337 0.6 > 6 93927 157848 0.6 >repeat(a,2) 2 1533 138 11.1 > 3 10165 579 17.5 > 4 103064 4701 21.9 > 5 1028850 58812 17.5 > 6 10010859 493157 20.3 >choose(a%2,(a,c)) 2 1312 73 18.0 > 3 1389 190 7.3 > 4 2479 1500 1.7 > 5 12516 16219 0.8 > 6 108841 142280 0.8 >ravel(b) 2 164 38 4.3 > 3 169 41 4.1 > 4 196 58 3.4 > 5 279 93 3.0 > 6 275 90 3.0 >nonzero(a) 2 1044 143 7.3 > 3 1133 285 4.0 > 4 2108 1929 1.1 > 5 12002 17974 0.7 > 6 111598 176598 0.6 >where(a%2,a,c) 2 1273 111 11.4 > 3 1380 242 5.7 > 4 2587 1752 1.5 > 5 13368 16785 0.8 > 6 117731 164187 0.7 >compress(a%2,a) 2 2489 198 12.6 > 3 2661 375 7.1 > 4 4575 2426 1.9 > 5 22933 22621 1.0 > 6 200138 219215 0.9 >diagonal(b) 2 2420 16883 0.1 > 3 2425 16896 0.1 > 4 2591 16932 0.2 > 5 2588 16875 0.2 > 6 2579 16861 0.2 >trace(b) 2 2611 16988 0.2 > 3 2625 16990 0.2 > 4 2750 17004 0.2 > 5 2758 17048 0.2 > 6 2766 16942 0.2 >searchsorted(a,a) 2 506 55 9.2 > 3 708 490 1.4 > 4 3054 6507 0.5 > 5 32101 77762 0.4 > 6 378341 940912 0.4 >sort(a) 2 554 85 6.4 > 3 940 365 2.6 > 4 5394 3533 1.5 > 5 55238 36049 1.5 > 6 607845 420334 1.4 >argsort(a) 2 1541 90 16.9 > 3 2061 467 4.4 > 4 8113 5130 1.6 > 5 76924 49497 1.6 > 6 853114 588399 1.4 >argmax(b) 2 1112081 8692 127.9 > 3 1110896 9320 119.2 > 4 1108330 8706 127.3 > 5 1113449 8720 127.7 > 6 1112556 8743 127.3 >array(s,typecode='i') 2 229 146 1.6 > 3 274 1103 0.2 > 4 764 10900 0.1 > 5 5450 105774 0.1 > 6 51500 1253819 0.0 >dot(b,b) 2 8825828 31402493 0.3 > 3 8804230 32136602 0.3 > 4 8748239 31123564 0.3 > 5 8811900 32910953 0.3 > 6 8827314 31784649 0.3 >clip(a,50,100) 2 2920 177 16.4 > 3 3089 370 8.3 > 4 5138 2471 2.1 > 5 24340 24737 1.0 > 6 211277 245540 0.9 >indices((100,100)) 2 5253 2696 1.9 > 3 5230 2634 2.0 > 4 5290 2687 2.0 > 5 5383 2732 2.0 > 6 5558 2733 2.0 >swapaxes(b,0,-1) 2 77 79 1.0 > 3 77 82 1.0 > 4 94 106 0.9 > 5 154 142 1.1 > 6 146 144 1.0 >concatenate((a,a,a)) 2 1021 35 29.1 > 3 1082 48 22.1 > 4 1868 431 4.3 > 5 8843 4945 1.8 > 6 75181 47495 1.6 >innerproduct(b,b) 2 8719124 8989714 1.0 > 3 8662431 8840564 1.0 > 4 8734306 9033604 1.0 > 5 8694994 8935343 1.0 > 6 8680599 8869750 1.0 >outerproduct(b[0],b[0]) 2 42320 27598 1.5 > 3 42299 27802 1.5 > 4 42842 27935 1.5 > 5 42346 27822 1.5 > 6 42257 27709 1.5 >resize(a,(2*10**i,)) 2 1708 160 10.6 > 3 1769 171 10.3 > 4 2648 468 5.6 > 5 9778 3515 2.8 > 6 75513 31778 2.4 >a[0] 2 10 10 1.0 > 3 10 10 1.0 > 4 13 12 1.1 > 5 25 20 1.3 > 6 25 22 1.1 >b[0] 2 46 15 3.1 > 3 48 17 2.8 > 4 56 18 3.0 > 5 106 33 3.1 > 6 100 35 2.9 >a[1000:2000] 2 303 16 18.0 > 3 298 15 18.7 > 4 275 20 13.8 > 5 354 33 10.4 > 6 344 34 9.8 >b[400:600,400:600] 2 320 25 12.8 > 3 317 25 12.2 > 4 348 30 11.3 > 5 432 46 9.2 > 6 418 47 8.7 >arange(10.**i) 2 411 37 11.1 > 3 448 74 6.0 > 4 923 570 1.6 > 5 5630 5592 1.0 > 6 50735 54222 0.9 >identity(i+2) 2 3147 226 13.9 > 3 3158 227 13.9 > 4 3214 258 12.4 > 5 3311 294 11.3 > 6 3316 291 11.4 >add(a,c) 2 76 27 2.7 > 3 90 40 2.2 > 4 313 225 1.4 > 5 2902 2128 1.4 > 6 28106 19972 1.4 >add(a,c,d) 2 40 22 1.9 > 3 46 33 1.4 > 4 144 132 1.1 > 5 1128 1165 1.0 > 6 10316 10827 1.0 >a+c 2 91 31 2.9 > 3 105 45 2.4 > 4 335 231 1.5 > 5 2945 2130 1.4 > 6 28102 19744 1.4 >cos(a) 2 84 53 1.6 > 3 212 235 0.9 > 4 1546 2351 0.7 > 5 15237 23074 0.7 > 6 151604 227898 0.7 >add.reduce(a) 2 97 31 3.0 > 3 109 41 2.7 > 4 207 123 1.7 > 5 1062 949 1.1 > 6 8974 8844 1.0 >add.reduce(b,0) 2 32527 9218 3.5 > 3 31652 9238 3.4 > 4 31376 9171 3.4 > 5 31412 9271 3.4 > 6 30443 9263 3.3 >add.reduce(b,-1) 2 9065 9389 1.0 > 3 9075 9360 1.0 > 4 9102 9271 1.0 > 5 9168 9389 1.0 > 6 9148 9394 1.0 >add.accumulate(a) 2 100 28 3.5 > 3 120 41 2.9 > 4 416 254 1.6 > 5 3584 2696 1.3 > 6 34837 25608 1.4 >add.accumulate(b,0) 2 63947 25673 2.5 > 3 65875 25691 2.6 > 4 65540 25708 2.5 > 5 65161 25776 2.5 > 6 66539 25646 2.6 >add.accumulate(b,1) 2 35048 25756 1.4 > 3 35022 25738 1.4 > 4 35048 25688 1.4 > 5 35092 25761 1.4 > 6 35110 25686 1.4 > > > > >------------------------------------------------------- >This sf.net email is sponsored by:ThinkGeek >Welcome to geek heaven. >http://thinkgeek.com/sf >_______________________________________________ >Numpy-discussion mailing list >Num...@li... >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
|
From: Todd M. <jm...@st...> - 2003-09-25 14:25:40
|
On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
> Nadav Horesh wrote:
> [snip]
>
> >As I recall, one of the targets of the numarray project is to be
> >included as a standard python library --- shouldn't it eventually comply
> >with the new-classes standard?
Can someone please explain what the "new-classes standard" is?
(Obviously meeting it is not one of our current goals)
> I hope that more consideration will be given to this as the numarray
> package evolves.
Of course. We don't need sub-classing internally, however, so it's
difficult to predict when we'll do better.
> From time to time, I am developing a matrix sub-class, it's a bit more
> convoluted than I would like it to be.
Posting it here would be good. Sending things directly to me tends to
dumb things down... to me.
> It would also be helpful if a current assessment of the speed of
> numarray could be published.
You asked... and... (sorry if this is less than succinct)
The numarray source distribution has a module called bench.py in the
Examples directory. Without any analysis, I've attached the output
from bench.py on my 1.7 GHz Pentium IV running under i386 linux. Each
row shows the timing numbers for numarray and Numeric for a particular
function call and parameter set; the important part is the ratio ( < 1
means numarray is faster. In general, numarray is slower; this is due
to the fact that numarray supports in-place operation on memory mapped
non-native data formats).
Suspect numbers are... suspect. Please investigate!
Caveats:
1) The benchmarks were run on a multi-tasking kernel with no specific
load. The benchmarks also vary from platform to platform, processor to
processor, compiler to compiler, etc. These are gcc version 3.2
20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.
2) The absolute measured units (microseconds) haven't really been
validated and shouldn't be taken too seriously. The relative time
comparisons should be valid.
3) a,c,d are 1D arrays. b is a 2D array.
4) the 10**x column shows the logarithmic array size.
2 --> 100 elements, 6 --> 1 million elements
5) The version of bench.py which made these measurements is in CVS, not
the numarray-0.7 tar-ball.
Wish-list:
1) The right way to eliminate the effects of host load on the
benchmarks.
2) A way to create a regression test which includes performance.
> I gather that some enhancements arrived with version 0.7.
I don't recall any performance enhancements. numarray-0.7 was mostly
about object array support and bug fixes.
bench.py results from numarray-0.7 09-25-03:
benchmark 10**x numarray (usec) Numeric (usec) numarray:Numeric
take(a,c) 2 1477 25 56.8
3 1592 108 14.7
4 2920 989 2.9
5 15568 9814 1.6
6 136960 96253 1.4
put(a,c,d) 2 1673 64 26.1
3 1798 195 9.2
4 3270 1587 2.1
5 16742 15334 1.1
6 147768 151932 1.0
transpose(b) 2 154 22 6.8
3 157 24 6.3
4 190 39 4.8
5 262 74 3.5
6 257 67 3.8
putmask(a,c%2,d) 2 1415 139 10.1
3 1504 271 5.6
4 2439 1735 1.4
5 11036 17337 0.6
6 93927 157848 0.6
repeat(a,2) 2 1533 138 11.1
3 10165 579 17.5
4 103064 4701 21.9
5 1028850 58812 17.5
6 10010859 493157 20.3
choose(a%2,(a,c)) 2 1312 73 18.0
3 1389 190 7.3
4 2479 1500 1.7
5 12516 16219 0.8
6 108841 142280 0.8
ravel(b) 2 164 38 4.3
3 169 41 4.1
4 196 58 3.4
5 279 93 3.0
6 275 90 3.0
nonzero(a) 2 1044 143 7.3
3 1133 285 4.0
4 2108 1929 1.1
5 12002 17974 0.7
6 111598 176598 0.6
where(a%2,a,c) 2 1273 111 11.4
3 1380 242 5.7
4 2587 1752 1.5
5 13368 16785 0.8
6 117731 164187 0.7
compress(a%2,a) 2 2489 198 12.6
3 2661 375 7.1
4 4575 2426 1.9
5 22933 22621 1.0
6 200138 219215 0.9
diagonal(b) 2 2420 16883 0.1
3 2425 16896 0.1
4 2591 16932 0.2
5 2588 16875 0.2
6 2579 16861 0.2
trace(b) 2 2611 16988 0.2
3 2625 16990 0.2
4 2750 17004 0.2
5 2758 17048 0.2
6 2766 16942 0.2
searchsorted(a,a) 2 506 55 9.2
3 708 490 1.4
4 3054 6507 0.5
5 32101 77762 0.4
6 378341 940912 0.4
sort(a) 2 554 85 6.4
3 940 365 2.6
4 5394 3533 1.5
5 55238 36049 1.5
6 607845 420334 1.4
argsort(a) 2 1541 90 16.9
3 2061 467 4.4
4 8113 5130 1.6
5 76924 49497 1.6
6 853114 588399 1.4
argmax(b) 2 1112081 8692 127.9
3 1110896 9320 119.2
4 1108330 8706 127.3
5 1113449 8720 127.7
6 1112556 8743 127.3
array(s,typecode='i') 2 229 146 1.6
3 274 1103 0.2
4 764 10900 0.1
5 5450 105774 0.1
6 51500 1253819 0.0
dot(b,b) 2 8825828 31402493 0.3
3 8804230 32136602 0.3
4 8748239 31123564 0.3
5 8811900 32910953 0.3
6 8827314 31784649 0.3
clip(a,50,100) 2 2920 177 16.4
3 3089 370 8.3
4 5138 2471 2.1
5 24340 24737 1.0
6 211277 245540 0.9
indices((100,100)) 2 5253 2696 1.9
3 5230 2634 2.0
4 5290 2687 2.0
5 5383 2732 2.0
6 5558 2733 2.0
swapaxes(b,0,-1) 2 77 79 1.0
3 77 82 1.0
4 94 106 0.9
5 154 142 1.1
6 146 144 1.0
concatenate((a,a,a)) 2 1021 35 29.1
3 1082 48 22.1
4 1868 431 4.3
5 8843 4945 1.8
6 75181 47495 1.6
innerproduct(b,b) 2 8719124 8989714 1.0
3 8662431 8840564 1.0
4 8734306 9033604 1.0
5 8694994 8935343 1.0
6 8680599 8869750 1.0
outerproduct(b[0],b[0]) 2 42320 27598 1.5
3 42299 27802 1.5
4 42842 27935 1.5
5 42346 27822 1.5
6 42257 27709 1.5
resize(a,(2*10**i,)) 2 1708 160 10.6
3 1769 171 10.3
4 2648 468 5.6
5 9778 3515 2.8
6 75513 31778 2.4
a[0] 2 10 10 1.0
3 10 10 1.0
4 13 12 1.1
5 25 20 1.3
6 25 22 1.1
b[0] 2 46 15 3.1
3 48 17 2.8
4 56 18 3.0
5 106 33 3.1
6 100 35 2.9
a[1000:2000] 2 303 16 18.0
3 298 15 18.7
4 275 20 13.8
5 354 33 10.4
6 344 34 9.8
b[400:600,400:600] 2 320 25 12.8
3 317 25 12.2
4 348 30 11.3
5 432 46 9.2
6 418 47 8.7
arange(10.**i) 2 411 37 11.1
3 448 74 6.0
4 923 570 1.6
5 5630 5592 1.0
6 50735 54222 0.9
identity(i+2) 2 3147 226 13.9
3 3158 227 13.9
4 3214 258 12.4
5 3311 294 11.3
6 3316 291 11.4
add(a,c) 2 76 27 2.7
3 90 40 2.2
4 313 225 1.4
5 2902 2128 1.4
6 28106 19972 1.4
add(a,c,d) 2 40 22 1.9
3 46 33 1.4
4 144 132 1.1
5 1128 1165 1.0
6 10316 10827 1.0
a+c 2 91 31 2.9
3 105 45 2.4
4 335 231 1.5
5 2945 2130 1.4
6 28102 19744 1.4
cos(a) 2 84 53 1.6
3 212 235 0.9
4 1546 2351 0.7
5 15237 23074 0.7
6 151604 227898 0.7
add.reduce(a) 2 97 31 3.0
3 109 41 2.7
4 207 123 1.7
5 1062 949 1.1
6 8974 8844 1.0
add.reduce(b,0) 2 32527 9218 3.5
3 31652 9238 3.4
4 31376 9171 3.4
5 31412 9271 3.4
6 30443 9263 3.3
add.reduce(b,-1) 2 9065 9389 1.0
3 9075 9360 1.0
4 9102 9271 1.0
5 9168 9389 1.0
6 9148 9394 1.0
add.accumulate(a) 2 100 28 3.5
3 120 41 2.9
4 416 254 1.6
5 3584 2696 1.3
6 34837 25608 1.4
add.accumulate(b,0) 2 63947 25673 2.5
3 65875 25691 2.6
4 65540 25708 2.5
5 65161 25776 2.5
6 66539 25646 2.6
add.accumulate(b,1) 2 35048 25756 1.4
3 35022 25738 1.4
4 35048 25688 1.4
5 35092 25761 1.4
6 35110 25686 1.4
|
|
From: Todd M. <jm...@st...> - 2003-09-25 14:00:29
|
On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote: > One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in > class name. It was perfect for the registration function name, but I'm > not sure it's so clear whether the class or numarray is being deferred > to when you say numarray.DeferTo. DeferToMe is more descriptive, but > seems sort of slangy. DeferredTo is better than DeferTo, but still not > as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as > numarray is included but is a disaster if you read it on your own. > Below I've put down all the ideas I could come up with > > class CustomArray(numarray.DeferTo) > class CustomArray(numarray.DefersTo) > class CustomArray(numarray.DeferredTo) > class CustomArray(numarray.DeferToMe) > class CustomArray(numarray.DeferredToByNumarray) > class CustomArray(DeferTo) > class CustomArray(DefersTo) > class CustomArray(DeferredTo) > class CustomArray(DeferToMe) > class CustomArray(DeferredToByNumarray) > > For me it's a toss up between DefferedTo, DeferToMe and > DeferredToByNumarray. The first is a little lacking in descriptive > power, the second is slangy and the third is wordy. For something as esoteric as this, I think wordy is probably best. Barring a Perrenial edict, DeferredToByNumArray it is. I'm psyched. The MA port should inter-operate with numarray... today. Thanks for all the lucid, constructive suggestions, Todd -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: Colin J. W. <cj...@sy...> - 2003-09-25 12:15:37
|
Nadav Horesh wrote: [snip] >As I recall, one of the targets of the numarray project is to be >included as a standard python library --- shouldn't it eventually comply >with the new-classes standard? > > I hope that more consideration will be given to this as the numarray package evolves. From time to time, I am developing a matrix sub-class, it's a bit more convoluted than I would like it to be. It would also be helpful if a current assessment of the speed of numarray could be published. I gather that some enhancements arrived with version 0.7. Colin W. |
|
From: Nadav H. <na...@Vi...> - 2003-09-25 07:25:34
|
On Wed, 2003-09-24 at 14:31, Colin J. Williams wrote: > Nadav Horesh wrote: > > >Recently I had an idea to subclass numarray's array in order to > >implement a different scheme of array indexing via the __call__ method > >(ugly indeed, by my experience shows that it can be useful). It occurred > >to me that the array class is not in the spirit of Python 2.2+ > >classes/types unification: > > > I agree. > > > * The class name is not "trivial": numarraycore.NumArray > > * The class constructor is not built to be used by the commons. > > > I'm not sure what you have in mind here by "trivial" and "commons". commons == End users whose most scripts life-time is few days. trivial == A name one would expect, like "Array" or similar. > > >Since arrays are normally generated by functions (zeros, array, > >fromfunction etc.) and not by the constructor, subclassing > >numarraycore.NumArray is not very useful. If, for instance, "array" (or > >"Array") would be the class name, where the constructor would optionally > >have the same parameters as the array function (with the option to act > >as a copy constructor as the array function does), then it would be easy > >to subclass it: > > > >class MyArray(array): > > ... > > > > > >new_array = MyArray(zeros((10,5), type=Int64)) > > > > > > > This is desirable, with the present design, one has to go through a two > step process: > > new_array = MyArray(type=Int64, shape= (10, 5)) > load_some_data_into_array(new_array, zeros(10, 5)) > > Colin W. As I recall, one of the targets of the numarray project is to be included as a standard python library --- shouldn't it eventually comply with the new-classes standard? > > > Any comments? > > > > Nadav. > > > > > > > >------------------------------------------------------- > >This sf.net email is sponsored by:ThinkGeek > >Welcome to geek heaven. > >http://thinkgeek.com/sf > >_______________________________________________ > >Numpy-discussion mailing list > >Num...@li... > >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > |
|
From: Tim H. <tim...@ie...> - 2003-09-24 23:56:29
|
Todd Miller wrote: >On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote: > > >>Hi Todd, >> >>There are three ways to spell "defer to me" on the table (the precise >>details of >>each spelling are, of course, still open for debate): >> >> 1. numarray.defer_to(my_class) >> >> 2. class ArrayLike(numarray.DeferTo): >> # ... >> >> 3. class ArrayLike: >> _numarray_defer_to = True >> # ... >> >>I'd prefer a non-registration solution since those are both >>aesthetically displeasing and leave you open to the situation where a >>class in module A gets registered by module B, but module C expects it >>not to be registered and everything breaks. Not all that likely, I >>admit, but let's avoid the registration version if we can. >> >> > >I was picturing this as module A registering it's own classes only. >Nevertheless, inverting the problem and distributing the registration as >you suggested is better. > > The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months! >>The other two solutions are almost equivalent. The one case where 3 has >>an edge over 2 is if I have an object (not a class), I could potentially >>set a _numarray_defer_to on the object before passing it to numarray >>without having to mess with the class >>of the object at all. YAGNI, though. >> >> > >I was more concerned about the potential impact of lots of multiple >inheritance, but that's probably just my own personal blend of FUD. > > > >>The advantage of 2 in my view is that it *does* force you to subclass. >>With 3, there will be the temptation to poke into some other module and >>set _numarray_defer_to on some poor unsuspecting class. This has the >>same disadvantage as 1, that it could confuse some other poor >>unsuspecting module. The correct way to do get a deferred class from a >>third party module is to import and subclass. This works with either 2 >>or 3:: >> >> import A >> >> class Klass2(a.Klass, numarray.DeferTo): #2 >> #... >> >> class Klass3(a.Klass): #3 the good way >> _numarray_defer_to = True >> # ... >> >> A.Klass._numarray_defer_to = True #3 the evil way. >> >> Version 2 is cleaner and encourages you to do the right thing, so I'd >>prefer that solution. >> >> >> > >Good enough for me. If no one else has any comments, then >numarray.DeferTo is where I'll start implementing. Tomorrow. > > One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray) For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy. -tim [not that this matters much....] >>regards, >> >>-tim >> >> > >Thanks again, >Todd > > > > > > > |
|
From: Todd M. <jm...@st...> - 2003-09-24 23:00:21
|
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote: > Hi Todd, > > There are three ways to spell "defer to me" on the table (the precise > details of > each spelling are, of course, still open for debate): > > 1. numarray.defer_to(my_class) > > 2. class ArrayLike(numarray.DeferTo): > # ... > > 3. class ArrayLike: > _numarray_defer_to = True > # ... > > I'd prefer a non-registration solution since those are both > aesthetically displeasing and leave you open to the situation where a > class in module A gets registered by module B, but module C expects it > not to be registered and everything breaks. Not all that likely, I > admit, but let's avoid the registration version if we can. I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better. > > The other two solutions are almost equivalent. The one case where 3 has > an edge over 2 is if I have an object (not a class), I could potentially > set a _numarray_defer_to on the object before passing it to numarray > without having to mess with the class > of the object at all. YAGNI, though. I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD. > The advantage of 2 in my view is that it *does* force you to subclass. > With 3, there will be the temptation to poke into some other module and > set _numarray_defer_to on some poor unsuspecting class. This has the > same disadvantage as 1, that it could confuse some other poor > unsuspecting module. The correct way to do get a deferred class from a > third party module is to import and subclass. This works with either 2 > or 3:: > > import A > > class Klass2(a.Klass, numarray.DeferTo): #2 > #... > > class Klass3(a.Klass): #3 the good way > _numarray_defer_to = True > # ... > > A.Klass._numarray_defer_to = True #3 the evil way. > > Version 2 is cleaner and encourages you to do the right thing, so I'd > prefer that solution. > Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow. > > regards, > > -tim Thanks again, Todd |
|
From: Tim H. <tim...@ie...> - 2003-09-24 22:27:59
|
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise
details of
each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo):
# ...
3. class ArrayLike:
_numarray_defer_to = True
# ...
I'd prefer a non-registration solution since those are both
aesthetically displeasing and leave you open to the situation where a
class in module A gets registered by module B, but module C expects it
not to be registered and everything breaks. Not all that likely, I
admit, but let's avoid the registration version if we can.
The other two solutions are almost equivalent. The one case where 3 has
an edge over 2 is if I have an object (not a class), I could potentially
set a _numarray_defer_to on the object before passing it to numarray
without having to mess with the class
of the object at all. YAGNI, though.
The advantage of 2 in my view is that it *does* force you to subclass.
With 3, there will be the temptation to poke into some other module and
set _numarray_defer_to on some poor unsuspecting class. This has the
same disadvantage as 1, that it could confuse some other poor
unsuspecting module. The correct way to do get a deferred class from a
third party module is to import and subclass. This works with either 2
or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2
#...
class Klass3(a.Klass): #3 the good way
_numarray_defer_to = True
# ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd
prefer that solution.
regards,
-tim
Todd Miller wrote:
>On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:
>
>
>
>>I actually have no idea how you plan to make keyword arguments work
>>here, perhaps you could explain that in more detail. Metaclasses are
>>overkill, but a mixin, marker class could be used. That is, when
>>designing a class for use with numarray, one would derive a class from a
>>marker class in numarray::
>>
>> class MyArrayLikeClass(numarray.DeferToMe):
>> ....
>>
>>Hmmm. That's not too bad. Todd, what do you think about using this logic::
>>
>> def __mul__(self, operand):
>> if isinstance(operand, DeferToMe):
>> operand.__rmul__(self)
>> else:
>> self.__mul__(operand)
>>
>>
>>
>>
>
>I like the core idea a lot. My only doubt is whether forcing the use of
>inheritance is appropriate / a good thing. We might also consider
>spelling it like:
>
>class MyArrayLikeClass:
> _numarray_defer_to_me = True
>
>class NumArray:
> def __mul__(self, operand):
> if hasattr(operand, "_numarray_defer_to_me"):
> return operand.__rmul__(self)
> else:
> return ufunc.multiply(self, operand)
>
>
>
>>The only case where I see a potential problem is an old-style
>>C-extenstion that can't be subclassed. I think that might be a case of
>>YAGNI though.
>>
>>
>
>Sounds YAGNI to me.
>
>
>
>>Avoiding registration is appealing.
>>
>>
>
>Good to have this seconded.
>
>
>
>>-tim
>>
>>
>>
>
>All in all, great ideas!
>
>Todd
>
>
>
|
|
From: Colin J. W. <cj...@sy...> - 2003-09-24 11:31:07
|
Nadav Horesh wrote: >Recently I had an idea to subclass numarray's array in order to >implement a different scheme of array indexing via the __call__ method >(ugly indeed, by my experience shows that it can be useful). It occurred >to me that the array class is not in the spirit of Python 2.2+ >classes/types unification: > I agree. > * The class name is not "trivial": numarraycore.NumArray > * The class constructor is not built to be used by the commons. > I'm not sure what you have in mind here by "trivial" and "commons". >Since arrays are normally generated by functions (zeros, array, >fromfunction etc.) and not by the constructor, subclassing >numarraycore.NumArray is not very useful. If, for instance, "array" (or >"Array") would be the class name, where the constructor would optionally >have the same parameters as the array function (with the option to act >as a copy constructor as the array function does), then it would be easy >to subclass it: > >class MyArray(array): > ... > > >new_array = MyArray(zeros((10,5), type=Int64)) > > > This is desirable, with the present design, one has to go through a two step process: new_array = MyArray(type=Int64, shape= (10, 5)) load_some_data_into_array(new_array, zeros(10, 5)) Colin W. > Any comments? > > Nadav. > > > >------------------------------------------------------- >This sf.net email is sponsored by:ThinkGeek >Welcome to geek heaven. >http://thinkgeek.com/sf >_______________________________________________ >Numpy-discussion mailing list >Num...@li... >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
|
From: Nadav H. <na...@Vi...> - 2003-09-24 07:14:30
|
Recently I had an idea to subclass numarray's array in order to
implement a different scheme of array indexing via the __call__ method
(ugly indeed, by my experience shows that it can be useful). It occurred
to me that the array class is not in the spirit of Python 2.2+
classes/types unification:
* The class name is not "trivial": numarraycore.NumArray
* The class constructor is not built to be used by the commons.
Since arrays are normally generated by functions (zeros, array,
fromfunction etc.) and not by the constructor, subclassing
numarraycore.NumArray is not very useful. If, for instance, "array" (or
"Array") would be the class name, where the constructor would optionally
have the same parameters as the array function (with the option to act
as a copy constructor as the array function does), then it would be easy
to subclass it:
class MyArray(array):
...
new_array = MyArray(zeros((10,5), type=Int64))
Any comments?
Nadav.
|
|
From: Chris B. <Chr...@no...> - 2003-09-23 22:03:08
|
Perry Greenfield wrote:
> I don't recall if there was disagreement about this being useful,
probably not
> but perhaps there was disagreement that it should always do this.
absolutely, and I think there was a wide range of opinions about how
usefull (and or critical!) this is.
> numarray allows one to customize how errors are handled. You can
> choose:
>
> 1) to silently ignore all errors.
> 2) print a warning message (default)
> 3) raise an exception.
>
> One may separately set one of these three behaviors for each of
> the 4 ieee categories of floating point errors, namely
>
> 1) invalid results (i.e., NaNs)
> 2) divide by zeros (Infs)
> 3) overflows
> 4) underflows
Fabulous!!! You folks are wonderfull! I guess it's time to make the
switch to numarray...I've been waiting for a compelling reason!
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Perry G. <pe...@st...> - 2003-09-23 21:11:24
|
Chris Barker writes: > > There was a lot of discussion a long while back about how Numeric treats > erros in the middle of an array operation. The way it works is if one > operation in an array has a problem (overflow, division by zero, etc), > an exception is returned. You have no idea which element caused the > problem, and teh whole operation grinds to a halt. Personally, I would > MUCH prefer MATLAB's way of handling this: the problematic operation > results in a -Inf, Inf, or NaN result for that element, and the rest of > the array operation can continue. MATLAB also provides utility functions > along the line of isnan, isinf, etc, so you can check for the > problematic areas later. > > There was a fair bit of disagreement over whether Numeric should act > similarly, as well as the problem that not all compilers have the > required IEEE support to make this work. > I don't recall if there was disagreement about this being useful, but perhaps there was disagreement that it should always do this. > Anyway, how does numarray handle this issue? I'd love to have the MATLAB > behaviour...I'm writing this because I'm about to have to write a bunch > of code to keep making sure I don't pass a 0 into log(), when I'd be > much happier if I could just replace the NaNs with what I want later. > numarray allows one to customize how errors are handled. You can choose: 1) to silently ignore all errors. 2) print a warning message (default) 3) raise an exception. One may separately set one of these three behaviors for each of the 4 ieee categories of floating point errors, namely 1) invalid results (i.e., NaNs) 2) divide by zeros (Infs) 3) overflows 4) underflows for the first two modes the computation on floats just inserts the appropriate ieee value in the array (Infs and NaNs). There are portable constants and functions to test and set these values (see examples below). These modes also apply to integer calculations. I've pasted some examples I presented from some slides shown at the scipy conference to illustrate how it works and what's available. Customize how errors are handled >>> numeric.Error.setMode(all="warn") >>> numeric.Error.pushMode(dividebyzero="warn", .. Invalid="raise",overflow="warn",underflow="ignore") >>> numeric.Error.popMode() >>> a = 1/arange(10.) Warning: Encountered divide by zero(s) in divide Manipulate IEEE-754 special values >>> import numarray.ieeespecial as ieee >>> ieee.getinf(a) # return indices of Infs (array([0],) >>> a = arange(10.)/arange(10.) >>> ieee.setnan(a, 999) # set all NaNs to 999 >>> a array([999.,1.,1.,1.,1.,1.,1.,1.,1.,1.]) >>> a[ieee.getinf(a)] = 999 # equivalent Use defined constants >>> a[5] = ieee.nan >>> a[6] = ieee.plus_inf >>> a[7] = ieee.minus_inf >>> a[8] = ieee.inf >>> a array([999.,1.,1.,1.,1.,nan,inf,-inf,inf,1.] Perry |
|
From: Chris B. <Chr...@no...> - 2003-09-23 20:48:26
|
Hi all,
There was a lot of discussion a long while back about how Numeric treats
erros in the middle of an array operation. The way it works is if one
operation in an array has a problem (overflow, division by zero, etc),
an exception is returned. You have no idea which element caused the
problem, and teh whole operation grinds to a halt. Personally, I would
MUCH prefer MATLAB's way of handling this: the problematic operation
results in a -Inf, Inf, or NaN result for that element, and the rest of
the array operation can continue. MATLAB also provides utility functions
along the line of isnan, isinf, etc, so you can check for the
problematic areas later.
There was a fair bit of disagreement over whether Numeric should act
similarly, as well as the problem that not all compilers have the
required IEEE support to make this work.
Anyway, how does numarray handle this issue? I'd love to have the MATLAB
behaviour...I'm writing this because I'm about to have to write a bunch
of code to keep making sure I don't pass a 0 into log(), when I'd be
much happier if I could just replace the NaNs with what I want later.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Todd M. <jm...@st...> - 2003-09-23 15:10:38
|
The thread on what to do with boolean array formatting in numarray has died out, so I thought I should explain (what I saw) as the results of the poll: (1) There was a clear majority of numarray users who actually prefer the status quo of 0,1 for representing arrays of boolean values. (2) There was a sizable minority that would like to see consistency with Python-2.3 by using False, True for representing arrays of boolean values. (3) There was a (good) suggestion of using F, T for representing boolean values. However, on reflection and reviewing the poll comments, F, T is not really as good as 0, 1 visually, nor is it likely to satisfy those people interested in consistency with Python, so it is unlikely to make most numarray users any happier. (4) Also suggested was supporting configurable boolean formats. This approach is reasonable to implement, but I have a reservation that adding support for multiple formats may cause as much confusion as it gives satisfaction. Is it *really* worth it? (more later) Given 1-4, I think the best thing to do for now is keep 0,1 as the only format for boolean array values. If someone wants False, True bad enough to submit a complete patch for supporting multiple formats (and defaulting to 0,1), we will still accept it. If anyone is still interested in doing this, the patch should include modified TeX documentation and pass all the self tests in setupall.py (all of core numarray and all current add-ons). The patch should also run cleanly on both Python-2.2 and Python-2.3. Thanks everyone for responding, Todd -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: Todd M. <jm...@st...> - 2003-09-23 11:00:49
|
On Mon, 2003-09-22 at 19:41, Fernando Perez wrote: > Tim Hochberg wrote: > > I actually have no idea how you plan to make keyword arguments work > > here, perhaps you could explain that in more detail. Metaclasses are > > overkill, but a mixin, marker class could be used. That is, when > > designing a class for use with numarray, one would derive a class from a > > marker class in numarray:: > > Well, the joys of being vague :) As I said, I haven't followed in enough > detail to be too specific, so perhaps my idea plain doesn't work. But what I > had in mind was something along the lines of: > > class ArrayLike(numarray): > def __init__(self,...): > ... > numarray.__init__(defers=1) > I think this behavior is already granted by Python at the abstract object level; that is, if you subclass, the r-method of the subclass is given preference over the l-method of the superclass. That's my impression anyway. Thus, I think we already get "deference" for free with subclasses, but that is too weak in general because often (like MA for instance) people won't want to subclass from NumArray. > This is ultimately equivalent to a registration process, but since it is done > in the constructor, it feels less like a separate dangling side-effect. On > the other hand, it makes your new constructor more expensive, and it feels > ugly to be doing at the instance level things which truly belong at the class > level. Hence my comment about this perhaps being better done via metaclasses. > > As I said, I knew I was being vague. But hopefully these comments at least > suggest a way out of a separate registration step. They certainly appear to have done so. Thanks for commenting. Todd -- Todd Miller <jm...@st...> |
|
From: Todd M. <jm...@st...> - 2003-09-23 10:34:04
|
On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote: > I actually have no idea how you plan to make keyword arguments work > here, perhaps you could explain that in more detail. Metaclasses are > overkill, but a mixin, marker class could be used. That is, when > designing a class for use with numarray, one would derive a class from a > marker class in numarray:: > > class MyArrayLikeClass(numarray.DeferToMe): > .... > > Hmmm. That's not too bad. Todd, what do you think about using this logic:: > > def __mul__(self, operand): > if isinstance(operand, DeferToMe): > operand.__rmul__(self) > else: > self.__mul__(operand) > > I like the core idea a lot. My only doubt is whether forcing the use of inheritance is appropriate / a good thing. We might also consider spelling it like: class MyArrayLikeClass: _numarray_defer_to_me = True class NumArray: def __mul__(self, operand): if hasattr(operand, "_numarray_defer_to_me"): return operand.__rmul__(self) else: return ufunc.multiply(self, operand) > The only case where I see a potential problem is an old-style > C-extenstion that can't be subclassed. I think that might be a case of > YAGNI though. Sounds YAGNI to me. > Avoiding registration is appealing. Good to have this seconded. > -tim > All in all, great ideas! Todd -- Todd Miller <jm...@st...> |
|
From: Fernando P. <fp...@co...> - 2003-09-22 23:41:16
|
Tim Hochberg wrote: > I actually have no idea how you plan to make keyword arguments work > here, perhaps you could explain that in more detail. Metaclasses are > overkill, but a mixin, marker class could be used. That is, when > designing a class for use with numarray, one would derive a class from a > marker class in numarray:: Well, the joys of being vague :) As I said, I haven't followed in enough detail to be too specific, so perhaps my idea plain doesn't work. But what I had in mind was something along the lines of: class ArrayLike(numarray): def __init__(self,...): ... numarray.__init__(defers=1) This is ultimately equivalent to a registration process, but since it is done in the constructor, it feels less like a separate dangling side-effect. On the other hand, it makes your new constructor more expensive, and it feels ugly to be doing at the instance level things which truly belong at the class level. Hence my comment about this perhaps being better done via metaclasses. As I said, I knew I was being vague. But hopefully these comments at least suggest a way out of a separate registration step. Regards, Fernando. |
|
From: Tim H. <tim...@ie...> - 2003-09-22 23:27:40
|
Fernando Perez wrote:
> Tim Hochberg wrote:
>
>> I agree that using try/except is probably not worth the danger.
>> However, I think I'd prefer the inverse of what you propose. That is:
>>
>> def __mul__(self, operand):
>> if isinstance(operand, _numarray_nondeferred_types):
>> self.__mul__(operand)
>> else:
>> operand.__rmul__(self)
>>
>>
>> and of course a dont_defer_to registration function. Hopefully with a
>> better name. The choice comes down to whether we defer by default or
>> handle by default. I'm marginally on the side of deferring, but don't
>> feel too strongly either way. Either one would be a big improvement.
>>
>> Perhaps someone else out there has some profound thoughts.
>
>
> Well, certainly not profound, but here's my proverbial $0.02. I tend
> to really dislike separate registration functions, since they reek too
> much of 'programming by side-effect'. Why not put the necessary
> information into a keyword argument to the constructor? Once a
> sensible default is chosen, then it can be overridden with
> 'defers=True' (if the default was False) at __init__ time, for
> example. Since this is a class-wide issue, perhaps it may make more
> sense to handle it via a meta-class mechanism. But I'm not an expert
> on metaclasses, so I'll shut up there :)
I actually have no idea how you plan to make keyword arguments work
here, perhaps you could explain that in more detail. Metaclasses are
overkill, but a mixin, marker class could be used. That is, when
designing a class for use with numarray, one would derive a class from a
marker class in numarray::
class MyArrayLikeClass(numarray.DeferToMe):
....
Hmmm. That's not too bad. Todd, what do you think about using this logic::
def __mul__(self, operand):
if isinstance(operand, DeferToMe):
operand.__rmul__(self)
else:
self.__mul__(operand)
The only case where I see a potential problem is an old-style
C-extenstion that can't be subclassed. I think that might be a case of
YAGNI though. Avoiding registration is appealing.
-tim
|
|
From: Fernando P. <fp...@co...> - 2003-09-22 22:45:41
|
Tim Hochberg wrote: > I agree that using try/except is probably not worth the danger. However, > I think I'd prefer the inverse of what you propose. That is: > > def __mul__(self, operand): > if isinstance(operand, _numarray_nondeferred_types): > self.__mul__(operand) > else: > operand.__rmul__(self) > > > and of course a dont_defer_to registration function. Hopefully with a > better name. The choice comes down to whether we defer by default or > handle by default. I'm marginally on the side of deferring, but don't > feel too strongly either way. Either one would be a big improvement. > > Perhaps someone else out there has some profound thoughts. Well, certainly not profound, but here's my proverbial $0.02. I tend to really dislike separate registration functions, since they reek too much of 'programming by side-effect'. Why not put the necessary information into a keyword argument to the constructor? Once a sensible default is chosen, then it can be overridden with 'defers=True' (if the default was False) at __init__ time, for example. Since this is a class-wide issue, perhaps it may make more sense to handle it via a meta-class mechanism. But I'm not an expert on metaclasses, so I'll shut up there :) I know that the above is rather vague, partly because I haven't followed the whole thread in detail. But I hope that my intent, as a design idea, is clear (and perhaps even useful :) While the actual implementation details (constructor keyword, metaclass, class-wide global like self.__defers, etc.) will have to be settled by the experts, I hope that an approach which doesn't rely on separate registration functions can be achieved. Best regards, Fernando. |
|
From: Tim H. <tim...@ie...> - 2003-09-22 22:27:57
|
Hi Todd,
Glad to have you back. I hope you suffered no permanant damage from the
hurricane. Or the worm for that matter.
Todd Miller wrote:
>On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
>
>
>Sorry for the delay; I was pretty much shut down by hurricane Isabelle
>from the end of last week.
>
>I hit and bypassed this issue trying to port MA.
>
>
>
[SNIP]
>Looking in Python's Objects/abstract.c, the function binop1 shows one
>way to squirm around this issue: subclass from NumArray. It would be
>interesting to know how Numeric does it.
>
>
I think that since Numeric arrays are C classes, they always defer to
any Python class although I'm not sure of the details. In any event, I'm
pretty sure it's an accident.
>>In my case I'm thinking of proxy objects that don't do
>>some computations till they are actually required. So my current
>>thinking is that __mul__ and friends would be best implemented as::
>>
>> def __mul__(self, operand):
>> if not isinstance(operand, knownTypes):
>>
>>
>
>If the "not" belongs there, I'm lost.
>
>
It does belong there. KnownTypes (perhaps _numarray_nondeferred_types)
would be a better term) are types that numarray knows that it should
handle itself. Let me reverse the logic -- maybe it'll be clearer.
def __mul__(self, operand):
if isinstance(operand, _numarray_nondeferred_types):
return operand.__rmul__(self)
else:
try:
return ufunc.multiply(self, operand)
except:
return operand.__rmul__(self)
>I agree with your heuristic, but I am leery of putting an unqualified
>try/except in front of the numarray ufunc code. On the other hand,
>qualifying it seems too complicated.
>
>
I agree on all counts.
>What about adding a tuple of types to be deferred to?
>
> def __mul__(self, operand):
> if isinstance(operand, _numarray_deferred_types):
> operand.__rmul__(self)
> else:
> self.__mul__(operand)
>
>Then other libraries could register classes with something like:
>
>numarray.defer_to(my_class)
>
>which is slightly painful, but avoids masking the ufunc exceptions and
>only needs to be done once for each library base class. How does that
>sound?
>
>
I agree that using try/except is probably not worth the danger. However,
I think I'd prefer the inverse of what you propose. That is:
def __mul__(self, operand):
if isinstance(operand, _numarray_nondeferred_types):
self.__mul__(operand)
else:
operand.__rmul__(self)
and of course a dont_defer_to registration function. Hopefully with a
better name. The choice comes down to whether we defer by default or
handle by default. I'm marginally on the side of deferring, but don't
feel too strongly either way. Either one would be a big improvement.
Perhaps someone else out there has some profound thoughts.
-tim
>Todd
>
>
>
>>-tim
>>
>>
>>
>>
>>----
>>
>>
>>
>
>
>
>>import numarray as na
>>import Numeric as np
>>
>>class Zero:
>> def __add__(self, other):
>> return other
>> __radd__ = __add__
>>zero = Zero()
>>
>>#~ print zero + np.arange(5)
>>#~ print np.arange(5) + zero
>>#~ print zero + na.arange(5)
>>#~ print na.arange(5) + zero
>>
>>a = na.arange(5)
>>
>>import copy
>>copy.deepcopy(a)
>>
>>
|
258 messages has been excluded from this view by a project administrator.
