numpy-discussion — Discussion list for all users of Numerical Python

Re: FW: [Numpy-discussion] typecodes in numarray

[Francesc A. <fa...@op...>]

A Divendres 24 Gener 2003 21:15, Todd Miller va escriure:
> >
> >My current thinking is something like:
> >
> >recarrDescr =3D {
> >    "name"        : defineType(CharType, 16, ""),  # 16-character Stri=
ng
> >    "TDCcount"    : defineType(UInt8, 1, 0),    # unsigned byte
> >    "ADCcount"    : defineType(Int16, 1, 0),    # signed short integer
> >    "grid_i"      : defineType(Int32, 1, 9),    # integer
> >    "grid_j"      : defineType(Int32, 1, 9),    # integer
> >    "pressure"    : defineType(Float32, 1, 1.),  # float=20
> > (single-precision) "temperature" : defineType(Float64, 32, arange(32)=
),=20
> > # double[32] "idnumber"    : defineType(Int64, 1, 0),    # signed lon=
g
> > long }
> >
> >where defineType is a class that accepts (type, shape, default)
> > parameters. It can be extended safely in the future if more needs app=
ear.
>
> You're way ahead of me here.  The only thing I don't like about this is
> the additional relative complexity because of the addition of field
> names and default values.   It would be nice to layer this more.
>

Well, I think a map between field names and values is valuable from the
user's point of view. It may help him to label the different information =
on
the recarray. Moreover, if __getattr__ and __setattr__ methods (or
__getitem__ and __setitem__) would get implemented on recarray (as they a=
re
in my recarray2 version, for example), the field name can become a very
convenient manner to access a specific field by name (this introduce the
limitation that field name must be a valid python identifier, but I think
this is not a big restriction). By looking at the description dictionary,
the user can have a quick idea of what he can find in every field (with n=
o
need of counting, which can be a big advantage specially for long records=
).

With regard to default values, you can make this parameter (even the shap=
e)
a keyword parameter in order to make it optional. In that way, the
definition can be as simple as "defineType(CharType)" (or even just
"Chartype", if you add a bit of code) or as complete as
"defineType(Chartype, shape, default, whatever_you_want)". I think this i=
s
a quite flexible approach.

>One more thing I don't understand looking at this:  a dictionary is=20
>unordered.

Yeah, but this can be regarded as an advantage rather than a drawback in =
the
sense that you can choose the order you (the developer) prefer. For examp=
le,
I was using first a alphanumerical order to arrange the data fields, but
now, I'm considering that a arrangement that optimizes the alignment of t=
he
fields could be far better. As for one, say that you have a (Int8, Int32,
Float64) record; in principle it could be easy to create a routine that
arranges this record in the form (Float64,Int32, Int8) that optimizes the
different field access (it may be even possible to introduce automatic
padding later on if recarrays would support them in the future).

Maybe you are getting confused in thinking that recarrDescr will create t=
he
recarray. Not at all, this a *metadata* definition that can be passed to =
the
actual recarray funtion for recarray creation. Its function would be
similar to the formats parameter (with typical values like "3a,4i,3w") in
recarray.array, but with more verbosity and all the reported advantages.

> >instead of
> >
> >((Int16, 3),
> > (Int32, 4),
> > (Float64, 20),
> > )
>
> This is pretty much exactly what I was thinking.  It is straightforward
> to imagine and difficult to forget.
>
> >the former being more handy in lots of situations.
>
> Would you please name some of these so we can explore handling them bot=
h
> ways?
>

Well, I'm afraid that the best advantage would be when dealing with
recarrays in C extension modules. In this kind of situation it would be f=
ar
better to deal with a "3a4i3w" array than a tuple of python objects. But
maybe I'm wrong and the latter is not so-complicated to manage; however, =
I
used to work a lot with records (even before meeting recarray) and I was
quite comfortable with formats in string mode.

Or perhaps it would be enough to provide a method for converting from the
standard metadata layout (dictionary or tuple or whatever), to a string
format. This should be not very difficult.

> >
> >Well, if charcodes finally stay in, this have an additional advantage =
in
> >that python crew has provided meaningful ways to express padding
> > (character "x"), endianess ("=3D", "<", ">") and alignment ("@").
>
> We might also add these to the type-repetition tuple.

It would be nice, of course.

--=20
Francesc Alted

Re: [Numpy-discussion] Extensive use of methods instead of functions

[Andrew P. L. Jr. <bs...@al...>]

On Fri, 24 Jan 2003, [iso-8859-15] 'José Fonseca' wrote:

> Of course that I have no problems if the Numeric/numarray maintainers
> decide to turn it down. I'll most probably just use UserArray.py to create a
> "method-ized" version of Numeric, so that my algorithms can work with
> both Numeric array and sparse matrices. (I do have a real case need of
> for this.)

Sparse matricies are common enough that they really should be a base part 
of Numeric rather than requiring subclassing/extending/etc.  I know that 
Travis O. was working on some sparse matrix stuff a while back so you 
might want to contact him to get the current status of that work.

-a

Re: [Numpy-discussion] Extensive use of methods instead of functions

[F. <j_r...@ya...>]

On Fri, Jan 24, 2003 at 12:34:54PM -0800, Paul F Dubois wrote:
> 
> Every time the subject of subclassing a numeric array comes up, it as
> if nobody ever thought of it before.

Why do you treat me as if I was trying to sell the "Next Big Thing"!?

First, I must tell you that the first time I came across the idea of
subclassing Numeric arrays was while reading the "Subclassing"
subsection, in the "Special Topics" section of the Numeric Python
manual. Your name, Paul, appears as one of the authors.

Second, subclassing Numeric arrays may be useful. Again, the
distribution of Numeric Python even has one big example: making a linear
algebra oriented version of Numeric python, where the operations would
be the standard matrix and vector operations instead of the element-wise
operations. 
> Been there, done that.  It doesn't turn out to be all that useful. 

As seen by the examples above is obvious you did. Still, I don't see how
can you possibly say it isn't useful...

> To see why, consider a + b where a and b are Foo instances, and Foo
> inherits from numarray.
> 
> a. a + b will be a numarray, not a Foo instance, unless you write a
> new + operator.  b. Attempting to have numarray itself apply a
> subclass constructor to the result runs into the problem that numarray
> does not have any idea what the constructor's signature is or what
> information is needed to fill out that constructor.  c. Even if the
> subclass accepts numarray's constructor signature, it would rarely
> produced satisfactory results just "losing" the Foo'ness details of a
> and b.
> 
> This same argument applies to every method that returns a Foo
> instance, and every ufunc. So you end up redoing everything anyway.

[In general it may be usefully to subclass Numeric arrays if one just
want to add/overload methods, but no new properties.]

And third, if you read my thread you'd notice that the use of methods
instead of functions has implications/benefits much beyond the
subclassing issue. It's particularly important for Numeric-alike arrays. 

All objects in Python are virtual so you don't actually need to subclass
to use different kind of objects in the same piece as code.

While you're right in the sense that for many practical applications
there is little use of subclassing - a sparse matrix class is one of
them for instance -, you can't deny that is quite useful to have
Numeric-alike arrays, in the same basis as is currently done with the
file-alike objects in Python, i.e., they could be strings, web pages but
as long as they define a set of methods, these.

> In short, worrying about subclassing is way down the list of things we
> ought to consider. 

If so, then why did your comment only focused on the subclassing issue?
The subclassing was a mere introduction [perhaps unfortunate, I confess]
to the method overloading issue.  Now, if you could (re)read my first
post and comment on my actual suggestion I would appreciate.

Of course that I have no problems if the Numeric/numarray maintainers
decide to turn it down. I'll most probably just use UserArray.py to create a
"method-ized" version of Numeric, so that my algorithms can work with
both Numeric array and sparse matrices. (I do have a real case need of
for this.)

BTW, there is an alternative to create full-methodized Numeric array:
just add a attribute which points to the module which the class belongs,
e.g., "myarray.module.take" would point to "Numeric.take" if it was a
Numeric array, or "Sparse.take" if it was a sparse matrix.

Regards,

José Fonseca
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com

Re: [Numpy-discussion] Extensive use of methods instead of functions

[F. <j_r...@ya...>]

On Fri, Jan 24, 2003 at 09:07:21PM +0100, Konrad Hinsen wrote:
> José Fonseca <j_r...@ya...> writes:
> 
> > With the ability of subclassing types in recent versions of the Python
> > language, more people will be interested in subclassing Numeric arrays
> > for specific purposes.  Still the use of functions instead of methods
> > takes away many of the advantages, the ability of being overloaded.
> 
> True. On the other hand, there is also an advantage: NumPy routines
> can be used on standard Python data types such as number and sequence
> types.
> 
> In the ideal world (which might come one day), core NumPy
> functionality would be part of standard Python, and then all these
> operations would work on other built-in types as well.
> 
> Until then, I am not sure that changing NumPy functions to methods
> is a good idea. I need to call them on scalar numbers much more
> often than I subclass arrays.

You've got a good point there. I often want to use with other Numeric
array-alike classes, but I've also used them with standard Python data
types for convenience. 

Still, it's perfectly possible to both interfaces to co-exist. Of course
that when one would use the .method version it can't expect to work with
standard Python data types and has to make a choice, or to use asarray()
or something equivalent before using it.

Regards,

José Fonseca
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com

Re: FW: [Numpy-discussion] typecodes in numarray

[Todd M. <jm...@st...>]

>
>
>> My current thinking is something like:
>>
>> recarrDescr = {
>>    "name"        : defineType(CharType, 16, ""),  # 16-character String
>>    "TDCcount"    : defineType(UInt8, 1, 0),    # unsigned byte
>>    "ADCcount"    : defineType(Int16, 1, 0),    # signed short integer
>>    "grid_i"      : defineType(Int32, 1, 9),    # integer
>>    "grid_j"      : defineType(Int32, 1, 9),    # integer
>>    "pressure"    : defineType(Float32, 1, 1.),  # float  
>> (single-precision)
>>    "temperature" : defineType(Float64, 32, arange(32)),  # double[32]
>>    "idnumber"    : defineType(Int64, 1, 0),    # signed long long    }
>>
>> where defineType is a class that accepts (type, shape, default) 
>> parameters.
>> It can be extended safely in the future if more needs appear.
>>
> You're way ahead of me here.  The only thing I don't like about this 
> is the additional relative complexity because of the addition of field 
> names and default values.   It would be nice to layer this more. 

One more thing I don't understand looking at this:  a dictionary is 
unordered.

Todd

RE: [Numpy-discussion] Extensive use of methods instead of functions

[Perry G. <pe...@st...>]

Paul Dubois writes:
>
> Every time the subject of subclassing a numeric array comes up, it as if
> nobody ever thought of it before. Been there, done that. It
> doesn't turn out
> to be all that useful. To see why, consider a + b where a and b are Foo
> instances, and Foo inherits from numarray.
>
> a. a + b will be a numarray, not a Foo instance, unless you write a new +
> operator.
> b. Attempting to have numarray itself apply a subclass constructor to the
> result runs into the problem that numarray does not have any idea what the
> constructor's signature is or what information is needed to fill out that
> constructor.
> c. Even if the subclass accepts numarray's constructor signature, it would
> rarely produced satisfactory results just "losing" the Foo'ness
> details of a
> and b.
>
> This same argument applies to every method that returns a Foo
> instance, and
> every ufunc. So you end up redoing everything anyway.
>
> In short, worrying about subclassing is way down the list of
> things we ought
> to consider.
>
Paul illustrates some important points. While I'm not as down on the
ability to subclass (more on that later), he is absolutely right that
most think that subclassing is a breeze and don't realize that it
is far from being so.

The arguments for this would be helped immensely by a practical
example of a desired subclass. This does far more to illustrate
the issues than an abstract discussion. For most instances that I
have considered or thought about it is unavoidable that one must
override virtually all (if not all) the operators and functions.
Nevertheless, subclassing can still save a great deal of work
over implementing a completely new extension. But you'll have to
deal with defining how all the operators and functions should behave.

In our view, the most valuable subclassing in numarray comes from
subclassing NDArray, which handles all the structural operations
for arrays (recarray makes heavy use of this). But recarrays don't
try to support numerical operations, and that makes it fairly easy.
Subclassing numarrays is significantly more work for the reasons cited.

Perry

RE: [Numpy-discussion] Extensive use of methods instead of functions

[Paul F D. <pa...@pf...>]

Every time the subject of subclassing a numeric array comes up, it as if
nobody ever thought of it before. Been there, done that. It doesn't turn =
out
to be all that useful. To see why, consider a + b where a and b are Foo
instances, and Foo inherits from numarray.

a. a + b will be a numarray, not a Foo instance, unless you write a new =
+
operator.
b. Attempting to have numarray itself apply a subclass constructor to =
the
result runs into the problem that numarray does not have any idea what =
the
constructor's signature is or what information is needed to fill out =
that
constructor.
c. Even if the subclass accepts numarray's constructor signature, it =
would
rarely produced satisfactory results just "losing" the Foo'ness details =
of a
and b.

This same argument applies to every method that returns a Foo instance, =
and
every ufunc. So you end up redoing everything anyway.

In short, worrying about subclassing is way down the list of things we =
ought
to consider.=20

> -----Original Message-----
> From: num...@li...=20
> [mailto:num...@li...] On=20
> Behalf Of Konrad Hinsen
> Sent: Friday, January 24, 2003 12:07 PM
> To: Jos=E9 Fonseca
> Cc: num...@li...
> Subject: Re: [Numpy-discussion] Extensive use of methods=20
> instead of functions
>=20
>=20
> Jos=E9 Fonseca <j_r...@ya...> writes:
>=20
> > With the ability of subclassing types in recent versions of=20
> the Python=20
> > language, more people will be interested in subclassing=20
> Numeric arrays=20
> > for specific purposes.  Still the use of functions instead=20
> of methods=20
> > takes away many of the advantages, the ability of being overloaded.
>=20
> True. On the other hand, there is also an advantage: NumPy=20
> routines can be used on standard Python data types such as=20
> number and sequence types.
>=20
> In the ideal world (which might come one day), core NumPy=20
> functionality would be part of standard Python, and then all=20
> these operations would work on other built-in types as well.
>=20
> Until then, I am not sure that changing NumPy functions to=20
> methods is a good idea. I need to call them on scalar numbers=20
> much more often than I subclass arrays.
>=20
> Konrad.
> --=20
> --------------------------------------------------------------
> -----------------
> Konrad Hinsen                            | E-Mail:=20
> hi...@cn...
> Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
> Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
> 45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
> France                                   | Nederlands/Francais
> --------------------------------------------------------------
> -----------------
>=20
>=20
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld =3Domething 2=20
> See! http://www.vasoftware.com=20
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>=20

Re: [Numpy-discussion] Extensive use of methods instead of functions

[Konrad H. <hi...@cn...>]

Jos=E9 Fonseca <j_r...@ya...> writes:

> With the ability of subclassing types in recent versions of the Python
> language, more people will be interested in subclassing Numeric arrays
> for specific purposes.  Still the use of functions instead of methods
> takes away many of the advantages, the ability of being overloaded.

True. On the other hand, there is also an advantage: NumPy routines
can be used on standard Python data types such as number and sequence
types.

In the ideal world (which might come one day), core NumPy
functionality would be part of standard Python, and then all these
operations would work on other built-in types as well.

Until then, I am not sure that changing NumPy functions to methods
is a good idea. I need to call them on scalar numbers much more
often than I subclass arrays.

Konrad.
--=20
-------------------------------------------------------------------------=
------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------=
------

Re: FW: [Numpy-discussion] typecodes in numarray

[Todd M. <jm...@st...>]

>
>
>>A means for specifying a recarray format might be created from tuples,
>>type objects,  and integer repetition factors.
>>
>>The verbosity of this approach might be a litte tedious,  but it would
>>also be transparent, maintainable, and conflict free.
>>    
>>
>
>I think this is a very good idea. In fact, while working in PyTables I was
>lately pondering what would be the best way to define record arrays, and I
>also think that a verbose approach should be the beast.
>
>After considering metaclasses, and tuples, I ended to a compromise solution
>between both which are dictionaries combined with some function or class to
>refine the definition.
>
>My current thinking is something like:
>
>recarrDescr = {
>    "name"        : defineType(CharType, 16, ""),  # 16-character String
>    "TDCcount"    : defineType(UInt8, 1, 0),    # unsigned byte
>    "ADCcount"    : defineType(Int16, 1, 0),    # signed short integer
>    "grid_i"      : defineType(Int32, 1, 9),    # integer
>    "grid_j"      : defineType(Int32, 1, 9),    # integer
>    "pressure"    : defineType(Float32, 1, 1.),  # float  (single-precision)
>    "temperature" : defineType(Float64, 32, arange(32)),  # double[32]
>    "idnumber"    : defineType(Int64, 1, 0),    # signed long long 
>    }
>
>where defineType is a class that accepts (type, shape, default) parameters.
>It can be extended safely in the future if more needs appear.
>
You're way ahead of me here.  The only thing I don't like about this is 
the additional relative complexity because of the addition of field 
names and default values.   It would be nice to layer this more.

>Perhaps you may want to consider this for using in recarray definition.
>
We'll definitely consider it as we hash this out.  

>
>  
>
>>I think we should add an "obsolescent feature" warning to numarray and
>>recarray which flags any use of character typecodes when the appropriate
>>command line switches are set.
>>    
>>
>
>Well, I don't fully agree with that. I do believe that classes typecodes to
>be a more meaningful way for describing types, but charcodes can be quite
>advantageous in certain situations, like in describing in compact way the
>contents of a record, or passing this info to C-routines to deal with the
>data.
>
Yeah, I know.

>For example, consider the benefits of describing a recarray format as:
>
>"3s4i20d"
>
I know.

>
>instead of
>
>((Int16, 3), 
> (Int32, 4),
> (Float64, 20),
> )
>
This is pretty much exactly what I was thinking.  It is straightforward 
to imagine and difficult to forget.  

>
>the former being more handy in lots of situations.
>  
>
Would you please name some of these so we can explore handling them both 
ways?

>I certainly believe that a coexistence of both can be very beneficious, specially for 3rd party extension makers (like me :).
>
If there's a reasonable way to avoid supporting both,  we should.

>>>Suggestion: if recarray charcodes are not necessary to match the Numeric
>>>ones, I propose that using the Python convention maybe a good idea.
>>>Look at the table in:
>>>http://www.python.org/doc/current/lib/module-struct.html.
>>>      
>>>
>>This sounds good to me,  except that it will break an existing interface
>>that I don't have control over.  Therefore,  I suggest we correct the
>>problem by coming up with something better.
>>    
>>
>
>Well, if charcodes finally stay in, this have an additional advantage in
>that python crew has provided meaningful ways to express padding (character
>"x"), endianess ("=", "<", ">") and alignment ("@"). 
>
We might also add these to the type-repetition tuple.

Regards,
Todd

RE: [Numpy-discussion] typecodes in numarray

[Perry G. <pe...@st...>]

I think Todd was referring to the recent addition of unsigned types
to Numeric, along with came new typecodes. These types were already
in numarray at the time.

Perry

> -----Original Message-----
> From: Paul F Dubois [mailto:pa...@pf...]
> Sent: Friday, January 24, 2003 12:42 PM
> To: 'Perry Greenfield'; fa...@op...;
> num...@li...
> Subject: RE: [Numpy-discussion] typecodes in numarray
>
>
> I don't understand this remark:
>
> <snip >but I am making an argument that perhaps
> > numarray should only go so far in the support of what I regard as an
> > obsolescent feature.  If the Numeric developers choose to continue
> > extending the use of typecodes in ways that are incompatible with
> > numarray,  one way of dealing with it is to "just say no".
> > We are going
> > beyond the scope of backwards compatability to on-going compatabilty.
> > (Which we may still have to do but needs to be discussed and
> > considered)
> >
>
> There is no "on-going" Numeric development. It stops the minute
> numarray is
> ready. Period. We developers all agreed on that. The whole reason for
> numarray is that Numeric was pronounced unmaintainable and unextendable by
> those who frequently had to work on it. To do anything else will fragment
> the entire numerical python community and software set.
>
>
>
>
>
>

[Fwd: Re: [Numpy-discussion] typecodes in numarray]

[Todd M. <jm...@st...>]

Re: FW: [Numpy-discussion] typecodes in numarray

[Francesc A. <fa...@op...>]

A Divendres 24 Gener 2003 18:02, Todd Miller va escriure:
>
> My [i.e. Todd's]  thoughts about it:
>
> No.  It shows you're thinking about it carefully.   Having looked at al=
l
> of the examples below,  I have some comments:

I mostly agree with your comments, but let point out some thoughts

>
> 1.  The sparseness and obscurity of the typecode "wordspace" are both
> demonstrated here.  There are so few letters to choose from,  they're
> often already used in some other context.  Even given the large number
> of unused letters,  it's often difficult to choose good ones and to
> remember what has been chosen.  I think this is one of the reasons Perr=
y
> chose to replace typecodes with true type objects which have rich,
> regular, and predictable symbolic names.

I completely agree that type objects is a brilliant idea.

> 3. STSCI has layered other software on top of numarray and recarray
> which astronomers use to do work.   It is the friction of that interfac=
e
> which makes correcting these consistency problems more difficult than
> might be immediately apparent.

Yeah, I know...

>
> >I think it's important to agree with a definitive set of charcodes and=
 use
> >them uniformly throughout numarray.
>
> I wish this were possible,  but I'm thinking we should try to find an
> alternative approach altogether,  one which may be more verbose but
> implicitly free of conflict.
>
> A means for specifying a recarray format might be created from tuples,
> type objects,  and integer repetition factors.
>
> The verbosity of this approach might be a litte tedious,  but it would
> also be transparent, maintainable, and conflict free.

I think this is a very good idea. In fact, while working in PyTables I wa=
s
lately pondering what would be the best way to define record arrays, and =
I
also think that a verbose approach should be the beast.

After considering metaclasses, and tuples, I ended to a compromise soluti=
on
between both which are dictionaries combined with some function or class =
to
refine the definition.

My current thinking is something like:

recarrDescr =3D {
    "name"        : defineType(CharType, 16, ""),  # 16-character String
    "TDCcount"    : defineType(UInt8, 1, 0),    # unsigned byte
    "ADCcount"    : defineType(Int16, 1, 0),    # signed short integer
    "grid_i"      : defineType(Int32, 1, 9),    # integer
    "grid_j"      : defineType(Int32, 1, 9),    # integer
    "pressure"    : defineType(Float32, 1, 1.),  # float  (single-precisi=
on)
    "temperature" : defineType(Float64, 32, arange(32)),  # double[32]
    "idnumber"    : defineType(Int64, 1, 0),    # signed long long=20
    }

where defineType is a class that accepts (type, shape, default) parameter=
s.
It can be extended safely in the future if more needs appear.

Dictionary has the advantage over tuple in that you can map column name t=
o
their contents quite easily, and is more flexible than defining the field=
s
with a metaclass descendent (see
http://pytables.sourceforge.net/html-doc/usersguide-html3.html#subsection=
3.1.2)
because dictionarys can be built-up in run-time (although that also migth
metaclass descendents, but in a more misterious way that I think is not
worth of). In addition, dictionary object is available in all python vers=
ion
whereas metaclasses only from 2.2 on. However, I regard metaclasses as th=
e
most elegant solution (but elegance is not always equivalent to convenien=
ce
:().

Perhaps you may want to consider this for using in recarray definition.

>
> I think we should add an "obsolescent feature" warning to numarray and
> recarray which flags any use of character typecodes when the appropriat=
e
> command line switches are set.

Well, I don't fully agree with that. I do believe that classes typecodes =
to
be a more meaningful way for describing types, but charcodes can be quite
advantageous in certain situations, like in describing in compact way the
contents of a record, or passing this info to C-routines to deal with the
data.

For example, consider the benefits of describing a recarray format as:

"3s4i20d"

instead of

((Int16, 3),=20
 (Int32, 4),
 (Float64, 20),
 )

the former being more handy in lots of situations.

I certainly believe that a coexistence of both can be very beneficious,
specially for 3rd party extension makers (like me :).

>
> >Suggestion: if recarray charcodes are not necessary to match the Numer=
ic
> >ones, I propose that using the Python convention maybe a good idea.
> >Look at the table in:
> >http://www.python.org/doc/current/lib/module-struct.html.
>
> This sounds good to me,  except that it will break an existing interfac=
e
> that I don't have control over.  Therefore,  I suggest we correct the
> problem by coming up with something better.

Well, if charcodes finally stay in, this have an additional advantage in
that python crew has provided meaningful ways to express padding (charact=
er
"x"), endianess ("=3D", "<", ">") and alignment ("@"). So having a compac=
t
expresion like "@3sx4i20d", apart from resembling chinese to occidentals,
may give a lot of info in a handy way.

--=20
Francesc Alted

RE: [Numpy-discussion] typecodes in numarray

[Paul F D. <pa...@pf...>]

I don't understand this remark:

<snip >but I am making an argument that perhaps 
> numarray should only go so far in the support of what I regard as an 
> obsolescent feature.  If the Numeric developers choose to continue 
> extending the use of typecodes in ways that are incompatible with 
> numarray,  one way of dealing with it is to "just say no".  
> We are going 
> beyond the scope of backwards compatability to on-going compatabilty. 
> (Which we may still have to do but needs to be discussed and 
> considered)
> 

There is no "on-going" Numeric development. It stops the minute numarray is
ready. Period. We developers all agreed on that. The whole reason for
numarray is that Numeric was pronounced unmaintainable and unextendable by
those who frequently had to work on it. To do anything else will fragment
the entire numerical python community and software set.

FW: [Numpy-discussion] typecodes in numarray

[Perry G. <pe...@st...>]

Todd Miller had some further comments that I thought were worth
posting as well (and I think he makes some very good points).

************************************************************************

My [i.e. Todd's]  thoughts about it:

>Maybe I'm becoming a bit tedious with this, but if you look at:
>
No.  It shows you're thinking about it carefully.   Having looked at all 
of the examples below,  I have some comments:

1.  The sparseness and obscurity of the typecode "wordspace" are both 
demonstrated here.  There are so few letters to choose from,  they're 
often already used in some other context.  Even given the large number 
of unused letters,  it's often difficult to choose good ones and to 
remember what has been chosen.  I think this is one of the reasons Perry 
chose to replace typecodes with true type objects which have rich, 
regular, and predictable symbolic names.

2. Typecodes were added as a backwards compatability feature of 
numarray,  and I think it's probable that numarray beat Numeric to 
supporting most of these types, because otherwise they'd have been 
copied directly and there would be no problem.  I'm not really trying to 
play a blame-game here,  but I am making an argument that perhaps 
numarray should only go so far in the support of what I regard as an 
obsolescent feature.  If the Numeric developers choose to continue 
extending the use of typecodes in ways that are incompatible with 
numarray,  one way of dealing with it is to "just say no".  We are going 
beyond the scope of backwards compatability to on-going compatabilty. 
(Which we may still have to do but needs to be discussed and considered)

3. STSCI has layered other software on top of numarray and recarray 
which astronomers use to do work.   It is the friction of that interface 
which makes correcting these consistency problems more difficult than 
might be immediately apparent.

>I think it's important to agree with a definitive set of charcodes and use
>them uniformly throughout numarray.
>
I wish this were possible,  but I'm thinking we should try to find an 
alternative approach altogether,  one which may be more verbose but 
implicitly free of conflict.

A means for specifying a recarray format might be created from tuples, 
type objects,  and integer repetition factors.

The verbosity of this approach might be a litte tedious,  but it would 
also be transparent, maintainable, and conflict free.

I think we should add an "obsolescent feature" warning to numarray and 
recarray which flags any use of character typecodes when the appropriate 
command line switches are set.

>Suggestion: if recarray charcodes are not necessary to match the Numeric
>ones, I propose that using the Python convention maybe a good idea.
>Look at the table in:
>http://www.python.org/doc/current/lib/module-struct.html.
>
This sounds good to me,  except that it will break an existing interface 
that I don't have control over.  Therefore,  I suggest we correct the 
problem by coming up with something better.

RE: [Numpy-discussion] typecodes in numarray

[Perry G. <pe...@st...>]

> -----Original Message-----
> From: num...@li...
> [mailto:num...@li...]On Behalf Of
> Francesc Alted
> Sent: Friday, January 24, 2003 7:00 AM
> To: num...@li...
> Subject: [Numpy-discussion] typecodes in numarray
> 
> 
> Maybe I'm becoming a bit tedious with this, but if you look at:
> 
No, this sort of feedback is very valuable. We'll think about this a
bit, but I'd agree that consistency with Numeric codes is important. Some
of the history of the codes used by recarray arise from conventions
used in other software not related to Python or Numeric. But if
recarray is to be generic and used by others, we should hide, remove
or layer such conventions in a subclass. Let us think about how we should
do that.

Thanks, Perry 

[Numpy-discussion] typecodes in numarray

[Francesc A. <fa...@op...>]

Maybe I'm becoming a bit tedious with this, but if you look at:

>>> import numerictypes
>>> numerictypes.typecode
{Complex64: 'D', Int32: 'l', UInt16: 's', Complex32: 'F', Float64: 'd',
UInt8: 'b', Int16: 's', Float32: 'f', Int8: '1'}

you can find some incongruencies that lead to weird things like:

>>> array([1,2], Int16).typecode()
's'
>>> array([1,2], UInt16).typecode()
's'  #  --> same as Int16!
>>> array([1,2], Int64).typecode()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.2/site-packages/numarray/numarray.py", lin=
e=20
730, in typecode
    return numerictypes.typecode[self._type]
KeyError: numarray type: Int64
>>> array([1,2], UInt64).typecode()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.2/site-packages/numarray/numarray.py", lin=
e=20
730, in typecode
    return numerictypes.typecode[self._type]
KeyError: numarray type: UInt64

Also, 'l' is used here to map Int32, while in recarray is used to map Boo=
lean.

Moreover, Numeric 22.0 introduced the equivalent of UInt16 and UInt32 typ=
es
as 'w' and 'u' respectively. But, again, 'u' is used in recarray as synon=
ym
of Uint8.

I think it's important to agree with a definitive set of charcodes and us=
e
them uniformly throughout numarray.

Suggestion: if recarray charcodes are not necessary to match the Numeric
ones, I propose that using the Python convention maybe a good idea.
Look at the table in:
http://www.python.org/doc/current/lib/module-struct.html.

--=20
Francesc Alted

[Numpy-discussion] Extensive use of methods instead of functions

[F. <j_r...@ya...>]

With the ability of subclassing types in recent versions of the Python
language, more people will be interested in subclassing Numeric arrays
for specific purposes.  Still the use of functions instead of methods
takes away many of the advantages, the ability of being overloaded.

Taking this statement as an example:

	Numeric.put(myarray, myindices, myvalues)

In the current state of affairs, if we wanted to have to statment to
work with asparse matrix class derived from a Numeric array, it would
have to be something like:

	Sparse.put(myarray, myindices, myvalues)

That is, it forces to the underlaying code to know whether is dealing
with Numeric arrays, or some other equivalent class. But it would be
much more useful to have simply:

	myarray.put(myindices, myvalues)

which would work regardless of the actual type of myarray, provided it
supplied the put() method. This would improve enormously code
reusability and extensability.

I know that there are certain implementations details that may difficult
this (like many functions being implemented in pure Python), but any
advances made in this since will be an improvement of the current
situation.

Also, I know that this example is a little unhappy because numarray will
do these things with the __getitem__ and __setitem__ operators. But
others could easily be shown.

Regards,

José Fonseca
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com

Re: [Numpy-discussion] Have a problem: what is attribute 'compress'

[Todd M. <jm...@st...>]

Sebastian Haase wrote:

>Hi,
>I can print numarray of any int time just fine, but
>
OK.  I am assuming you deleted all of your old numarray installations as 
I recommended and reinstalled numarray-0.4.

What is your PYTHONPATH?

>I still get the compress error message with Float (or complex)
>data:
>  
>
>>>>c
>>>>array([[0, 0, 0, ..., 0, 0, 0],
>>>>        
>>>>
>       [0, 0, 0, ..., 0, 0, 0],
>       [0, 0, 0, ..., 0, 0, 0],
>       ...,
>       [0, 0, 0, ..., 0, 0, 0],
>       [0, 0, 0, ..., 0, 0, 0],
>       [0, 0, 0, ..., 0, 0, 0]], type=UInt16)
>  
>
>>>>c.astype(na.Float)
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<input>", line 1, in ?
>  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
>__repr__
>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
>array2string
>    separator, array_output)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
>_array2string
>    format, item_length = _floatFormat(data, precision, suppress_small)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
>_floatFormat
>    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
>data))
>AttributeError: 'module' object has no attribute 'compress'
>
>I get this on Windows (2000) and on Linux. Both numarray 0.4
>  
>
I'm not sure what's going on here,  but I develop on both platforms, 
 and Linux constantly.    The self tests definitely pass in Linux.   It 
must be some kind of environment issue or runtime issue.  What happens 
when you type:

 >>> import numtestall
 >>> numtestall.test()
... what gets printed here? ...

>Thanks,
>Sebastian
>
>
>
>----- Original Message -----
>From: "Todd Miller" <jm...@st...>
>To: "Sebastian Haase" <ha...@ms...>
>Cc: <Num...@li...>
>Sent: Thursday, December 19, 2002 5:58 AM
>Subject: Re: [Numpy-discussion] Have a problem: what is attribute 'compress'
>
>
>  
>
>>Sebastian Haase wrote:
>>
>>    
>>
>>>Hi!
>>>Somehow I have a problem with numarray. Please take a look at this:
>>>
>>>      
>>>
>>Hi Sebastian,
>>
>>I've don't recall seeing anything like this,  nor can I reproduce it
>>now.   If you've been following numarray for a while now,  I can say
>>that it is important to remove the old version of numarray before
>>installing the new version.   I recommend deleting your current
>>installation and reinstalling numarray.
>>
>>compress() is a ufunc,  much like add() or put().  It is defined in
>>ndarray.py,  right after the import of the modules ufunc and _ufunc.
>>_ufunc in particular is a problematic module,  because it has followed
>>the atypical development path of moving from C-code to Python code.
>> Because of this, and the fact that a .so or .dll overrides a .py,
>> older installations interfere with newer ones.  The atypical path was
>>required because the original _ufuncmodule.c was so large that it could
>>not be compiled on some systems;  as a result,  I split _ufuncmodule.c
>>into pieces by data type and now use _ufunc.py to glue the pieces
>>    
>>
>together.
>  
>
>>Good luck!    Please let me know if reinstalling doesn't clear up the
>>problem.
>>
>>Todd
>>
>>    
>>
>>>      
>>>
>>>>>>import numarray as na
>>>>>>na.array([0, 0])
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>array([0, 0])
>>>
>>>
>>>      
>>>
>>>>>>na.array([0.0, 0.0])
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>Traceback (most recent call last):
>>> File "<input>", line 1, in ?
>>> File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
>>>__repr__
>>>   MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
>>> File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163,
>>>      
>>>
>in
>  
>
>>>array2string
>>>   separator, array_output)
>>> File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125,
>>>      
>>>
>in
>  
>
>>>_array2string
>>>   format, item_length = _floatFormat(data, precision, suppress_small)
>>> File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246,
>>>      
>>>
>in
>  
>
>>>_floatFormat
>>>   non_zero = numarray.abs(numarray.compress(numarray.not_equal(data,
>>>      
>>>
>0),
>  
>
>>>data))
>>>AttributeError: 'module' object has no attribute 'compress'
>>>
>>>The same workes fine with Numeric. But I would prefer numarray because
>>>      
>>>
>I'm
>  
>
>>>writing C++-extensions and I need "unsigned shorts".
>>>
>>>What is this error about?
>>>
>>>Thanks,
>>>Sebastian
>>>
>>>
>>>
>>>
>>>-------------------------------------------------------
>>>This SF.NET email is sponsored by: Order your Holiday Geek Presents Now!
>>>Green Lasers, Hip Geek T-Shirts, Remote Control Tanks, Caffeinated Soap,
>>>MP3 Players,  XBox Games,  Flying Saucers,  WebCams,  Smart Putty.
>>>T H I N K G E E K . C O M       http://www.thinkgeek.com/sf/
>>>_______________________________________________
>>>Numpy-discussion mailing list
>>>Num...@li...
>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>>
>>>      
>>>
>>
>>
>>    
>>
>
>
>
>-------------------------------------------------------
>This SF.NET email is sponsored by:
>SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
>http://www.vasoftware.com
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>

Re: [Numpy-discussion] Have a problem: what is attribute 'compress'

[Sebastian H. <ha...@ms...>]

Hi,
I can print numarray of any int time just fine, but
I still get the compress error message with Float (or complex)
data:
>>>c
>>>array([[0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       ...,
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0]], type=UInt16)
>>>c.astype(na.Float)
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
__repr__
    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
array2string
    separator, array_output)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
_array2string
    format, item_length = _floatFormat(data, precision, suppress_small)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
_floatFormat
    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
data))
AttributeError: 'module' object has no attribute 'compress'

I get this on Windows (2000) and on Linux. Both numarray 0.4

Thanks,
Sebastian



----- Original Message -----
From: "Todd Miller" <jm...@st...>
To: "Sebastian Haase" <ha...@ms...>
Cc: <Num...@li...>
Sent: Thursday, December 19, 2002 5:58 AM
Subject: Re: [Numpy-discussion] Have a problem: what is attribute 'compress'


> Sebastian Haase wrote:
>
> >Hi!
> >Somehow I have a problem with numarray. Please take a look at this:
> >
> Hi Sebastian,
>
> I've don't recall seeing anything like this,  nor can I reproduce it
> now.   If you've been following numarray for a while now,  I can say
> that it is important to remove the old version of numarray before
> installing the new version.   I recommend deleting your current
> installation and reinstalling numarray.
>
> compress() is a ufunc,  much like add() or put().  It is defined in
> ndarray.py,  right after the import of the modules ufunc and _ufunc.
> _ufunc in particular is a problematic module,  because it has followed
> the atypical development path of moving from C-code to Python code.
>  Because of this, and the fact that a .so or .dll overrides a .py,
>  older installations interfere with newer ones.  The atypical path was
> required because the original _ufuncmodule.c was so large that it could
> not be compiled on some systems;  as a result,  I split _ufuncmodule.c
> into pieces by data type and now use _ufunc.py to glue the pieces
together.
>
> Good luck!    Please let me know if reinstalling doesn't clear up the
> problem.
>
> Todd
>
> >
> >
> >>>>import numarray as na
> >>>>na.array([0, 0])
> >>>>
> >>>>
> >array([0, 0])
> >
> >
> >>>>na.array([0.0, 0.0])
> >>>>
> >>>>
> >Traceback (most recent call last):
> >  File "<input>", line 1, in ?
> >  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
> >__repr__
> >    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
> >  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163,
in
> >array2string
> >    separator, array_output)
> >  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125,
in
> >_array2string
> >    format, item_length = _floatFormat(data, precision, suppress_small)
> >  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246,
in
> >_floatFormat
> >    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data,
0),
> >data))
> >AttributeError: 'module' object has no attribute 'compress'
> >
> >The same workes fine with Numeric. But I would prefer numarray because
I'm
> >writing C++-extensions and I need "unsigned shorts".
> >
> >What is this error about?
> >
> >Thanks,
> >Sebastian
> >
> >
> >
> >
> >-------------------------------------------------------
> >This SF.NET email is sponsored by: Order your Holiday Geek Presents Now!
> >Green Lasers, Hip Geek T-Shirts, Remote Control Tanks, Caffeinated Soap,
> >MP3 Players,  XBox Games,  Flying Saucers,  WebCams,  Smart Putty.
> >T H I N K G E E K . C O M       http://www.thinkgeek.com/sf/
> >_______________________________________________
> >Numpy-discussion mailing list
> >Num...@li...
> >https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> >
>
>
>
>

Re: [Numpy-discussion] Incomplete support in certain Numeric emulation functions in numarray

[Francesc A. <fa...@op...>]

A Dimecres 22 Gener 2003 15:51, Todd Miller va escriure:
>
> I did a simple implementation of PyArray_DescrFromType trying to add
> support for f2py.

> There are 2 real issues with it that I see:
>
> 1.  It still doesn't handle character codes.  I think it could handle
> both NumericTypes and character codes without conflict because of the
> way the ASCII character set is layed out.

I think so

>
> 2. I just added it so that it *could* be called since I think f2py
> needed it.  I didn't call it anywhere from the other compatability
> functions.
>

I tried to patch your PyArray_DescrFromType, but nothing has changed
because, as you said, any compatabilty function call it.

> Care to do another patch?

Well, I've tried to patch the NA_NewAll funtion in newarray.c:

        typeObject =3D pNumType[type];
        if (!typeObject) {
           /* Test if it is a Numeric charcode */
           sprintf(strcharcode, "%c", type);
           charcode =3D PyString_FromString(strcharcode);
           typeobj =3D PyDict_GetItemString(pNumericTypesTDict, strcharco=
de);
           if (typeobj) {
              typeObject =3D typeobj;
           } else
             return (PyArrayObject *) PyErr_Format(_Error,
                   "Type object lookup returned NULL for type %d", type);
        }

instead of the original code:

        typeObject =3D pNumType[type];
        if (!typeObject)
                return (PyArrayObject *) PyErr_Format(_Error,
                    "Type object lookup returned NULL for type %d", type)=
;
       =20
with no luck as the segmentation fault continues to appear.

Anyway, I've already patched my original code to use only integer codes, =
not
character, so it would be a problem (at least for me).

> They're still not quite right,  because the interface is written in
> terms of int arrays,  which is not good for LP64 platforms where long i=
s
> really what is needed to avoid creating 2G bottlenecks.  The naming is
> also not consistent and I will want to make it so before release of
> numarray-0.5.

Ok, so perhaps it's better to use the PyArray_FromDims rather than NA_Emp=
ty
(at least, until the C-API stabilizes). It's good to know that!.

BTW, during the patching work of numarray sources I perceived some missin=
g
character code types in numerictypes.py. These are the correspondents to:
UInt16, Int64 and UInt64. In recarray, they don't appear neither (except =
for
Int64 which appears as 'N' in numfmt, but with no correspondant in revfmt=
),
so one can't build-up recarrays with these types because you need a charc=
ode
for the "formats" string.

Is this intentional? Do you plan to fill these gaps (it would be nice,
specially for recarrays)?

Thanks,

--=20
Francesc Alted

Re: [Numpy-discussion] Incomplete support in certain Numeric emulation functions in numarray

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>Hi,
>
>I have discovered that the Numeric emulation functions in numarray doesn't
>accept a character typecode as type parameter.
>
Interesting.  

>
>This is not immediately apparent because type parameter is of type 'int',
>and passing it a 'char' maybe not a good practice. 
>
I wrote the emulation functions using the manual and intuition rather 
than the existing code.  There will be others like this.

>But the fact is that
>Numeric *do* accept the charcodes in the type parameter. 
>
>  
>
No argument here.  numarray can "always" be more compatible than it is 
"now",  for any value of always or now.  I think the only real way to 
avoid that would be to build Numeric into numarray,  which sounds 
dubious. :)

>For example, this is the normal way to call the PyArray_FromDims function:
>
>arr = PyArray_FromDims(self.rank, self.dimensions, tFloat64)
>
>but, in Numeric, this other manner also works:
>
>arr = PyArray_FromDims(self.rank, self.dimensions, 'd')
>  
>
This was nicely illustrated.

>Now, in numarray, if you pass a character to the type parameter, a
>"segmentation fault" is issued.
>  
>
Decidedly not good.

>Look at the end of Numeric-22.0/Src/arraytypes.c, to see how characters are
>handled as types in Numeric. I think something like this should be added to
>the deferred_libnumarray_init in numarray-0.4/Src/newarray.ch.
>
I did a simple implementation of PyArray_DescrFromType trying to add 
support for f2py.

There are 2 real issues with it that I see:

1.  It still doesn't handle character codes.  I think it could handle 
both NumericTypes and character codes without conflict because of the 
way the ASCII character set is layed out.

2. I just added it so that it *could* be called since I think f2py 
needed it.  I didn't call it anywhere from the other compatability 
functions.

Care to do another patch?  

>Another thing. It seems to me that NA_New and NA_Empty functions are not
>well documented in the numarray documentation as they differ from the
>definitions in numarray-0.4/Src/newarray.ch. I hope that the latter will
>stay, because I prefer them a lot more than the documented ones :-)
>
If you're working from CVS,  the form they're in now was the result of 
someone's detailed comments.

They're still not quite right,  because the interface is written in 
terms of int arrays,  which is not good for LP64 platforms where long is 
really what is needed to avoid creating 2G bottlenecks.  The naming is 
also not consistent and I will want to make it so before release of
numarray-0.5.

>Bye,
>
>  
>
Todd

[Numpy-discussion] Incomplete support in certain Numeric emulation functions in numarray

[Francesc A. <fa...@op...>]

Hi,

I have discovered that the Numeric emulation functions in numarray doesn'=
t
accept a character typecode as type parameter.

This is not immediately apparent because type parameter is of type 'int',
and passing it a 'char' maybe not a good practice. But the fact is that
Numeric *do* accept the charcodes in the type parameter.=20

For example, this is the normal way to call the PyArray_FromDims function=
:

arr =3D PyArray_FromDims(self.rank, self.dimensions, tFloat64)

but, in Numeric, this other manner also works:

arr =3D PyArray_FromDims(self.rank, self.dimensions, 'd')

Now, in numarray, if you pass a character to the type parameter, a
"segmentation fault" is issued.

Look at the end of Numeric-22.0/Src/arraytypes.c, to see how characters a=
re
handled as types in Numeric. I think something like this should be added =
to
the deferred_libnumarray_init in numarray-0.4/Src/newarray.ch.

Another thing. It seems to me that NA_New and NA_Empty functions are not
well documented in the numarray documentation as they differ from the
definitions in numarray-0.4/Src/newarray.ch. I hope that the latter will
stay, because I prefer them a lot more than the documented ones :-)

Bye,

--=20
Francesc Alted

Re: [Numpy-discussion] Conversion functions between Numeric and numarray arrays?

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>I think this should be not too difficult to achieve and I'll try to explain
>why.
>
>When going from numarray to Numeric, numarray already have NA_InputArray
>C-API function that returns a well-behaved array. But strictly speaking, we
>don't even need a well-behaved array (this is a too restrictive condition)
>as both Numeric and numarray support discontiguous data. Even the byteorder
>should be not a problem, because, as Numeric itself has no such a property,
>we can create a Numeric array that is in native order as the result and
>byteswap the numarray object (if needed) before doing the conversion.
>
In-place byteswapping sounds like a bad idea to me.  What if the array 
is based upon a readonly buffer?  We've just started using these at 
STSCI because a readonly memory map imposes no load on the system swap 
file.  With a read only mapping,  the buffer itself has readonly pages; 
 these cannot be swapped in-place.

>So, non-alignment remains as the only issue that may cause a buffer copy
>during numarray ==> Numeric conversion. Is that correct?. 
>
I don't think so.

>If yes, it is
>possible to do a workaround about that, i.e. we can still get a Numeric from
>a numarray without copying the data in case of numarray misaligned objects?.
>  
>
I don't see how.  The primary source of misaligned arrays is numerical 
columns in recarrays.  It seems to me that if the data is misaligned, 
 you either have to copy it to someplace else which is aligned,  or 
teach the function which is going to process it how to access it 
byte-wise.  Only the former sounds feasible to me.

>Regarding to going in the other sense (ie. Numeric ==> numarray), as
>numarray supports discontiguity, misalignment and byteswapped data, this
>conversion should not imply a data buffer copy at all. 
>  
>
This sounds correct.  

>Once we have a pointer to the data buffer, it is only a matter of
>wrapping a Numeric or numarray object around it getting this info from the
>original object, and returning the new object as a result.
>
>All in all, this conversion *seems* to be not a too difficult task.
>  
>
It seems straightforward in principle,  but the memory management issues 
seem a little tricky to me.   It's easy to get buffers from numarrays, 
and create numarrays from buffers.  I guess we need a module which does 
the same for Numeric.  

There are two easy ways to "get a buffer" from a Numeric array:

1.  Wrap the Numeric data in a buffer object.
2.  Add support for the buffer API to the Numeric object.

Off hand,  I'm not sure which is better,  although (1) is less intrusive 
to Numeric and I suppose is the place to start.  This should be easy.

But,  I'm not sure how to create a Numeric array from a buffer.  It's 
easy to get the data pointer from a buffer, and to construct a Numeric 
array from a data pointer,   but we also need a way to stash the pointer 
to the buffer object.    I don't like the idea of modifying Numeric's 
PyArrayObject.  

>Making such a conversion functions (in C, but also having Python
>counterparts) available might represent to open the door to a co-existence
>of Numeric and numarray objects in the same program, and that would easy the
>numarray deployment in existing Numeric software.
>
>Comments?
>  
>
All in all,  I think this is a great idea which would really boost 
interoperability.  I wish there was a simpler approach which required no 
modifications to Numeric.

Todd 

Re: [Numpy-discussion] array operation

[Chris B. <Chr...@no...>]

Perry Greenfield wrote:

> If so then for numarray this ought to work.
> 
> index = nonzero(code==1) # want indices of all the obs where class code = 1
> selected_obs = data[index]

of for Numeric, use take():

selected_obs = take(data,nonzero(code == 1),1)

(this will select columns coresponding to where the code == 1, which is
how I read your question)


By the way, choose() and where() do something similar, but give you an
array back that is the saem size as the one you start with, with some
(or all) of the elements replaced. take() gives you a smaller array that
is a subset of the original one, which I think is what you want here.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

RE: [Numpy-discussion] array operation

[Perry G. <pe...@st...>]

Dave Bazell writes:
> I am trying to see if I can use where() or choose() to do this.  I can't
> really figure it out.
> 
> I have a 2-d array data where each row is an observation and each 
> column is
> an attribute of the observation:
> 
> data =
> [[.3, .2, 2.3,...]    <- observation 1
>  [.7, 1.2, .4...]     <- observation 2
> ...]]
> 
> I have another 1-d array that contains a code for the class of object:
> 
> class = [0,1,0,1,1,3,2,0,...]

Note that using class is illegal, it is a reserved keyword.
> 
> where class[i] = the class of the ith object in the data array.  Thus,
> observation 1 above is class 0, observation 2 is class 1, and so on.
> 
> I want to select all objects of a given class from data array.  I can do
> this with a loop
> 
I assume you mean you want to select all the rows corresponding to all
the observations where the code for the class corresponding to that
observation equals some particular value.

If so then for numarray this ought to work.

index = nonzero(code==1) # want indices of all the obs where class code = 1
selected_obs = data[index]

(or in one line if you wish: selected_obs = data[nonzero(code==1)]  )

> for i in range(ndat):
>     if class == 0:
>         do something
>    ....
> 
> Is there a way to use where() or choose() to do this?  Would it be more
> efficient?
> 
Perry