Re: [Numpy-discussion] Object arrays for numarray / What do you use Numeric object arrays for?

[Tim C. <tc...@op...>]

On Wed, 2003-07-16 at 05:34, Todd Miller wrote:
> I am adding arrays of Python objects to numarray and so I am curious
> about the uses people have found for Numeric's object arrays.  If you
> have found Numeric's object arrays useful,  please tell us about what
> you used them for so that we can make certain that numarray can satisfy
> the same need.

We use NumPy to store vectors (rank-1 arrays) of numbers representing
columns in a dataset. The NumPy arrays, which are large and numerous)=20
are memory-mapped (using an extension) to disc to conserve real memory.
However, in some vectors (columns) we need to store variable-length, and
in others, variable length sequences of integers or floats (and possibly
even sets in the future). NumPy's object arrays are more
memory-efficient that Python lists of lists or lists of strings from
these purposes, and of course they support NumPy functions such as
take(), which makes life simpler. But we haven't been able to memory-map
these object arrays, which is a problem. Is there any prospect of
numarray supporting memory-mapped arrays of sequences/strings? I know
that is a big ask! We have an extension module which stores variable
length blobs in a single memory-mapped file which might be useful - the
code could be made available to the numarray project, I think.

We also use MA extensively (because in the health care domain life is
full of missing data) - I'll jot down some thoughts on how MA could be
improved in the next few days.
--=20

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint =3D 8C22 BF76 33BA B3B5 1D5B  EB37 7891 46A9 EAF9 93D0