Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

Hola Antonio,

Sorry for the late response, but I was very busy lately (PyTables 1.0
was the responsible of that ;)

A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> Hi Francesc,
> the problem on scalar values seems to be fixed now.
> In the new patch is included your test suite with some changes.
> Let me know if it is good for you.

Thanks. It seems cleaner to me indeed. Unfortunately, the patch does
not apply well against PyTables 1.0. Could you make the necessary
modifications to this?

>
> Anyway there are still a couple of things, concerning the attributes
> handling, I don't like too much in this solution (pytables snapshot
> 20050504 + mdattr patch).
>
> - there is no way to set an HDF5 scalar attribute or an attribute with
>   shape (1,) having a data type different from H5T_NATIVE_INT or
>   H5T_NATIVE_DOUBLE

What about using H5T_ARRAY types for keeping multidimensional
attributes? That way you can distinguish between a regular
H5T_NATIVE_INT or H5T_NATIVE_DOUBLE and one with shape (1,). That
would be an elegant solution, IMO.

> - numeric HDF5 attributes with shape () or (1,) are always "imported"
>   as python int or python float (the original data type and sign are
>   lost)

You can prevent this from happening if you use H5T_ARRAY types.

> - scalar attributes are stored as HDF5 attributes with shape (1,)
>   and not as HDF5 attributes with shape ()

Again, H5T_ARRAY types can save the day. Then scalar attributes can
still be saved as regular attributes with shape (1,).

> I could try to implement a couple of additional methods for the
> AttributeSet class that could be used as workaround for the first
> two points:
>
> 	AttributeSet.setNAAttribute(self, name, value)
> 	AttributeSet.getNAAttribute(self, name)
>
> could be used in alternative to
>
> 	node._v_attrs.name =3D value
> 	value =3D node._v_attrs.name
>
> to force pytables to preserve datatype and shape into HDF5 numerical
> attributes. The mapping would be
>
> numarray.Int8      : H5T_NATIVE_CHAR
> numarray.Int16     : H5T_NATIVE_SHORT
> numarray.Int32     : H5T_NATIVE_INT
> numarray.Int64     : H5T_NATIVE_LLONG
> numarray.UInt8     : H5T_NATIVE_UCHAR
> numarray.UInt16    : H5T_NATIVE_USHORT
> numarray.UInt32    : H5T_NATIVE_UINT
> numarray.UInt64    : H5T_NATIVE_ULLONG
> numarray.Float32   : H5T_NATIVE_FLOAT
> numarray.Float64   : H5T_NATIVE_DOUBLE
>
> This is but an elegant solution in my opinion.

H5T_ARRAY would be more elegant.

> In alternative one could think to completely change the attribute
> handling mechanism and store all python objects including int and
> float as HDF5 strings using cpickle, and map only numarray objects
> on numerical HDF5 attributes.
> Of course this would break the backward compatibility but, in my
> opinion, will make the attribute handling mechanism more clear
> (also the relative pytables code will result a lot simpler).

Did I mentioned the H5T_ARRAY solution? ;)

> P.S. excuse me for my bad english.

Don't worry, mine is not any better actually!.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"