Re: R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

Hola Antonio,

On Sunday 15 May 2005 11:37, you wrote:
> Hi Francesc,
> I have uploaded the patch that implements what we discussed in the
> previous posts on the patch section of the pytables project on
> SourceForge (pytables-20050514-mdattr.patch - file format_version >=
> 1.4).
>
> Some notes:
>
> - now all python objects are saved as cpickle strings and all numarray
>   objects are saved as HDF5 numeric attributes. The only exceptions
>   are the numeric sys attributes (NROWS, EXTDIM, AUTOMATIC_INDEX,
>   REINDEX, DIRTY, NODE_TYPE_VERSION) that are still stored as scalar
>   HDF5 attributes and retrieved as python int.
>   This because I found that in some cases they are set from the C
>   extension code:
>     H5ARRAY.c       H5ARRAYmake            EXTDIM
>     H5TB.c          H5TBdelete_record      NROWS
>     H5TB.c          H5TBmake_table         NROWS
>     H5TB-opt.c      H5TBOclose_append      NROWS
>   In my opinion this is not so bad in that they are treated as special
>   attributes in any case.


That's good. You had a small glitch converting NROWS to Python ints
instead of longs (remember that PyTables supports objects with more
than 2**31 rows). This has been corrected. If you would have run the
tests in heavy mode you should have been warned about that.

>
> - I run the complete test suite an all tests are passed (see the
>   attached testlog.txt). Unfortunately now come out some warning
>   that I am unable to fix, I'm sorry :((
>     Warning: Encountered invalid numeric result(s)  in greater_equal
>     Warning: Encountered invalid numeric result(s)  in less_equal
>     Warning: Encountered invalid numeric result(s)  in greater_equal
>     Warning: Encountered invalid numeric result(s)  in less
>     Warning: Encountered invalid numeric result(s)  in greater
>     Warning: Encountered invalid numeric result(s)  in less_equal
>     Warning: Encountered invalid numeric result(s)  in greater
>     Warning: Encountered invalid numeric result(s)  in less
>   I only know that they come from the test_Numeric.py test suite.
>

I'm not so sure that these warnings come from test_Numeric.py. I know
that these warnings appears from time to time when running the tests,
even before applying your patches. They seems unproblematic, though.

> - I noticed that, due to the rework of the attributes related code,
>   now there are many functions that are no more used:
>     # hdfExtension.pyx
>     AttributeSet._g_setAttrChar
>     AttributeSet._g_setAttrShort
>     AttributeSet._g_setAttrInt
>     AttributeSet._g_setAttrFloat
>     AttributeSet._g_setAttrDouble
>
>     #H5LT.h/H5LT.c
>     H5LTset_attribute_char
>     H5LTset_attribute_short
>     H5LTset_attribute_int
>     H5LTset_attribute_long
>
>     H5LTset_attribute_uchar
>     H5LTset_attribute_ushort
>     H5LTset_attribute_uint
>     H5LTset_attribute_ulong
>
>     H5LTset_attribute_float
>     H5LTset_attribute_double
>
>     H5LTget_attribute_char
>     H5LTget_attribute_short
>     H5LTget_attribute_long
>
>     H5LTget_attribute_uchar
>     H5LTget_attribute_ushort
>     H5LTget_attribute_uint
>     H5LTget_attribute_ulong
>
>     H5LTget_attribute_float
>     H5LTget_attribute_double
>
>   The H5LTget_attribute_int is still used in hdf5Extension.pyx from
>   the Array._openArray method.

Good. I should do some timmings in order to see how the changes affect
to opening times of complex PyTables files.

> Comments and suggestions are welcome.

Very good work indeed. Many thanks in name of the PyTables community!

The changes has been uploaded into the SVN repository.

Ciao,

Francesc