pytables-users

[Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Hi Francesc,
the problem on scalar values seems to be fixed now.
In the new patch is included your test suite with some changes.
Let me know if it is good for you.

Anyway there are still a couple of things, concerning the attributes
handling, I don't like too much in this solution (pytables snapshot
20050504 + mdattr patch).

- there is no way to set an HDF5 scalar attribute or an attribute with
  shape (1,) having a data type different from H5T_NATIVE_INT or
  H5T_NATIVE_DOUBLE
- numeric HDF5 attributes with shape () or (1,) are always "imported"
  as python int or python float (the original data type and sign are
  lost)
- scalar attributes are stored as HDF5 attributes with shape (1,)
  and not as HDF5 attributes with shape ()

I could try to implement a couple of additional methods for the
AttributeSet class that could be used as workaround for the first
two points:

	AttributeSet.setNAAttribute(self, name, value)
	AttributeSet.getNAAttribute(self, name)

could be used in alternative to=20

	node._v_attrs.name =3D value
	value =3D node._v_attrs.name

to force pytables to preserve datatype and shape into HDF5 numerical
attributes. The mapping would be

numarray.Int8      : H5T_NATIVE_CHAR
numarray.Int16     : H5T_NATIVE_SHORT
numarray.Int32     : H5T_NATIVE_INT
numarray.Int64     : H5T_NATIVE_LLONG
numarray.UInt8     : H5T_NATIVE_UCHAR
numarray.UInt16    : H5T_NATIVE_USHORT
numarray.UInt32    : H5T_NATIVE_UINT
numarray.UInt64    : H5T_NATIVE_ULLONG
numarray.Float32   : H5T_NATIVE_FLOAT
numarray.Float64   : H5T_NATIVE_DOUBLE

This is but an elegant solution in my opinion.
In alternative one could think to completely change the attribute
handling mechanism and store all python objects including int and
float as HDF5 strings using cpickle, and map only numarray objects
on numerical HDF5 attributes.
Of course this would break the backward compatibility but, in my
opinion, will make the attribute handling mechanism more clear
(also the relative pytables code will result a lot simpler).

I would like to know your (Francesc) opinion and the opinion of other
pytables users about that.

P.S. excuse me for my bad english.

Ciao
=20
Il giorno mar, 03-05-2005 alle 20:31 +0200, Francesc Altet ha scritto:
> Hi Antonio,
>=20
> First of all, thanks for your patch!
>=20
> I've tried the second version of the patch and seems to work quite
> well. There are, though, some issues with shape of scalar arrays and
> retrieved types that need to be addressed before commiting the changes
> to the repository.
>=20
> I'm attaching a test suite for attributes that exposes these problems.
> Run them in verbose mode to get more info on what is failing. If you
> can address those flaws that would be very nice!
>=20
> Cheers,
>=20
> A Divendres 29 Abril 2005 22:30, v=C3=A0reu escriure:
> > Patches item #1192700, was opened at 2005-04-29 22:30
> > Message generated for change (Tracker Item Submitted) made by Item
> > Submitter You can respond by visiting:
> > https://sourceforge.net/tracker/?func=3Ddetail&atid=3D504146&aid=3D1192=
700&group_
> >id=3D63486
> >
> > Category: None
> > Group: None
> > Status: Open
> > Resolution: None
> > Priority: 5
> > Submitted By: v_antonio (v_antonio)
> > Assigned to: Nobody/Anonymous (nobody)
> > Summary: Multi-dimensional HDF5 attributes
> >
> > Initial Comment:
> > I needed support to manage multidimensional hdf5
> > attributes so
> > I implemented a patch for pytables-20050429 snapshot that
> > adds this feature.
> > With this patch I'm able to store numarray objects (not
> > tuples or
> > lists) with arbitrary shape (according with hdf5
> > limitations) into
> > hdf5 attributes.
> > Datatype supported are:
> >
> > numarray.Int8      : H5T_NATIVE_CHAR,
> > numarray.Int16     : H5T_NATIVE_SHORT,
> > numarray.Int32     : H5T_NATIVE_INT,
> > numarray.Int64     : H5T_NATIVE_LLONG,
> > numarray.UInt8     : H5T_NATIVE_UCHAR,
> > numarray.UInt16    : H5T_NATIVE_USHORT,
> > numarray.UInt32    : H5T_NATIVE_UINT,
> > numarray.UInt64    : H5T_NATIVE_ULLONG,
> > numarray.Float32   : H5T_NATIVE_FLOAT,
> > numarray.Float64   : H5T_NATIVE_DOUBLE
> >
> > other datatypes are still stored into strings using
> > cpickle.
> > It is also possible to store mono-dimensional attributes
> > having more control over the datatype (e.g. one can
> > store H5T_NATIVE_UCHAR attributes).
> > Note: a H5T_NATIVE_UCHAR attribute is still retrieved
> > as python int.
> >
> >
> > Attached files:
> > pytables-20050429-mdattr.patch
> > test_mdattr.py (a small test program)
> >
> >
> > ----------------------------------------------------------------------
> >
> > You can respond by visiting:
> > https://sourceforge.net/tracker/?func=3Ddetail&atid=3D504146&aid=3D1192=
700&group_
> >id=3D63486

--=20
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

Hola Antonio,

Sorry for the late response, but I was very busy lately (PyTables 1.0
was the responsible of that ;)

A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> Hi Francesc,
> the problem on scalar values seems to be fixed now.
> In the new patch is included your test suite with some changes.
> Let me know if it is good for you.

Thanks. It seems cleaner to me indeed. Unfortunately, the patch does
not apply well against PyTables 1.0. Could you make the necessary
modifications to this?

>
> Anyway there are still a couple of things, concerning the attributes
> handling, I don't like too much in this solution (pytables snapshot
> 20050504 + mdattr patch).
>
> - there is no way to set an HDF5 scalar attribute or an attribute with
>   shape (1,) having a data type different from H5T_NATIVE_INT or
>   H5T_NATIVE_DOUBLE

What about using H5T_ARRAY types for keeping multidimensional
attributes? That way you can distinguish between a regular
H5T_NATIVE_INT or H5T_NATIVE_DOUBLE and one with shape (1,). That
would be an elegant solution, IMO.

> - numeric HDF5 attributes with shape () or (1,) are always "imported"
>   as python int or python float (the original data type and sign are
>   lost)

You can prevent this from happening if you use H5T_ARRAY types.

> - scalar attributes are stored as HDF5 attributes with shape (1,)
>   and not as HDF5 attributes with shape ()

Again, H5T_ARRAY types can save the day. Then scalar attributes can
still be saved as regular attributes with shape (1,).

> I could try to implement a couple of additional methods for the
> AttributeSet class that could be used as workaround for the first
> two points:
>
> 	AttributeSet.setNAAttribute(self, name, value)
> 	AttributeSet.getNAAttribute(self, name)
>
> could be used in alternative to
>
> 	node._v_attrs.name =3D value
> 	value =3D node._v_attrs.name
>
> to force pytables to preserve datatype and shape into HDF5 numerical
> attributes. The mapping would be
>
> numarray.Int8      : H5T_NATIVE_CHAR
> numarray.Int16     : H5T_NATIVE_SHORT
> numarray.Int32     : H5T_NATIVE_INT
> numarray.Int64     : H5T_NATIVE_LLONG
> numarray.UInt8     : H5T_NATIVE_UCHAR
> numarray.UInt16    : H5T_NATIVE_USHORT
> numarray.UInt32    : H5T_NATIVE_UINT
> numarray.UInt64    : H5T_NATIVE_ULLONG
> numarray.Float32   : H5T_NATIVE_FLOAT
> numarray.Float64   : H5T_NATIVE_DOUBLE
>
> This is but an elegant solution in my opinion.

H5T_ARRAY would be more elegant.

> In alternative one could think to completely change the attribute
> handling mechanism and store all python objects including int and
> float as HDF5 strings using cpickle, and map only numarray objects
> on numerical HDF5 attributes.
> Of course this would break the backward compatibility but, in my
> opinion, will make the attribute handling mechanism more clear
> (also the relative pytables code will result a lot simpler).

Did I mentioned the H5T_ARRAY solution? ;)

> P.S. excuse me for my bad english.

Don't worry, mine is not any better actually!.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Alle 14:08, marted=EC 10 maggio 2005, Francesc Altet ha scritto:
> Hola Antonio,

Hi

> Sorry for the late response, but I was very busy lately (PyTables 1.0
> was the responsible of that ;)
>
> A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> > Hi Francesc,
> > the problem on scalar values seems to be fixed now.
> > In the new patch is included your test suite with some changes.
> > Let me know if it is good for you.
>
> Thanks. It seems cleaner to me indeed. Unfortunately, the patch does
> not apply well against PyTables 1.0. Could you make the necessary
> modifications to this?

I don't have my laptop with me in this moment. I'll send you a pach again=
st=20
PyTables 1.0 as soon as I can.

> > Anyway there are still a couple of things, concerning the attributes
> > handling, I don't like too much in this solution (pytables snapshot
> > 20050504 + mdattr patch).
> >
> > - there is no way to set an HDF5 scalar attribute or an attribute wit=
h
> >   shape (1,) having a data type different from H5T_NATIVE_INT or
> >   H5T_NATIVE_DOUBLE
>
> What about using H5T_ARRAY types for keeping multidimensional
> attributes? That way you can distinguish between a regular
> H5T_NATIVE_INT or H5T_NATIVE_DOUBLE and one with shape (1,). That
> would be an elegant solution, IMO.

I never used H5T_ARRAY. It sounds good anyway :) I will think about..
Unfortunately this should be good solution for PyTables but not so good f=
or=20
me. :(
I'm working to a project related to COSMO SkyMED

http://www.aleniaspazio.it/earth_observation_page.aspx?IdProg=3D23

My problem is that i have to deal hdf5 data that have a fixed structure=20
(specified in a "product specification document"). For each node and leaf=
 it=20
is stated the number of attributes and their name, shape and datatype (an=
d=20
none of them is H5T_ARRAY :((( ).
With the current mdattr-patch (together with another pach for chunked lay=
out=20
datasets that I will submit soon) I can "import" this kind of data in the=
=20
PyTables format.
I still have some problems for the "export" operation.

I think that having to import/export ganeric HDF5 files is a situation no=
t so=20
unusual and I would like a lot that PyTables could have some mechanism to=
=20
allow that.
My doubt is that using exclusively H5T_ARRAY to manage multidimentional=20
attributes could make more difficult the managing of generic HDF5 files.
What is your opinion about?

> > - numeric HDF5 attributes with shape () or (1,) are always "imported"
> >   as python int or python float (the original data type and sign are
> >   lost)
>
> You can prevent this from happening if you use H5T_ARRAY types.
>
> > - scalar attributes are stored as HDF5 attributes with shape (1,)
> >   and not as HDF5 attributes with shape ()
>
> Again, H5T_ARRAY types can save the day. Then scalar attributes can
> still be saved as regular attributes with shape (1,).
>
> > I could try to implement a couple of additional methods for the
> > AttributeSet class that could be used as workaround for the first
> > two points:
> >
> > =09AttributeSet.setNAAttribute(self, name, value)
> > =09AttributeSet.getNAAttribute(self, name)
> >
> > could be used in alternative to
> >
> > =09node._v_attrs.name =3D value
> > =09value =3D node._v_attrs.name
> >
> > to force pytables to preserve datatype and shape into HDF5 numerical
> > attributes. The mapping would be
> >
> > numarray.Int8      : H5T_NATIVE_CHAR
> > numarray.Int16     : H5T_NATIVE_SHORT
> > numarray.Int32     : H5T_NATIVE_INT
> > numarray.Int64     : H5T_NATIVE_LLONG
> > numarray.UInt8     : H5T_NATIVE_UCHAR
> > numarray.UInt16    : H5T_NATIVE_USHORT
> > numarray.UInt32    : H5T_NATIVE_UINT
> > numarray.UInt64    : H5T_NATIVE_ULLONG
> > numarray.Float32   : H5T_NATIVE_FLOAT
> > numarray.Float64   : H5T_NATIVE_DOUBLE
> >
> > This is but an elegant solution in my opinion.
>
> H5T_ARRAY would be more elegant.
>
> > In alternative one could think to completely change the attribute
> > handling mechanism and store all python objects including int and
> > float as HDF5 strings using cpickle, and map only numarray objects
> > on numerical HDF5 attributes.
> > Of course this would break the backward compatibility but, in my
> > opinion, will make the attribute handling mechanism more clear
> > (also the relative pytables code will result a lot simpler).
>
> Did I mentioned the H5T_ARRAY solution? ;)
>
> > P.S. excuse me for my bad english.
>
> Don't worry, mine is not any better actually!.
>
> Cheers,

ciao

--=20
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Il giorno mar, 10-05-2005 alle 14:08 +0200, Francesc Altet ha scritto:
> Hola Antonio,
> 
> Sorry for the late response, but I was very busy lately (PyTables 1.0
> was the responsible of that ;)
> 
> A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> > Hi Francesc,
> > the problem on scalar values seems to be fixed now.
> > In the new patch is included your test suite with some changes.
> > Let me know if it is good for you.
> 
> Thanks. It seems cleaner to me indeed. Unfortunately, the patch does
> not apply well against PyTables 1.0. Could you make the necessary
> modifications to this?
> 

Hi Francesc,
I have just uploaded the mdattr patch against PyTables 1.0.

I also submitted a new patch that should provide a whider support for 
chunked layout datasets.
Please take a look and let me know what do you think about.

ciao

[...]


> 
> Cheers,

-- 
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

Antonio,

I've uploaded your patch for dealing multidimensional attributes in
the SVN repository. I've modified it a bit so that:

=2D When you save python scalars, they become saved as HDF5 scalars
(before, they were saved as arrays of shape (1,)), which is wrong IMO.

=2D When you save numarray scalars, they become saved as HDF5 scalars as
well. I'm hesitating here because I wonder myself whether saving them
as a pickle would be better. I really prefer the implemented solution
but, what do you think about?

=2D For keeping backward compatibility, I added code so that for old
PyTables files so that an HDF5 attribute with shape (1,) will be
mapped to a Python scalar.

With that I think all the requirements for multidimensional attributes
are fulfilled (except perhaps how to serialize numarray scalars on
disk).

Wait until a snapshot will be made this midnight in the carabos
repository of pytables in order to have a try at final code.

I'll look into your second patch when I have more time (I'm a bit busy
lately :-/). Thanks for it anyway!

Ciao,

=46rancesc

A Dimarts 10 Maig 2005 23:55, Antonio Valentino va escriure:
> Il giorno mar, 10-05-2005 alle 14:08 +0200, Francesc Altet ha scritto:
> > Hola Antonio,
> >
> > Sorry for the late response, but I was very busy lately (PyTables 1.0
> > was the responsible of that ;)
> >
> > A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> > > Hi Francesc,
> > > the problem on scalar values seems to be fixed now.
> > > In the new patch is included your test suite with some changes.
> > > Let me know if it is good for you.
> >
> > Thanks. It seems cleaner to me indeed. Unfortunately, the patch does
> > not apply well against PyTables 1.0. Could you make the necessary
> > modifications to this?
>
> Hi Francesc,
> I have just uploaded the mdattr patch against PyTables 1.0.
>
> I also submitted a new patch that should provide a whider support for
> chunked layout datasets.
> Please take a look and let me know what do you think about.
>
> ciao
>
> [...]
>
> > Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Alle 15:10, mercoled=EC 11 maggio 2005, Francesc Altet ha scritto:
> Antonio,
>
> I've uploaded your patch for dealing multidimensional attributes in
> the SVN repository. I've modified it a bit so that:
>
> - When you save python scalars, they become saved as HDF5 scalars
> (before, they were saved as arrays of shape (1,)), which is wrong IMO.

I agree!
I didn't modify that because of back compatibility reasons but I see that=
 you=20
solved this point..

> - When you save numarray scalars, they become saved as HDF5 scalars as
> well. I'm hesitating here because I wonder myself whether saving them
> as a pickle would be better. I really prefer the implemented solution
> but, what do you think about?

In my opinion saving numarray scalars as HDF5 scalars is the best choice.
The problem is that when you get it back from file you always have a=20
Python_int or Python _float.
Attributes can't have a "Flavor"
That is why I suggested to map HDF5 numeric attributes onto numarray obje=
cts=20
and handle all python objects using cpickle strings.
This is absolutely unambiguous and allow the user to have the full contro=
l=20
over HDF5 attributes.

> - For keeping backward compatibility, I added code so that for old
> PyTables files so that an HDF5 attribute with shape (1,) will be
> mapped to a Python scalar.

At the moment you introduce extra code for keeping backward compatibility=
 you=20
can add still a little more an have and attribute handling system complet=
e=20
and clear.
Ofcourse whatever solution you decide to adopt I will be happy to help yo=
u.
Let me know your decisions.

> With that I think all the requirements for multidimensional attributes
> are fulfilled (except perhaps how to serialize numarray scalars on
> disk).
>
> Wait until a snapshot will be made this midnight in the carabos
> repository of pytables in order to have a try at final code.
>
> I'll look into your second patch when I have more time (I'm a bit busy
> lately :-/). Thanks for it anyway!
>
> Ciao,
>
> Francesc

ciao

> A Dimarts 10 Maig 2005 23:55, Antonio Valentino va escriure:
> > Il giorno mar, 10-05-2005 alle 14:08 +0200, Francesc Altet ha scritto=
:
> > > Hola Antonio,
> > >
> > > Sorry for the late response, but I was very busy lately (PyTables 1=
=2E0
> > > was the responsible of that ;)
> > >
> > > A Dijous 05 Maig 2005 22:37, Antonio Valentino va escriure:
> > > > Hi Francesc,
> > > > the problem on scalar values seems to be fixed now.
> > > > In the new patch is included your test suite with some changes.
> > > > Let me know if it is good for you.
> > >
> > > Thanks. It seems cleaner to me indeed. Unfortunately, the patch doe=
s
> > > not apply well against PyTables 1.0. Could you make the necessary
> > > modifications to this?
> >
> > Hi Francesc,
> > I have just uploaded the mdattr patch against PyTables 1.0.
> >
> > I also submitted a new patch that should provide a whider support for
> > chunked layout datasets.
> > Please take a look and let me know what do you think about.
> >
> > ciao
> >
> > [...]
> >
> > > Cheers,

--=20
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy

Re: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

A Dimecres 11 Maig 2005 16:09, Antonio Valentino va escriure:
> > - When you save numarray scalars, they become saved as HDF5 scalars as
> > well. I'm hesitating here because I wonder myself whether saving them
> > as a pickle would be better. I really prefer the implemented solution
> > but, what do you think about?
>
> In my opinion saving numarray scalars as HDF5 scalars is the best choice.
> The problem is that when you get it back from file you always have a
> Python_int or Python _float.
> Attributes can't have a "Flavor"
> That is why I suggested to map HDF5 numeric attributes onto numarray
> objects and handle all python objects using cpickle strings.
> This is absolutely unambiguous and allow the user to have the full control
> over HDF5 attributes.

Yeah, may be you are right. So, your proposal is to pickle numarray
scalars, so that they do not interfere with Python scalars
deserialization, isn't it?  I'd like to hear more opinions on that
matter.

> Ofcourse whatever solution you decide to adopt I will be happy to help yo=
u.
> Let me know your decisions.

Well, if you are willing to manage the case for scalar numarray
pickling, you are very welcome!

Ad=E9u,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Valentino A. <val...@co...>]

> -----Messaggio originale-----
> Da: Francesc Altet [mailto:fa...@ca...]
> Inviato: mercoledì 11 maggio 2005 18.42
> A: Antonio Valentino
> Cc: pyt...@li...
> Oggetto: Re: [Pytables-users] Re: [ pytables-Patches-1192700 ]
> Multi-dimensional HDF5 attributes
>
>
> A Dimecres 11 Maig 2005 16:09, Antonio Valentino va escriure:
> > > - When you save numarray scalars, they become saved as HDF5 scalars as
> > > well. I'm hesitating here because I wonder myself whether saving them
> > > as a pickle would be better. I really prefer the implemented solution
> > > but, what do you think about?
> >
> > In my opinion saving numarray scalars as HDF5 scalars is the
> best choice.
> > The problem is that when you get it back from file you always have a
> > Python_int or Python _float.
> > Attributes can't have a "Flavor"
> > That is why I suggested to map HDF5 numeric attributes onto numarray
> > objects and handle all python objects using cpickle strings.
> > This is absolutely unambiguous and allow the user to have the
> full control
> > over HDF5 attributes.
>
> Yeah, may be you are right. So, your proposal is to pickle numarray
> scalars, so that they do not interfere with Python scalars
> deserialization, isn't it?  I'd like to hear more opinions on that
> matter.

No. The contrary. In my opinion, python integers and float should be
pickled.
Numarray objects, including scalars, should be saved as numeric HDF5
attributes.
In this way numeric HDF5 attributes will be *always* loaded into pytables as
numarray objects, and numarray objects will be *always* saved on file as
numeric
HDF5 attributes (H5T_NATIVE_INT, H5T_NATIVE_FLOAT and so on).
Python objects, including int, float, complex, decimal.Decimal, list, tuple,
dict, etc, should be saved as cpickle strings in order to not have
ambiguities with
numeric attributes.
In this way you can write code like this

leaf.attrs.uchar = na.array(1, type=na.UInt8)   # --> H5T_NATIVE_UCHAR
integet = 1
leaf.attrs.int = integer   # --> cpickle string
leaf.attrs.int32 = na.asarray(integer)  # --> H5T_NATIVE_INT

and then retrieve them back unambiguously.
Stated that extra code is needed in order to preserve the compatibility with
old pytables version, I don't know if this approach can araise other
problems.
I hope I have not been too much confusing.

> > Ofcourse whatever solution you decide to adopt I will be happy
> to help you.
> > Let me know your decisions.
>
> Well, if you are willing to manage the case for scalar numarray
> pickling, you are very welcome!
>
> Adéu,
>
> --
> >0,0<   Francesc Altet     http://www.carabos.com/
> V   V   Cárabos Coop. V.   Enjoy Data
>  "-"

ciao
Antonio Valentino

Re: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

A Dijous 12 Maig 2005 09:16, v=E0reu escriure:
> No. The contrary. In my opinion, python integers and float should be
> pickled.
> Numarray objects, including scalars, should be saved as numeric HDF5
> attributes.
> In this way numeric HDF5 attributes will be *always* loaded into pytables
> as numarray objects, and numarray objects will be *always* saved on file =
as
> numeric
> HDF5 attributes (H5T_NATIVE_INT, H5T_NATIVE_FLOAT and so on).
> Python objects, including int, float, complex, decimal.Decimal, list,
> tuple, dict, etc, should be saved as cpickle strings in order to not have
> ambiguities with
> numeric attributes.
> In this way you can write code like this
>
> leaf.attrs.uchar =3D na.array(1, type=3Dna.UInt8)   # --> H5T_NATIVE_UCHAR
> integet =3D 1
> leaf.attrs.int =3D integer   # --> cpickle string
> leaf.attrs.int32 =3D na.asarray(integer)  # --> H5T_NATIVE_INT
>
> and then retrieve them back unambiguously.
> Stated that extra code is needed in order to preserve the compatibility
> with old pytables version, I don't know if this approach can araise other
> problems.
> I hope I have not been too much confusing.

No, I understand clearly now. Well, after discussing this issue with
Ivan, we think that your suggestion is very consistent. And, as we can
distinguish between files made with PyTables 1.0 and earlier from
those made with PyTables 1.1 and after, I think there will be no
problem in implementing that.

May you provide the necessary patch based on the latest snapshot? If
you can't, I can do it my self.

Thanks,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Valentino A. <val...@co...>]

> -----Messaggio originale-----
> Da: Francesc Altet [mailto:fa...@ca...]
> Inviato: giovedì 12 maggio 2005 11.05
> A: Valentino Antonio
> Cc: pyt...@li...
> Oggetto: Re: R: [Pytables-users] Re: [ pytables-Patches-1192700 ]
> Multi-dimensional HDF5 attributes
>
>
> A Dijous 12 Maig 2005 09:16, vàreu escriure:
> > No. The contrary. In my opinion, python integers and float should be
> > pickled.
> > Numarray objects, including scalars, should be saved as numeric HDF5
> > attributes.
> > In this way numeric HDF5 attributes will be *always* loaded
> into pytables
> > as numarray objects, and numarray objects will be *always*
> saved on file as
> > numeric
> > HDF5 attributes (H5T_NATIVE_INT, H5T_NATIVE_FLOAT and so on).
> > Python objects, including int, float, complex, decimal.Decimal, list,
> > tuple, dict, etc, should be saved as cpickle strings in order
> to not have
> > ambiguities with
> > numeric attributes.
> > In this way you can write code like this
> >
> > leaf.attrs.uchar = na.array(1, type=na.UInt8)   # --> H5T_NATIVE_UCHAR
> > integet = 1
> > leaf.attrs.int = integer   # --> cpickle string
> > leaf.attrs.int32 = na.asarray(integer)  # --> H5T_NATIVE_INT
> >
> > and then retrieve them back unambiguously.
> > Stated that extra code is needed in order to preserve the compatibility
> > with old pytables version, I don't know if this approach can
> araise other
> > problems.
> > I hope I have not been too much confusing.
>
> No, I understand clearly now.

:)))

> Well, after discussing this issue with
> Ivan, we think that your suggestion is very consistent. And, as we can
> distinguish between files made with PyTables 1.0 and earlier from
> those made with PyTables 1.1 and after, I think there will be no
> problem in implementing that.

I'm happy to hear this.

> May you provide the necessary patch based on the latest snapshot? If
> you can't, I can do it my self.

Of course I can.
I will make it available this weekend (I hope).

> Thanks,
>
> --
> >0,0<   Francesc Altet     http://www.carabos.com/
> V   V   Cárabos Coop. V.   Enjoy Data
>  "-"
>

ciao

--
Antonio Valentino

Re: R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

Hola Antonio,

On Sunday 15 May 2005 11:37, you wrote:
> Hi Francesc,
> I have uploaded the patch that implements what we discussed in the
> previous posts on the patch section of the pytables project on
> SourceForge (pytables-20050514-mdattr.patch - file format_version >=
> 1.4).
>
> Some notes:
>
> - now all python objects are saved as cpickle strings and all numarray
>   objects are saved as HDF5 numeric attributes. The only exceptions
>   are the numeric sys attributes (NROWS, EXTDIM, AUTOMATIC_INDEX,
>   REINDEX, DIRTY, NODE_TYPE_VERSION) that are still stored as scalar
>   HDF5 attributes and retrieved as python int.
>   This because I found that in some cases they are set from the C
>   extension code:
>     H5ARRAY.c       H5ARRAYmake            EXTDIM
>     H5TB.c          H5TBdelete_record      NROWS
>     H5TB.c          H5TBmake_table         NROWS
>     H5TB-opt.c      H5TBOclose_append      NROWS
>   In my opinion this is not so bad in that they are treated as special
>   attributes in any case.


That's good. You had a small glitch converting NROWS to Python ints
instead of longs (remember that PyTables supports objects with more
than 2**31 rows). This has been corrected. If you would have run the
tests in heavy mode you should have been warned about that.

>
> - I run the complete test suite an all tests are passed (see the
>   attached testlog.txt). Unfortunately now come out some warning
>   that I am unable to fix, I'm sorry :((
>     Warning: Encountered invalid numeric result(s)  in greater_equal
>     Warning: Encountered invalid numeric result(s)  in less_equal
>     Warning: Encountered invalid numeric result(s)  in greater_equal
>     Warning: Encountered invalid numeric result(s)  in less
>     Warning: Encountered invalid numeric result(s)  in greater
>     Warning: Encountered invalid numeric result(s)  in less_equal
>     Warning: Encountered invalid numeric result(s)  in greater
>     Warning: Encountered invalid numeric result(s)  in less
>   I only know that they come from the test_Numeric.py test suite.
>

I'm not so sure that these warnings come from test_Numeric.py. I know
that these warnings appears from time to time when running the tests,
even before applying your patches. They seems unproblematic, though.

> - I noticed that, due to the rework of the attributes related code,
>   now there are many functions that are no more used:
>     # hdfExtension.pyx
>     AttributeSet._g_setAttrChar
>     AttributeSet._g_setAttrShort
>     AttributeSet._g_setAttrInt
>     AttributeSet._g_setAttrFloat
>     AttributeSet._g_setAttrDouble
>
>     #H5LT.h/H5LT.c
>     H5LTset_attribute_char
>     H5LTset_attribute_short
>     H5LTset_attribute_int
>     H5LTset_attribute_long
>
>     H5LTset_attribute_uchar
>     H5LTset_attribute_ushort
>     H5LTset_attribute_uint
>     H5LTset_attribute_ulong
>
>     H5LTset_attribute_float
>     H5LTset_attribute_double
>
>     H5LTget_attribute_char
>     H5LTget_attribute_short
>     H5LTget_attribute_long
>
>     H5LTget_attribute_uchar
>     H5LTget_attribute_ushort
>     H5LTget_attribute_uint
>     H5LTget_attribute_ulong
>
>     H5LTget_attribute_float
>     H5LTget_attribute_double
>
>   The H5LTget_attribute_int is still used in hdf5Extension.pyx from
>   the Array._openArray method.

Good. I should do some timmings in order to see how the changes affect
to opening times of complex PyTables files.

> Comments and suggestions are welcome.

Very good work indeed. Many thanks in name of the PyTables community!

The changes has been uploaded into the SVN repository.

Ciao,

Francesc

Re: R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Alle 13:33, domenica 15 maggio 2005, Francesc Altet ha scritto:
> Hola Antonio,
>
> On Sunday 15 May 2005 11:37, you wrote:
> > Hi Francesc,
> > I have uploaded the patch that implements what we discussed in the
> > previous posts on the patch section of the pytables project on
> > SourceForge (pytables-20050514-mdattr.patch - file format_version >=3D
> > 1.4).
> >
> > Some notes:
> >
> > - now all python objects are saved as cpickle strings and all numarra=
y
> >   objects are saved as HDF5 numeric attributes. The only exceptions
> >   are the numeric sys attributes (NROWS, EXTDIM, AUTOMATIC_INDEX,
> >   REINDEX, DIRTY, NODE_TYPE_VERSION) that are still stored as scalar
> >   HDF5 attributes and retrieved as python int.
> >   This because I found that in some cases they are set from the C
> >   extension code:
> >     H5ARRAY.c       H5ARRAYmake            EXTDIM
> >     H5TB.c          H5TBdelete_record      NROWS
> >     H5TB.c          H5TBmake_table         NROWS
> >     H5TB-opt.c      H5TBOclose_append      NROWS
> >   In my opinion this is not so bad in that they are treated as specia=
l
> >   attributes in any case.
>
> That's good. You had a small glitch converting NROWS to Python ints
> instead of longs (remember that PyTables supports objects with more

I'm sorry.

> than 2**31 rows). This has been corrected. If you would have run the
> tests in heavy mode you should have been warned about that.

I have run the the test in heavy mode today on the PC at work (not my lap=
top)
and got 3 errors for the snapshot-20060516

[...]

Reading an old version file node. ...=20
/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py:248:=20
UserWarning: problems loading leaf ``/test``: unsubscriptable object; it =
will=20
become an ``UnImplemented`` node
  warnings.warn("""\
ERROR
Writing an old version file node. ... ERROR
Accessing attributes in an old version file node. ... ERROR

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Reading an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Writing an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Accessing attributes in an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

----------------------------------------------------------------------
Ran 2456 tests in 781.418s

FAILED (errors=3D3)
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
PyTables version:  1.1-alpha
Extension version: $Id: hdf5Extension.pyx 852 2005-05-15 11:31:50Z faltet=
 $
HDF5 version:      1.6.4
numarray version:  1.3.1
Zlib version:      1.1.4
BZIP2 version:     1.0.2 (30-Dec-2001)
Python version:    2.4.1 (#1, May  4 2005, 11:47:31)
[GCC 3.2.2]
Platform:          linux2-i686
Byte-ordering:     little
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
Performing the complete test suite!
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
Skipping Numeric test suite
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D



> > - I run the complete test suite an all tests are passed (see the
> >   attached testlog.txt). Unfortunately now come out some warning
> >   that I am unable to fix, I'm sorry :((
> >     Warning: Encountered invalid numeric result(s)  in greater_equal
> >     Warning: Encountered invalid numeric result(s)  in less_equal
> >     Warning: Encountered invalid numeric result(s)  in greater_equal
> >     Warning: Encountered invalid numeric result(s)  in less
> >     Warning: Encountered invalid numeric result(s)  in greater
> >     Warning: Encountered invalid numeric result(s)  in less_equal
> >     Warning: Encountered invalid numeric result(s)  in greater
> >     Warning: Encountered invalid numeric result(s)  in less
> >   I only know that they come from the test_Numeric.py test suite.
>
> I'm not so sure that these warnings come from test_Numeric.py. I know
> that these warnings appears from time to time when running the tests,
> even before applying your patches. They seems unproblematic, though.

OK

[...]

> > Comments and suggestions are welcome.
>
> Very good work indeed. Many thanks in name of the PyTables community!

Thank you for pytables

> The changes has been uploaded into the SVN repository.
>
> Ciao,
>
> Francesc

ciao

--=20
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy
Tel.:      +39 0835 309180
Fax:       +39 0835 264705
Home Page: www.consorzio-innova.it
Email:     val...@co...

Re: R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Antonio V. <val...@co...>]

Alle 16:28, luned=EC 16 maggio 2005, Antonio Valentino ha scritto:
> Alle 13:33, domenica 15 maggio 2005, Francesc Altet ha scritto:
> > Hola Antonio,
> >

[...]

> > That's good. You had a small glitch converting NROWS to Python ints
> > instead of longs (remember that PyTables supports objects with more
>
> I'm sorry.
>
> > than 2**31 rows). This has been corrected. If you would have run the
> > tests in heavy mode you should have been warned about that.
>
> I have run the the test in heavy mode today on the PC at work (not my
> laptop) and got 3 errors for the snapshot-20060516

[...]

ops. This also happens in *non* heavy mode.

[...]
Reading an old version file node. ...=20
/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py:248:=20
UserWarning: problems loading leaf ``/test``: unsubscriptable object; it =
will=20
become an ``UnImplemented`` node
  warnings.warn("""\
ERROR
Writing an old version file node. ... ERROR
Accessing attributes in an old version file node. ... ERROR

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Reading an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Writing an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: Accessing attributes in an old version file node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valentino/pyvm/src/pytables-20050516/test/test_filenode.py"=
,=20
line 909, in setUp
    oldh5f =3D tables.openFile(self.oldh5fname)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
229, in openFile
    return File(path, mode, title, new, trMap, rootUEP, isPTFile, filters=
)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
409, in __init__
    self.root =3D self.__getRootGroup(rootUEP)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/File.py",=
 line=20
460, in __getRootGroup
    rootGroup =3D RootGroup(self, '/', rootUEP, new=3Dself._v_new)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 831, in __init__
    self._g_openFile()
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 253, in _g_openFile
    objleaf._g_putUnder(objgroup, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Leaf.py",=
 line=20
222, in _g_putUnder
    super(Leaf, self)._g_putUnder(parent, name)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Node.py",=
 line=20
693, in _g_putUnder
    parent._g_refNode(self, ptname, validate)
  File "/home/valentino/pyvm/lib/python2.4/site-packages/tables/Group.py"=
,=20
line 323, in _g_refNode
    raise NodeError(
NodeError: group ``/`` already has a child node named ``test``

----------------------------------------------------------------------
Ran 1798 tests in 206.001s

FAILED (errors=3D3)
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
PyTables version:  1.1-alpha
Extension version: $Id: hdf5Extension.pyx 852 2005-05-15 11:31:50Z faltet=
 $
HDF5 version:      1.6.4
numarray version:  1.3.1
Zlib version:      1.1.4
BZIP2 version:     1.0.2 (30-Dec-2001)
Python version:    2.4.1 (#1, May  4 2005, 11:47:31)
[GCC 3.2.2]
Platform:          linux2-i686
Byte-ordering:     little
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
Performing only a light (yet comprehensive) subset of the test
suite.  If you have a big system and lots of CPU to waste and want to
do a more complete test, try passing the --heavy flag to this script.
The whole suite will take more than 10 minutes to complete on a
relatively modern CPU and around 100 MB of main memory.
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D
Skipping Numeric test suite
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D


I will try on my laptop as soon as possible.

[...]

 ciao

--=20
Antonio Valentino
INNOVA - Consorzio per l'Informatica e la Telematica
via della Scienza - Zona Paip I
75100 Matera (MT) Italy
Tel.:      +39 0835 309180
Fax:       +39 0835 264705
Home Page: www.consorzio-innova.it
Email:     val...@co...

Re: R: R: [Pytables-users] Re: [ pytables-Patches-1192700 ] Multi-dimensional HDF5 attributes

[Francesc A. <fa...@ca...>]

A Dilluns 16 Maig 2005 16:41, Antonio Valentino va escriure:
> > I have run the the test in heavy mode today on the PC at work (not my
> > laptop) and got 3 errors for the snapshot-20060516
>
> [...]
>
> ops. This also happens in *non* heavy mode.

Well, I've commited a patch to solve this. I'm attaching it in case
you are interested in looking at it.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"