Re: Re Re: [Pytables-users] Suggestions for storing mutlidimensionnal data in several pytables arrays

[Francesc A. <fa...@ca...>]

A Friday 09 September 2005 10:21, Cyril Giraudon va escriure:
> That seems a very good news for the simulation methods based upon
> structured grid  (i, j, k) mesh like finite differences in time domain.
> For instance in electromagnetism, people use to stock matrices
> X[i],
> Y[j],
> Z[k],
> Ex[i,j,k],
> Ey[i,j,k],
> Ez[i,j,k],
> Hx[i,j,k],
> Hy[i,j,k],
> Hz[i,j,k]
> independantly.
>
> Why ? Because during the computation, there is no need of X, Y, Z.

All right, but if you include such information on tables *and* use
compression, your datasets will not grow as much as you might think.
I'd recommend to have a try on this. You may get (positively)
surprised.

> In pytables, one can write these matrices but there is no standard way
> to say :
> The first dimension of Ex is linking to X, the second to Y and the third
> to Z.

Yes. I've done some research and found this:

http://hdf.ncsa.uiuc.edu/RFC/H5DimScales/

So, it seems that what you are asking for are Dimension Scales. Well,
the good news is that the HDF group is working on implementing this.
Once their job would be done, pytables can use them in a much more
portable way (i.e. compatible with all HDF5 apps).

> We could even imaging :
> The first dimension of  Ex is linking  to X[i1:i2:i3]  (begin, end,
> step) and so on...

In the design document on dimension scales above, you can go further
and assign several (not just one) scales to a dimension, share scales
among different dimensions (even on the same dataset), even define
functions as scales would be possible. I don't know how difficult
would be defining scale functions in compiled languages like C or
=46ortran, but given the interpreted nature of Python that would be
relatively easy.

> And that 's the point, linking matrix sides to some arrays description.
> "netcdf" allows such a feature but netcdf is usefullless with python,
> and i like pytables, "very good job".
> It's the way to do simulation with scripting languages.

Thanks for the compliments :)

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"