[Pytables-users] Big troubles reading an EArray

[Nicolas G. <nic...@ne...>]

Hi all,
I'm experiencing big troubles when trying to read data from a dataset using 
pytables.

The data was generated by a fortran program and stored using the HDF5 API into 
a dataset which has the following properties:

- nb of dimensions:  3
- dimension sizes: 10x1024x1025
- data type: 32-bit floating point
- chunking: 10x1024x1
- compression: shuffle: nbytes=4, gzip:level=1, allocation time: late

Let g be the group to which the dataset d belongs.
Also, let d2 be a supposed identical copy of d:

    In [182]:shape(d)
    Out[182]:(10L, 1024L, 1025L)
    
    In [183]:d2=d.read()
    
    In [185]:shape(d2)
    Out[185]:(10, 1024, 1025)
    
    In [186]:type(d)
    Out[186]:<class 'tables.EArray.EArray'>
    
    In [187]:type(d2)
    Out[187]:<class 'numarray.numarraycore.NumArray'>


If I want to access the 2 arrays element by element, everything seems to be 
ok:

    In [189]:d[1,0:5,0]
    Out[189]:array([ 2.32650805,  2.32443571,  2.3218658 ,  
                     2.31947947,  2.31709099], type=Float32)
    
    In [190]:d2[1,0:5,0]
    Out[190]:array([ 2.32650805,  2.32443571,  2.3218658 ,  
                     2.31947947,  2.31709099], type=Float32)

BUT the big trouble occurs when accessing several elements:

    In [205]:mean(d[1,:,0])
    Out[205]:1.2737342119216919
    
    In [206]:mean(d2[1,:,0])
    Out[206]:0.48158463835716248

Also when plotting the data using matplotlib, the results are absolutely 
different:

    In [202]:plot(d[1,0:5,0])
    Out[202]:[<matplotlib.lines.Line2D instance at 0x99d932c>]
    --> http://nicolasgirard.net2.nerim.net/img/plot_d.png
    
    In [203]:clf()
    
    In [204]:plot(d2[1,0:5,0])
    Out[204]:[<matplotlib.lines.Line2D instance at 0x991958c>]
    --> http://nicolasgirard.net2.nerim.net/img/plot_d2.png


I hope you'll have a clue on how to solve this problem, because this is quite 
blocking (and tense) to me and I don't know much what to do currently...

Thanks in advance for any advice,
cheers,
Nicolas