Re: [Pytables-users] Storing large images in PyTable

[Mathieu D. <dub...@ya...>]

Hi,

Sorry for the late response.

First of all, I have managed to achieve what I wanted to do differently.

Then the code Francesc send works well (I had to adapt it because I use 
version 2.3.1 under Ubuntu 12.04).

I was able to reproduce something similar with a class like this (copied 
& pasted from the tutorial):

import tables as tb

import numpy as np

class Subject(tb.IsDescription):

      # Subject information

      Id       = tb.UInt16Col()

      Image    = tb.Float32Col(shape=(121, 145, 121))

h5file = tb.openFile("tutorial1.h5", mode = "w", title = "Test file")

group = h5file.createGroup("/", 'subject', 'Suject information')

table = h5file.createTable(group, 'readout', Subject, "Readout example")

subject = table.row

for i in xrange(10):

      subject['Id'] = i

      subject['Image'] = np.ones((121, 145, 121))

      subject.append()

This code works well  too.

So I don't really know why nothing was working yesterday: this was the 
same class and a very close program. I will try to investigate later on 
this.

Thanks for everything,
Mahtieu

Le 05/07/2013 16:54, Anthony Scopatz a écrit :
>
>
>
> On Fri, Jul 5, 2013 at 8:40 AM, Francesc Alted <fa...@gm... 
> <mailto:fa...@gm...>> wrote:
>
>     On 7/5/13 1:33 AM, Mathieu Dubois wrote:
>     > tables.tableExtension.Table._createTable
>     (tables/tableExtension.c:2181)
>     >>
>     >>     tables.exceptions.HDF5ExtError: Problems creating the table
>     >>
>     >>     I think that the size of the column is too large (if I
>     remove the
>     >>     Image
>     >>     field, everything works perfectly).
>     >>
>     >>
>     >> Hi Mathieu,
>     >>
>     >> This shouldn't be the case.  What is the value of IMAGE_SIZE?
>     >
>     > IMAGE_SIZE is a tuple containing (121, 145, 121).
>
>     This is a bit large for a row in the Table object.  My recommendation
>     for these cases is to use an associated EArray with shape (0, 121,
>     145,
>     121) and then append the images there.  You can always refer to the
>     image by issuing a __getitem__() operation on the EArray object
>     with the
>     index of the row in the table.  Easy as a pie and you will allow the
>     compression library (in case you are using compression) to work much
>     more efficiently for the table.
>
>
>
> Hi Francesc,
>
> I disagree that this shape is too large for a table.  Here is a 
> minimal example that works for me:
>
> import tables as tb
> import numpy as np
>
> images = np.ones(100, dtype=[('id', np.uint16),
>            ('image', np.float32, (121, 145, 121))
>            ])
>
> with tb.open_file('temp.h5', 'w') as f:
> f.create_table('/', 'images', images)
>
> I think that there is something else going on with the initialization 
> but Mathieu hasn't given us enough information to figure it out =/.  A 
> minimal failing script would be super helpful here!
>
> (BTW Mathieu, Tables can also take advantage of compression.  Though 
> Francesc's solution is nicer for a lot of reason too.)
>
> Be Well
> Anthony
>
>
>     HTH,
>
>     -- Francesc Alted
>
>     ------------------------------------------------------------------------------
>     This SF.net email is sponsored by Windows:
>
>     Build for Windows Store.
>
>     http://p.sf.net/sfu/windows-dev2dev
>     _______________________________________________
>     Pytables-users mailing list
>     Pyt...@li...
>     <mailto:Pyt...@li...>
>     https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
>
>
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users