Re: [Pytables-users] Storing large images in PyTable

[Anthony S. <sc...@gm...>]

On Fri, Jul 5, 2013 at 8:40 AM, Francesc Alted <fa...@gm...> wrote:

> On 7/5/13 1:33 AM, Mathieu Dubois wrote:
> > tables.tableExtension.Table._createTable (tables/tableExtension.c:2181)
> >>
> >>     tables.exceptions.HDF5ExtError: Problems creating the table
> >>
> >>     I think that the size of the column is too large (if I remove the
> >>     Image
> >>     field, everything works perfectly).
> >>
> >>
> >> Hi Mathieu,
> >>
> >> This shouldn't be the case.  What is the value of IMAGE_SIZE?
> >
> > IMAGE_SIZE is a tuple containing (121, 145, 121).
>
> This is a bit large for a row in the Table object.  My recommendation
> for these cases is to use an associated EArray with shape (0, 121, 145,
> 121) and then append the images there.  You can always refer to the
> image by issuing a __getitem__() operation on the EArray object with the
> index of the row in the table.  Easy as a pie and you will allow the
> compression library (in case you are using compression) to work much
> more efficiently for the table.
>


Hi Francesc,

I disagree that this shape is too large for a table.  Here is a minimal
example that works for me:

import tables as tb
import numpy as np

images = np.ones(100, dtype=[('id', np.uint16),
                             ('image', np.float32, (121, 145, 121))
                             ])

with tb.open_file('temp.h5', 'w') as f:
    f.create_table('/', 'images', images)

I think that there is something else going on with the initialization but
Mathieu hasn't given us enough information to figure it out =/.  A minimal
failing script would be super helpful here!

(BTW Mathieu, Tables can also take advantage of compression.  Though
Francesc's solution is nicer for a lot of reason too.)

Be Well
Anthony


>
> HTH,
>
> -- Francesc Alted
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>