Re: [Pytables-users] (no subject)

[Francesc A. <fa...@py...>]

Hi David,

A Divendres 07 Maig 2004 06:02, David Sumner va escriure:
> Is it possible to use pytables in a parallel or threaded environment? I
> am looking at developing an application that will support live additions
> of data and at the same time allow data to be extracted for reporting
> purposes. I would not be concerned with locking because I see only one
> thread/process that would add data. There would be one or more
> threads/processes that would allow queries to be run against the dataset
> whose results would be returned for live reporting.

A couple of weeks ago or so, another person wrote to me about the same
question.

PyTables 0.8 has a bug that prevents the buffers to be flushed to disk when
a call to Leaf.flush() is made. This is solved in CVS now, and the patch
will appear in the next public release of pytables (0.8.1). So, in
principle, you should be able to have a process writing and other reading
the same file. However, some problem that I currently have tracked down to
the HDF5 library, makes the reading process to not be aware of the updates
in the file, unless you close and re-open the file in the reader, which
could be quite inconvenient.

I wrote about that problem to the NCSA people, and they asked to me to
provide a C example on that, but this would require some nice amount of work
I haven't had time yet, and I don't really know if I will have some in the
next future. If you want to contribute to this effort, you may provide this
C example (or buy some of my time in order to do it).

Regarding to the threaded environment, I know that HDF5 supports it (flag
--enable-threadsafe during configuration), but I did not made any try with
that and pytables. If you try this, I would be glad to hear about your
experiences.

Regards,

-- 
Francesc Alted