Re: [Pytables-users] concurrent accesses

[Francesc A. <fa...@gm...>]

On 11/2/12 5:19 PM, Ben Elliston wrote:
> On Fri, Nov 02, 2012 at 04:56:55PM -0400, Francesc Alted wrote:
>
>> Hmm, that's strange.  Using lzo or zlib works for you?
> Well, it seems that switching compression algorithms could be a
> nightmare (or can I do this with ptrepack?).

Yes, ptrepack can do that very easily.

>    However, I may have a
> workaround: I now open the HDF5 file with tables.openFile at the start
> of each process rather than inherit the file descriptor from the
> parent.  That works, since it's just concurrent file I/O on the same
> read-only file, and the start-up overhead is acceptable in this case.

Mmh, I think that makes sense.  I think the problem before was that you 
was sharing the same file description with different processes, and 
hence you ended with sync problems.  Having different descriptors for 
different processes is definitely the way to go.

>
> Happy to try lzo or zlib, though, if you like.

Provided the above, I don't think you need to (I mean, I'd say that lzo 
and zlib would have exactly the same problem).

-- 
Francesc Alted