Re: [Pytables-users] Danger of file corruption? (was: Bug? flush() ignores new tables)

[Francesc A. <fa...@py...>]

A Dijous 30 Setembre 2004 10:27, Norbert Nemec va escriure:
> * Does pytables ever initiate writes except in explicit flush() and close() 
> commands?

Well, there are two levels of cache buffers: table buffers in PyTables and
HDF5 buffers. Now, PyTables triggers a write operation by calling an HDF5
function in two situations: 

1) When table buffers are full

2) In any other write operation (writing attributes or *Array elements)

then, is responsability of the underlying HDF5 library to write the data on
disk.

> * Are the elementary operations of the underlying hdf5 library atomic? I.e.: 
> If I shut down the program in the middle of a pytables write action, might 
> the resulting file be fundamentally corrupted at the hdf5 level, or would the 
> corruption be well localized and reparable?

I don't really know that.  I do know that HDF5 has a quite sophisticated
cache system (both for data and metadata), but I know nothing about
possible corruption scenarios and neither if they would be reparable or not.
That might be a good question for the HDF5 list. Do you want me to ask this?
Anyway, if you want to ask yourself, the HDF5 list is hd...@nc....

> All leading to the third question:
> 
> * Might there be some simple way to make a set of changes atomic? Even 
> if that means restricting the use of pytables?

By atomic you mean in a way that always let the file uncorrupted?. I just
don't know, it depends on how HDF5 deals with this situations, but a priori
my feeling is negative.

Cheers,

-- 
Francesc Alted