Re: [Pytables-users] corrupt hdf5 files?

[Andrew S. <str...@as...>]

Francesc Altet wrote:

>El dc 08 de 03 del 2006 a les 13:25 -0800, en/na Andrew Straw va
>escriure:
>  
>
>>Well, I've implemented an exit function using the atexit.register() 
>>function to properly close the file, at least where possible. I suspect 
>>this would have saved me from my rather careless action, but I don't 
>>plan on depending on this in the future -- I hope this will be a 
>>last-resort feature of my code rather than a heavily-depended upon 
>>feature. :)
>>
>>Anyhow, perhaps pytables could do something similar internally -- 
>>maintain a list of open files and register its own exit function with 
>>atexit.register() to make sure this sort of thing could be prevented in 
>>the future? It won't prevent against all unexpected program closes (e.g. 
>>signals) but my preliminary tests shows it does get called with ctrl-c. 
>>It should only be a few lines of code.
>>    
>>
>
>Right, in fact I thought about something like this some time ago, but
>never implemented it :-(
>
>Well, thanks for the suggestion! We will definitely look forward to
>include it in forthcoming 1.3 release.
>  
>
That sounds great -- I look forward to it.

On another note, it seems like part of the issue is that I was used to 
an older version of pytables (I think from 20050715 or so) in which the 
.h5 files seemed to be in an internally consistent state 99+% of the 
time, even while the app had them open. For example, I could make a copy 
of the still-growing .h5 file and open it with my analysis tools, and I 
never had the corrupt file issue. (Or do a simple h5ls on the file.) 
Now, however, with the 20060306 snapshot, I haven't once been able to do 
this. Can you think of anything different in pytables that may lead to 
this behavior? Is there anything I can do in my application like call 
h5file.sync() or something to attempt to force the file to be internally 
consistent?

BTW, I don't think it's my hdf5 implementation that has changed -- I 
haven't upgraded that. I'm using the standard 1.6.2 installed with 
debian sarge.

Cheers!
Andrew