Re: [Pytables-users] Switch off implicit flushing?

[Francesc A. <fa...@ca...>]

Hi Norbert,

A Dimarts 14 Desembre 2004 19:04, Norbert Nemec va escriure:
> I understand, that pytables buffers data until flush() is called. I also=
=20
> understand, that this buffer may also be flushed automatically when it is=
=20
> full.
>=20
> Would it be possible to prohibit this automatic flushing?

No, the flushing mechanism works exactly as you have described.

> The idea behind this is, that I would like to have the h5-file intact at =
least=20
> with a good probability if the program is interrupted or breaks down. If =
it=20
> would be possible to defer all writing actions to an explicit call of flu=
sh()=20
> (or the closing of the file), I could simply write to pytables in some ra=
ndom=20
> order, and then just call flush() when the data is in a coherent state. E=
ven=20
> though this will not give us real atomic operations, it would still be go=
od=20
> enough for many applications that continuously write to a file.
>=20
> For performance reasons, this should probably be an option.

Well, if you are interested in providing your own buffers to Table objects
you can always create a RecArray (or a list of Numeric/NumArray/CharArray)
objects on your own, fill it with your info and then call
Table.append(recarray) for writing it to disk. Table.append() forces an
immediate write to disk, and don't touch the internal table buffers at all.
This would be a nice approach for doing what you are interested in.

> It should also be clear which operations are buffered at all.

Normally, the only buffered operations in PyTables are those related with
the extension class Row (Row.append(), Row.__getitem__() and
Row.__setitem__()). All the rest does not use buffering at all (unless you
write big chunks of data on your own as described above).

=2D-=20
=46rancesc Altet
Who's your data daddy? =A0PyTables