Re: [Pytables-users] Problem flushing table with pytables 1.2

[Alain F. <ala...@fr...>]

Hi Francesc,

thanks for your quick answer.
unfortunatly due do software architecture I can not apply first method.
User can create row from highlevel API and then decide to flush after a=20
number of row created.

I so applied the patch you proposed, software runs ok now, but is slown=20
down.
It seems that the patch effect is that flush is automatically done after=20
each row creation (or something like this).
Ex. Create 100 000 rows and flush each 10 000.
 - runs in 50 seconds with pytables 1.1.1
 - runs in 1055 seconds with patch in pytables 1.2

For the time being I switch back to pytables 1.1.1, and will follow up th=
e=20
evolution of pytables 1.2.

Thanks for your help,

Best Regards

----- Original Message -----=20
From: "Francesc Altet" <fa...@ca...>
To: <pyt...@li...>
Cc: "Alain Fagot" <ala...@fr...>; "Waldemar Osuch"=20
<wal...@gm...>
Sent: Tuesday, November 29, 2005 8:55 PM
Subject: Re: [Pytables-users] Problem flushing table with pytables 1.2


Hello,

Yes, I was aware of this problem shortly after releasing 1.2 :-(
Fortunately there is an easy workaround, which is flushing the table
immediately after ending the append loop:

for i range(...):
   ....
   row.append()
table.flush()   # Add this after append loops

Alternatively, you can apply the next patch:

--- pytables-1.2/tables/Table.py  2005-11-07 17:30:41.000000000 +010
0
+++ tables/Table.py     2005-11-29 20:47:42.357539540 +0100
@@ -1861,6 +1861,8 @@
     def _g_cleanIOBuf(self):
         """Clean the I/O buffers."""

+        # Flush the buffers before to clean-up them
+        self.flush()
         if 'row' in self.__dict__:
             # Decrement pointers to I/O buffers in Row instance
             self.row._cleanIOBuf()

Cheers,


A Dimarts 29 Novembre 2005 20:00, Alain Fagot va escriure:
> Hello,
> I switched from pytables1.1.1 to pytables 1.2 and have a problem when
> flushing tables. I tryed to reproduce the problem I have on my applicat=
ion
> to tutorial1-1.py. (I have attached the modified tutorial1-1.py)
>
> I created two separate functions:
>  - create_file : which create an hdf5 file and create "detector" group =
and
> "readout" table - fill_10 : which put data in "readout" table
>  - _unittest : which call the two previous ones and try to flush the
> "readout" table and close the hdf5 file
>
> When running I have the following error:
> Traceback (most recent call last):
>
> File "tutorial1-1.py", line 87, in ?
>
> _unittest()
>
> File "tutorial1-1.py", line 82, in _unittest
>
> table.flush()# Close the file
>
> File "C:\Logiciels\Devt\Python241\Lib\site-packages\tables\Table.py", l=
ine
> 1880, in flush
>
> self._saveBufferedRows()
>
> File "C:\Logiciels\Devt\Python241\Lib\site-packages\tables\Table.py", l=
ine
> 1413, in _saveBufferedRows
>
> self._open_append(self._v_wbuffer)
>
> File "TableExtension.pyx", line 361, in TableExtension.Table._open_appe=
nd
>
> AttributeError: 'NoneType' object has no attribute '_data'

--=20
>0,0<   Francesc Altet http://www.carabos.com/
V   V   C=E1rabos Coop. V. Enjoy Data
 "-"