[Pytables-users] How would I update specific rows from a table with rows of a second table?

[PyTables O. <pyt...@go...>]

Forwarding,
~Josh

On Apr 30, 2012, at 5:04 PM, pyt...@li... wrote:
> From: Christian Werner <fre...@go...>
> Date: April 30, 2012 5:03:59 PM GMT+02:00
> To: pyt...@li...
> Subject: How would I update specific rows from a table with rows of a second table?
> 
> 
> Hi group.
> 
> Please consider this scenario. I have produced a large h5 file holding the outcome of some simulations (hdf file holds some groups and about 5 tables, but for my question this does not matter). 
> How I had to rerun my simulation but only for certain fields and thus I have a second h5 file with the same structure but only for certain fields of the original file (old age classes, see below).
> 
> Or to be more verbose:
> 
> Table (original)
> id	year	species	age	data1	data2	data3	...
> 1	1990	spruce	75	1.2	3.2	3.3 ...
> 1	1991	spruce	75	1.3	3.1	2.2 ...
> ...
> 1	1990	spruce	125	1.1	2.1	1.5	...
> ...
> 1	1990	pine	145	1.1	2.1	1.5	...
> ...
> 2	1990	spruce	45	1.2	3.2	3.3 ...
> 2	1991	spruce	55	1.3	3.1	2.2 ...
> ...
> 
> I had to rerun my simulations for old vegetation classes. So this table only contains the following lines (e.g., spruce age > 80)
> 
> Table (new)
> id	year	species	age	data1	data2	data3	...
> ...
> 1	1990	spruce	125	1.1	2.1	1.5	...
> ...
> 1	1990	pine	145	1.1	2.1	1.5	...
> 
> So basically I need to use the rows of the new table and overwrite the corresponding rows of the old table to form the updated one. I do need to retain the original order (sort order id > year > species > age). Those columns combined give a unique row identifier...
> 
> The final table would have the same size and order as the original table with only the rows containing data for old age classes updated with the new simulation results...
> 
> Can anyone give me a hint how to achieve this?
> 
> I guess I need to introduce a new (unique index) column first in both tables (e.g., combining  id + year + species + age so I can match those rows by this identifier), right? Does anyone have some example code for me?
> 
> Problem is, the 2 h5 files are about 7GB each.
> 
> Cheers,
> Christian
> 
> 
>