Re: [Pytables-users] using vlarray

[Francesc A. <fa...@py...>]

A Tuesday 15 December 2009 17:31:46 Ernesto escrigué:
> thank you for your explanation and for your time. I'll try to use a
> combined approach.
> A very stupid question is the following:
> 
> Normally I have to update each string of the vlarray (and in the table
> as well) adding a character at the end.
> 
> For example:
> vlarray = fileh.createVLArray(root, 'vlarray7', VLStringAtom(),
>                                "Variable Length String")
> vlarray.append("asd")
> vlarray.append("aaana")
> 
> if I write:
> 
> vlarray__setitem__(0,"acg")
> 
> The value at position 0 is correctly updated.
> If instead I write:
> 
> vlarray__setitem__(0,"acgt")
> 
> I get a ValueError as expected reading the manual.
> The question is:
> Is there a way to update the string of a vlarray?

As stated in the manual, you can update the value in a vlarray entry, but you 
cannot change the shape (length) of the entry.  So, if you need to enlarge 
your entries I can devise a couple of possibilities:

a) If you are not enlarging your vlarray entries frequently you can read the 
content of your entry, append the character and then append the new string in 
a new vlarray entry.  Then you only have to update the 'where' field in your 
table and you are done.  If you want to get rid of 'unreferenced' entries in 
your vlarray you can optionally setup a clean-up procedure by copying only the 
referenced entries into the final vlarray.

b) If the updates are very frequent, my advice is to use a table with a single 
field where you can append your characters (one per row).  This is very fast.  
When you finish your updates you can migrate these single-column tables into 
vlarray entries so that they can be manipulated more easily.

If taking the approach b) mind you to not create too many nodes (tables) 
because the underlying HDF5 library might require a *lot* of memory to handle 
this.  For a number of nodes not exceeding tens of thousands, this is okay.  
If you need more, you may need to go for another schema (like adding an 
intermediate earray where you save the intermediate length strings).  But 
let's see first if one of the above could work for you.

Ciao,

-- 
Francesc Alted