Re: [Pytables-users] Best way to store data that allows inserts/deletes

[Tom D. <td...@gm...>]

>> I'm writing a wrapper for sparse matrices (CSR format) and therefore
>> need to store three vectors and 3 scalars:
>>
>> - data        (float64 vector)
>> - indices    (int32 vector)
>> - indptr      (int32 vector)
>>
>> - nrows      (int32 scalar)
>> - ncols       (int32 scalar)
>> - nnz          (int32 scalar)
>>
>> data and indices will always be the same length as each other (=nnz)
>> but the indptr vector is much shorter.
>>
>> I've written routines that allow you to insert/update/delete rows or
>> columns of the matrix by acting on these vectors only. However I'm
>> struggling to work out the best pytables structure to store these,
>> that allows me to append/insert/delete elements easily, and is
>> efficient.
>>
>> I was using a Group with an EArray for each vector. This works ok but
>> it seems like you are unable to delete items - is that correct?
>>
>> I also tried using a Group with a separate Table for each of the
>> vectors (I could possibly just have two - one for data and indices and
>> the other for indptr), but this seems to add a lot of overhead in
>> manipulating the arrays.
>>
>> Is there something simple I'm missing?
>>
>
> Hi Tom,
>
> It seems to me that what you want to be using is a Table, which supports a
> lot of the functionality you desire, including removing rows (which EArrays
> don't easily support)
> http://pytables.github.com/usersguide/libref.html?highlight=remove#tables.Table.removeRows
> .
>
> Rather than having a Group of EArrays, have a table whose columns represent
> what you were storing in the EArrays.  You should take a look at the Tables
> tutorials to see how to do this:
> http://pytables.github.com/usersguide/tutorials.html
>
> Be Well
> Anthony
>
>

I think I've read those tutorials a dozen times over now ....

As far as I can tell there is no way of inserting a row (as opposed to
appending it to the end) - am I correct? I'm looking for the
equivalent of numpy.insert

Tom