Re: [Pytables-users] Pytables file structure

[Juan M. V. T. <jmv...@gm...>]

The column I´m requesting the data from has multidimensional cells, so each
time I request data from the table, I need to get a specific row for all
the multidimensional cells in the column. I hope this clarifies a bit.
I have at the office a Linux workstation, but it is part of a computing
cluster where all the users have access, so the files are in a folder of
the cluster, not in my hard drive.

Thank you,
Juanma

2012/7/15 Anthony Scopatz <sc...@gm...>

> Rereading the original post, I am a little confused are your trying to
> read the whole table, just a couple of rows that meet some condition, or
> just one whole column, or one part of the column.
>
> To request the whole table without looping over each row in Python, index
> every element:
>
> f.root.table[:]
>
>
> To just get certain rows, use where().
>
> To get a single column, use the cols namespace:
>
> f.root.table.cols.my_column[:]
>
>
> Why is this file elsewhere on the network?
>
> Be Well
> Anthony
>
> On Sun, Jul 15, 2012 at 4:08 PM, Juan Manuel Vázquez Tovar <
> jmv...@gm...> wrote:
>
>> Hello Anthony,
>>
>> I have to loop over the whole set of rows. Does the where method has any
>> advantages in that case?
>>
>> Thank you,
>> Juanma
>>
>> 2012/7/15 Anthony Scopatz <sc...@gm...>
>>
>>> Hello Juan,
>>>
>>> Try using the where() method [1],  It has a lot of nice features under
>>> the covers.
>>>
>>> Be Well
>>> Anthony
>>>
>>> 1.
>>> http://pytables.github.com/usersguide/libref.html?highlight=where#tables.Table.where
>>>
>>> On Sun, Jul 15, 2012 at 4:01 PM, Juan Manuel Vázquez Tovar <
>>> jmv...@gm...> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have been using pytables for a few moths. The main structure of my
>>>> files has a four column table, two of which have multidimensional cells,
>>>> (56,1) and (133,6) respectively. The previous structure had more columns
>>>> instead of storing the 56x1 array into the same cell. The largest file has
>>>> almost three million rows in the table.
>>>> I usually request data from the table looping through the entire table
>>>> and getting for each row one specific row of the 133x6 2d array.
>>>> Currently, each of the requests can take from 15 sec up to 10 minutes,
>>>> I believe that depending on the status of the office network.
>>>> Could you please advice about how to improve the reading time?
>>>> I have tried to compress the data with zlib, but it takes more or less
>>>> the same time.
>>>>
>>>> Thanks in advance,
>>>>
>>>> Juan Manuel
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Live Security Virtual Conference
>>>> Exclusive live event will cover all the ways today's security and
>>>> threat landscape has changed and how IT managers can respond.
>>>> Discussions
>>>> will include endpoint security, mobile security and the latest in
>>>> malware
>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>> _______________________________________________
>>>> Pytables-users mailing list
>>>> Pyt...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Pytables-users mailing list
>>> Pyt...@li...
>>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Pytables-users mailing list
>> Pyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>