Re: [Pytables-users] In-kernal for subset?

[Anthony S. <sc...@gm...>]

On Wed, Aug 15, 2012 at 12:33 PM, Adam Dershowitz
<ade...@ex...>wrote:

>  I am trying to find all cases where a value transitions above a
> threshold.  So, my code first does a getwherelist to find values that are
> above the threshold, then it uses that list to find immediately prior
> values that are below.  The code is working, but the second part, searching
> through just a smaller subset is much slower (First search is on the order
> of 1 second, while the second is a minute).
> Is there any way to get this second part of the search in-kernal?  Or any
> more general way to do a search for values above a threshold, where the
> prior value is below?
> Essentially, what I am looking for is a way to speed up that second search
> for "all rows in a prior defined list, where a condition is applied to the
> table"
>
>  My table is just seconds and values, in chronological order.
>
>  Here is the code that I am using now:
>
>  h5data = tb.openFile("AllData.h5","r")
> table1 = h5data.root.table1
>
>  #Find all values above threshold:
>  thelist= table1.getWhereList("""Value > 150""")
>
>  #From the above list find all values where the immediately prior value
> is below:
> transition=[]
> for i in thelist:
> if (table1[i-1]['Value'] < 150) and (i != 0) :
> transition.append(i)
>

Hey Adam,

Sorry for taking a while to respond.  Assuming you don't mind one of these
being <= or >=, you don't really need the second loop with a little index
arithmetic:

import numpy as np
inds = np.array(thelist)
dinds = inds[1:] - inds[:-1]
transition = dinds[(1 < dinds)]

This should get you an array of all of the transition indices since
wherever the difference in indices is greater than 1 the Value must have
dropped below the threshold and then returned back up.

Be Well
Anthony


>
>  Thanks,
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>