Re: [Pytables-users] readWhere, number of selectors issue

[Gaëtan de M. <gde...@gm...>]

On Wed, Jan 23, 2013 at 5:33 PM, Jeff Reback <jr...@ya...> wrote:

> It seems there is a limit to the condition sytax when using readWhere
>
> I get various exceptions when passing increasing number of terms
>
> is this some kind of hard coded limit?
>

This is a limitation of numexpr. FWIW, this is due to the fact that the
number of numexpr internal "registers" (including temporary ones) is
implicitly limited to 256 because they are coded in a single character in
its internal representation of your expression (internally a string called
"program").

In your case, Numexpr could theoretically do a much better job of
allocating temporary registers, so you could add a feature request at
https://code.google.com/p/numexpr/issues/ (but don't hold your breath on
it).

In the meantime, the best workaround I know of is to read chunks out of
pytables and call numexpr manually on them (because in that case you can
simply split your expression in multiple smaller exprs):
filter1 = ne.evaluate(c1 | c2 | c3 | ... | cx)
filter2 = ne.evaluate(filter1 | cy | ...)
-- 
Gaëtan de Menten