pytables-users

[Pytables-users] In-kernel searches not returning values?

[Giovanni L. C. <glc...@gm...>]

Hi,

I am new to PyTables and I like it very much though there are still some 
problems I am trying to solve. The latest is that I am seeing a strange behavior 
when using in-kernel searches. The seach condition is a simple equality test on 
a single column. Basically, when the column is indexed, in-kernel searches don't 
return the expected result, that is:

In [150]: [ row['visits'] for row in ap.where('rid == 665689') ]
Out[150]: []

In [151]: [ row['visits'] for row in ap if row['rid'] == 665689 ]
Out[151]: [18L]

When I remove the index, it works again:

In [153]: ap.cols.rid.removeIndex()

In [154]: [ row['visits'] for row in ap.where('rid == 665689') ]
Out[154]: [18L]

Am I doing something wrong? This is an excerpt of the contents of the file:

-> % h5ls -ld test.h5|head
AllPages Dataset {529000/Inf}
Data:
(0) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=665689,
(0) visits=18},
(1) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=2,
(1) visits=11},
(2) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=12,
(2) visits=1},
(3) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=612075,
(3) visits=8},

And this is the table description:

Out[152]:
/AllPages (Table(529000,), shuffle, zlib(5)) ''
description := {
"year": UInt16Col(shape=(), dflt=0, pos=0),
"month": UInt8Col(shape=(), dflt=0, pos=1),
"day": UInt8Col(shape=(), dflt=0, pos=2),
"hour": UInt8Col(shape=(), dflt=0, pos=3),
"minute": UInt8Col(shape=(), dflt=0, pos=4),
"epoch": UInt32Col(shape=(), dflt=0, pos=5),
"rid": UInt32Col(shape=(), dflt=0, pos=6),
"visits": UInt32Col(shape=(), dflt=0, pos=7)}
byteorder := 'little'
chunkshape := (233016,)
autoIndex := True
colindexes := {
"rid": Index(1, light, shuffle, zlib(1)).is_CSI=False}

Thanks!

-- 
Giovanni Luca Ciampaglia

Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University

✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
☞ http://cnets.indiana.edu/
✉ gci...@in...

Re: [Pytables-users] In-kernel searches not returning values?

[Anthony S. <sc...@gm...>]

Hello Giovanni!

This definitely seems like a bug.  How was the column indexed?  Could you
send a sample script that reproduces the problem from start to finish?
Thanks.

Be Well
Anthony


On Fri, Apr 26, 2013 at 6:14 PM, Giovanni Luca Ciampaglia <
glc...@gm...> wrote:

> Hi,
>
> I am new to PyTables and I like it very much though there are still some
> problems I am trying to solve. The latest is that I am seeing a strange
> behavior
> when using in-kernel searches. The seach condition is a simple equality
> test on
> a single column. Basically, when the column is indexed, in-kernel searches
> don't
> return the expected result, that is:
>
> In [150]: [ row['visits'] for row in ap.where('rid == 665689') ]
> Out[150]: []
>
> In [151]: [ row['visits'] for row in ap if row['rid'] == 665689 ]
> Out[151]: [18L]
>
> When I remove the index, it works again:
>
> In [153]: ap.cols.rid.removeIndex()
>
> In [154]: [ row['visits'] for row in ap.where('rid == 665689') ]
> Out[154]: [18L]
>
> Am I doing something wrong? This is an excerpt of the contents of the file:
>
> -> % h5ls -ld test.h5|head
> AllPages Dataset {529000/Inf}
> Data:
> (0) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600,
> rid=665689,
> (0) visits=18},
> (1) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=2,
> (1) visits=11},
> (2) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600, rid=12,
> (2) visits=1},
> (3) {year=2008, month=1, day=1, hour=0, minute=0, epoch=1199145600,
> rid=612075,
> (3) visits=8},
>
> And this is the table description:
>
> Out[152]:
> /AllPages (Table(529000,), shuffle, zlib(5)) ''
> description := {
> "year": UInt16Col(shape=(), dflt=0, pos=0),
> "month": UInt8Col(shape=(), dflt=0, pos=1),
> "day": UInt8Col(shape=(), dflt=0, pos=2),
> "hour": UInt8Col(shape=(), dflt=0, pos=3),
> "minute": UInt8Col(shape=(), dflt=0, pos=4),
> "epoch": UInt32Col(shape=(), dflt=0, pos=5),
> "rid": UInt32Col(shape=(), dflt=0, pos=6),
> "visits": UInt32Col(shape=(), dflt=0, pos=7)}
> byteorder := 'little'
> chunkshape := (233016,)
> autoIndex := True
> colindexes := {
> "rid": Index(1, light, shuffle, zlib(1)).is_CSI=False}
>
> Thanks!
>
> --
> Giovanni Luca Ciampaglia
>
> Postdoctoral fellow
> Center for Complex Networks and Systems Research
> Indiana University
>
> ✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
> ☞ http://cnets.indiana.edu/
> ✉ gci...@in...
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>