Re: [Pytables-users] pytable 30 - encoding

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi Jeff,

I have made some comments in the issue.  Thanks for investigating this
so thoroughly.

Be Well
Anthony

On Tue, Jun 4, 2013 at 8:16 PM, Jeff Reback <jr...@ya...> wrote:

> Anthony,
>
> I created an issue with more info
>
> I am not sure if this is a bug, or just a way both ne/pytables treat
> strings that need to touch an encoded value;
>
> I found workaround by specifying the condvars to readWhere. Any more
> thoughts on this?
>
> thanks Jeff
>
>
> https://github.com/PyTables/PyTables/issues/265
>
> I can be reached on my cell (917)971-6387
>     *From:* Anthony Scopatz <sc...@gm...>
> *To:* Jeff Reback <je...@re...>
> *Cc:* Discussion list for PyTables <pyt...@li...>
> *Sent:* Tuesday, June 4, 2013 6:39 PM
>
> *Subject:* Re: [Pytables-users] pytable 30 - encoding
>
> Hi Jeff,
>
> Hmmm, Could you try doing the same thing on just an in-memory numpy array
> using numexpr.  If this succeeds it tells us that the problem is in
> PyTables, not numexpr.
>
> Be Well
> Anthony
>
>
> On Tue, Jun 4, 2013 at 11:35 AM, Jeff Reback <jr...@ya...> wrote:
>
> Anthony,
>
> I am using numexpr 2.1 (latest)
>
> this is puzzling; doesn't matter what I pass (bytes or str) , same result?
>
> (column == 'str-2')
> > /mnt/code/arb/test/pytables-3.py(38)<module>()
> -> result = handle.root.test.table.readWhere(selector)
> (Pdb) handle.root.test.table.readWhere(selector)
> *** TypeError: string argument without an encoding
> (Pdb) handle.root.test.table.readWhere(selector.encode(encoding))
> *** TypeError: string argument without an encoding
> (Pdb)
>
>
>    *From:* Anthony Scopatz <sc...@gm...>
> *To:* Jeff Reback <je...@re...>; Discussion list for PyTables <
> pyt...@li...>
> *Sent:* Tuesday, June 4, 2013 12:25 PM
> *Subject:* Re: [Pytables-users] pytable 30 - encoding
>
> Hi Jeff,
>
> Have you also updated numexpr to the most recent version?  The error is
> coming from numexpr not compiling the expression correctly. Also, you might
> try making selector a str, rather than bytes:
>
> selector = "(column == 'str-2')"
>
> rather than
>
> selector = "(column == 'str-2')".encode(encoding)
>
> Be Well
> Anthony
>
>
> On Tue, Jun 4, 2013 at 8:51 AM, Jeff Reback <jr...@ya...> wrote:
>
> anthony,where am I going wrong here?
> #!/usr/local/bin/python3
> import tables
> import numpy as np
> import datetime, time
> encoding = 'UTF-8'
> test_file = 'test_select.h5'
> handle = tables.openFile(test_file, "w")
> node = handle.createGroup(handle.root, 'test')
> table = handle.createTable(node, 'table', dict(
> index = tables.Int64Col(),
>         column = tables.StringCol(25),
> values = tables.FloatCol(shape=(3)),
> ))
>
> # add data
> r = table.row
> for i in range(10):
> r['index'] = i
> r['column'] = ("str-%d" % (i % 5)).encode(encoding)
> r['values'] = np.arange(3)
> r.append()
> table.flush()
> handle.close()
> # read
> handle = tables.openFile(test_file,"r")
> result = handle.root.test.table.read()
> print("table data\n")
> print(result)
> # where
> print("\nselector\n")
> selector = "(column == 'str-2')".encode(encoding)
> print(selector)
> result = handle.root.test.table.readWhere(selector)
> print(result)
> and the following out:
>
> [sheep-jreback-/code/arb/test] python3 pytables-3.py
> table data
> [(b'str-0', 0, [0.0, 1.0, 2.0]) (b'str-1', 1, [0.0, 1.0, 2.0])
> (b'str-2', 2, [0.0, 1.0, 2.0]) (b'str-3', 3, [0.0, 1.0, 2.0])
> (b'str-4', 4, [0.0, 1.0, 2.0]) (b'str-0', 5, [0.0, 1.0, 2.0])
> (b'str-1', 6, [0.0, 1.0, 2.0]) (b'str-2', 7, [0.0, 1.0, 2.0])
> (b'str-3', 8, [0.0, 1.0, 2.0]) (b'str-4', 9, [0.0, 1.0, 2.0])]
> selector
> b"(column == 'str-2')"
> Traceback (most recent call last):
> File "pytables-3.py", line 37, in <module>
> result = handle.root.test.table.readWhere(selector)
> File
> "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/_past.py",
> line 35, in oldfunc
> return obj(*args, **kwargs)
> File
> "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
> line 1522, in read_where
> self._where(condition, condvars, start, stop, step)]
> File
> "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
> line 1484, in _where
> compiled = self._compile_condition(condition, condvars)
> File
> "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
> line 1358, in _compile_condition
> compiled = compile_condition(condition, typemap, indexedcols)
> File
> "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/conditions.py",
> line 419, in compile_condition
> func = NumExpr(expr, signature)
> File
> "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
> line 559, in NumExpr
> precompile(ex, signature, context)
> File
> "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
> line 511, in precompile
> constants_order, constants = getConstants(ast)
> File
> "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
> line 294, in getConstants
> for a in constants_order]
> File
> "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
> line 294, in <listcomp>
> for a in constants_order]
> File
> "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
> line 284, in convertConstantToKind
> return kind_to_type[kind](x)
> TypeError: string argument without an encoding
> Closing remaining open files: test_select.h5... done
>
>
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. A cloud service to automate IT design, transition and operations
> 2. Dashboards that offer high-level views of enterprise services
> 3. A single system of record for all IT processes
> http://p.sf.net/sfu/servicenow-d2d-j
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
>
>
>
>
>
>