Re: [Pytables-users] pytable 30 - encoding

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Anthony,

I created an issue with more info

I am not sure if this is a bug, or just a way both ne/pytables treat strings that need to touch an encoded value;

I found workaround by specifying the condvars to readWhere. Any more thoughts on this?

thanks Jeff

https://github.com/PyTables/PyTables/issues/265

I can be reached on my cell (917)971-6387

________________________________
 From: Anthony Scopatz <sc...@gm...>
To: Jeff Reback <je...@re...> 
Cc: Discussion list for PyTables <pyt...@li...> 
Sent: Tuesday, June 4, 2013 6:39 PM
Subject: Re: [Pytables-users] pytable 30 - encoding

Hi Jeff,
Hmmm, Could you try doing the same thing on just an in-memory numpy array using numexpr.  If this succeeds it tells us that the problem is in PyTables, not numexpr.

Be Well
Anthony

On Tue, Jun 4, 2013 at 11:35 AM, Jeff Reback <jr...@ya...> wrote:

Anthony,
>  
>I am using numexpr 2.1 (latest)
> 
>this is puzzling; doesn't matter what I pass (bytes or str) , same result?
> 
>(column == 'str-2')
>> /mnt/code/arb/test/pytables-3.py(38)<module>()
>-> result = handle.root.test.table.readWhere(selector)
>(Pdb) handle.root.test.table.readWhere(selector)
>*** TypeError: string argument without an encoding
>(Pdb) handle.root.test.table.readWhere(selector.encode(encoding))
>*** TypeError: string argument without an encoding
>(Pdb) 
>
>
> From: Anthony Scopatz <sc...@gm...>
>To: Jeff Reback <je...@re...>; Discussion list for PyTables <pyt...@li...> 
>Sent: Tuesday, June 4, 2013 12:25 PM
>Subject: Re: [Pytables-users] pytable 30 - encoding
> 
>
>
>Hi Jeff, 
>
>
>Have you also updated numexpr to the most recent version?  The error is coming from numexpr not compiling the expression correctly. Also, you might try making selector a str, rather than bytes: 
>
>
>selector = "(column == 'str-2')"
>
>
>
>rather than
>
>
>selector = "(column == 'str-2')".encode(encoding)
>
>
>
>Be Well
>Anthony
>
>
>
>On Tue, Jun 4, 2013 at 8:51 AM, Jeff Reback <jr...@ya...> wrote:
>
>anthony,where am I going wrong here?  
>>#!/usr/local/bin/python3
>>import tables
>>import numpy as np
>>import datetime, time
>>encoding = 'UTF-8'
>>test_file = 'test_select.h5'
>>handle = tables.openFile(test_file, "w")
>>node   = handle.createGroup(handle.root, 'test')
>>table  = handle.createTable(node, 'table', dict(
>>index   = tables.Int64Col(),
>>        column  = tables.StringCol(25),
>>values  = tables.FloatCol(shape=(3)),
>>))
>>
>># add data
>>r = table.row
>>for i in range(10):
>>r['index'] = i
>>r['column'] = ("str-%d" % (i % 5)).encode(encoding)
>>r['values'] = np.arange(3)
>>r.append()
>>table.flush()
>>handle.close()
>># read
>>handle =
 tables.openFile(test_file,"r")
>>result = handle.root.test.table.read()
>>print("table data\n")
>>print(result)
>># where
>>print("\nselector\n")
>>selector = "(column == 'str-2')".encode(encoding)
>>print(selector)
>>result = handle.root.test.table.readWhere(selector)
>>print(result)
>>
>>and the following out:
>>
>>[sheep-jreback-/code/arb/test] python3 pytables-3.py
>>table data
>>[(b'str-0', 0, [0.0, 1.0, 2.0]) (b'str-1', 1, [0.0, 1.0, 2.0])
>>(b'str-2', 2, [0.0, 1.0, 2.0]) (b'str-3', 3, [0.0, 1.0, 2.0])
>>(b'str-4', 4, [0.0, 1.0, 2.0]) (b'str-0', 5, [0.0, 1.0, 2.0])
>>(b'str-1', 6, [0.0, 1.0, 2.0]) (b'str-2', 7, [0.0, 1.0, 2.0])
>>(b'str-3', 8, [0.0, 1.0, 2.0]) (b'str-4', 9, [0.0, 1.0, 2.0])]
>>selector
>>b"(column == 'str-2')"
>>Traceback (most recent call last):
>>File "pytables-3.py", line 37, in <module>
>>result =
 handle.root.test.table.readWhere(selector)
>>File "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/_past.py", line 35, in oldfunc
>>return obj(*args, **kwargs)
>>File "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py", line 1522, in read_where
>>self._where(condition, condvars, start, stop, step)]
>>File "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py", line 1484, in _where
>>compiled = self._compile_condition(condition, condvars)
>>File "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py", line 1358, in _compile_condition
>>compiled = compile_condition(condition, typemap, indexedcols)
>>File "/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/conditions.py", line 419, in compile_condition
>>func = NumExpr(expr, signature)
>>File "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py", line 559, in NumExpr
>>precompile(ex, signature, context)
>>File "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py", line 511, in precompile
>>constants_order, constants = getConstants(ast)
>>File "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py", line 294, in getConstants
>>for a in constants_order]
>>File "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py", line 294, in <listcomp>
>>for a in constants_order]
>>File "/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py", line 284, in convertConstantToKind
>>return kind_to_type[kind](x)
>>TypeError: string argument without an encoding
>>Closing remaining open files:
 test_select.h5... done 
>>
>>------------------------------------------------------------------------------
>>How ServiceNow helps IT people transform IT departments:
>>1. A cloud service to automate IT design, transition and operations
>>2. Dashboards that offer high-level views of enterprise services
>>3. A single system of record for all IT processes
>>http://p.sf.net/sfu/servicenow-d2d-j
>>_______________________________________________
>>Pytables-users mailing list
>>Pyt...@li...
>>https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
>   