[cx-oracle-users] Extracting data in Native Language: Error UnicodeEncodeError

[Wong W. Meng-R. <r3...@fr...>]

Hello there,

I am migrating my application from python 1.5.2 to 2.7.1, from using oracledb to cx_oracle.

My database is setup with AL32UTF8 characterset, in order to support Chinese character entry.

I am in testing stage now. I have never used nVarchar type before as my column type but now I am using it so that I can be sure exactly the number of bytes the field can store is predictable regardless it is plain English or Unicode characters.

When I do insert and update, I do not explicitly pass in some prefix (u for unicode) onto the data in my SQL statement. However, when I do a SQL query, the return list tuple from python cx_Oracle seems to attach a prefix u in front of the data with column type nVarchar. When I want to extract or parse the data, I encountered an error (expectedly). I am new to this can anyone tell what I should look at in order to parse native language characters successfully after getting the raw reply from a cx_Oracle's query?

E.g.

>>> reply=a.execute("select * from TEST271")
>>> reply
[('\xb2\xe2\xca\xd4', u'\u6d4b\u8bd5')]
>>> for (text1, text2) in reply:
...     print text1, text2
...
ýãòå Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/home/r32813/genesis/GEN_DEV_271/Enablers/Python/lib/python2.7/encodings/hp_roman8.py", line 17, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-1: character maps to <undefined>
>>>

SQL> desc test271
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 TEXT1                                              VARCHAR2(16)
 TEXT2                                              NVARCHAR2(16)


Regards,
Wah Meng