Re: [cx-oracle-users] Unicode support
Brought to you by:
atuining
Menu
▾
▴
Re: [cx-oracle-users] Unicode support
|
From: Sébastien S. <sa...@us...> - 2008-10-17 10:00:15
|
Hi Anthony,
thank you for those improvements!
I don't expect to work again on Unicode in my application before a few
weeks; but when I will, I will give you some feedback whether it works
or not.
For your information, I made a prototype of my application running in
Unicode (with some Chinese text) for a demo which worked great. I used
cx_Oracle revision 142 with a database encoded in AL32UTF8 (AL16UTF16
was working fine also; UTF-8 was not).
regards
--
Sébastien Sablé
Anthony Tuininga a écrit :
> Hi Sebastien,
>
> I just committed changes to use byte semantics instead of character
> semantics and all appears to be well -- but I'm not sure completely so
> I'd appreciate some testing. Note that I am also in the process of
> adding "full" unicode support in preparation for a port to Python 3.0.
> This implies that all strings are both expected in unicode and will be
> returned in unicode -- but its not complete yet. The test cases
> currently pass but there may still be some holes. Please advise if you
> find one. :-)
>
> Anthony
>
> On Mon, Oct 6, 2008 at 10:21 AM, Sébastien Sablé
> <sa...@us...> wrote:
>> Hi Anthony,
>>
>> I did the test with a database using AL16UTF16 and it works fine.
>> So I guess the problem only happens when using variable width character
>> set. I will stick with AL16UTF16 for the moment.
>>
>> Thanks for your help
>>
>> --
>> Sébastien Sablé
>>
>>
>>
>> Anthony Tuininga a écrit :
>>> Hmm, do you have access to a database with a server side fixed width
>>> character set? If not, I'll have to try creating a database with a
>>> variable width character set like UTF8 and try it -- but my time is
>>> stretched right now so it will have to wait if you want me to look
>>> into it. :-)
>>>
>>> Anthony
>>>
>>> On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé
>>> <sa...@us...> wrote:
>>>> Hi Anthony,
>>>>
>>>> I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on
>>>> sunos 10.
>>>>
>>>> Here are the parameters in my database:
>>>>
>>>> $ sqlplus SSA_UTF8/sungard@UTF102NP
>>>>
>>>> SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008
>>>>
>>>> Copyright (c) 1982, 2005, Oracle. All rights reserved.
>>>>
>>>>
>>>> Connected to:
>>>> Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
>>>> With the Partitioning, OLAP and Data Mining options
>>>>
>>>> SQL> select * from nls_database_parameters where instr(parameter,
>>>> 'CHARACTERSET') != 0;
>>>> 2
>>>> PARAMETER
>>>> ------------------------------
>>>> VALUE
>>>> --------------------------------------------------------------------------------
>>>> NLS_CHARACTERSET
>>>> UTF8
>>>>
>>>> NLS_NCHAR_CHARACTERSET
>>>> UTF8
>>>>
>>>>
>>>> So this may be linked to the different charset being used.
>>>> regards
>>>>
>>>> --
>>>> Sébastien Sablé
>>>>
>>>>
>>>> Anthony Tuininga a écrit :
>>>>> Hi,
>>>>>
>>>>> I tried the code you provided as is and it worked for me. :-) I
>>>>> checked the error message and the cause claims to be that you cannot
>>>>> use OCI_ATTR_CHAR_COUNT when the server or client character set is
>>>>> variable length. I was using a fixed length character set so I tried a
>>>>> variable length character set but I still didn't get an error. I am
>>>>> using the Oracle 10.2.0.3 instant client on Windows and the full
>>>>> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any
>>>>> difficulty. Can anyone else tell me if they are getting the same
>>>>> behavior? Please specify the Oracle client being used and the value
>>>>> for connection.encoding and connection.encoding as well as the results
>>>>> of
>>>>>
>>>>> select * from nls_database_parameters where instr(parameter,
>>>>> 'CHARACTERSET') != 0;
>>>>>
>>>>> For example, with the Windows instant client:
>>>>>
>>>>> Oracle Instant client version 10.2.0.3
>>>>> connection.encoding = "WINDOWS-1252"
>>>>> connection.nencoding = "WINDOWS-1252"
>>>>>
>>>>> PARAMETER VALUE
>>>>> ------------------------------ ----------------------------------------
>>>>> NLS_NCHAR_CHARACTERSET AL16UTF16
>>>>> NLS_CHARACTERSET WE8MSWIN1252
>>>>>
>>>>> Anthony
>>>>>
>>>>> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé
>>>>> <sa...@us...> wrote:
>>>>>> Hi Anthony,
>>>>>>
>>>>>> thank you for this evolution of cx_Oracle!
>>>>>>
>>>>>> It tested it and it seems to work fine for output, however when doing
>>>>>> some input I always get the same kind of error:
>>>>>>
>>>>>> $ python test_oracle.py
>>>>>> Traceback (most recent call last):
>>>>>> File "test_oracle.py", line 18, in <module>
>>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def'])
>>>>>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here
>>>>>>
>>>>>> Here is the trivial test I ran:
>>>>>>
>>>>>> import cx_Oracle
>>>>>>
>>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale):
>>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR):
>>>>>> return cursor.var(unicode, size, cursor.arraysize)
>>>>>>
>>>>>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP')
>>>>>> connection.outputtypehandler = OutputTypeHandler
>>>>>> cursor = connection.cursor()
>>>>>>
>>>>>> try:
>>>>>> cursor.execute('drop table personne')
>>>>>> except cx_Oracle.DatabaseError:
>>>>>> pass
>>>>>>
>>>>>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))')
>>>>>>
>>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def'])
>>>>>>
>>>>>> cursor.execute("select * from personne")
>>>>>> for row in cursor:
>>>>>> print "Row:", row
>>>>>>
>>>>>>
>>>>>>
>>>>>> Is there something to configure in order to allow unicode strings input?
>>>>>> Thanks in advance
>>>>>>
>>>>>> --
>>>>>> Sébastien Sablé
>>>>>>
>>>>>>
>>>>>> Anthony Tuininga a écrit :
>>>>>>> All,
>>>>>>>
>>>>>>> I just checked in to Subversion support for Unicode. Thanks go to
>>>>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and
>>>>>>> to everyone else for their patience. :-) Since I have little
>>>>>>> experience with languages other than English and I only have access to
>>>>>>> little endian machines I'd appreciate those of you with such
>>>>>>> experience or machines to test the new code for me and let me know the
>>>>>>> results.
>>>>>>>
>>>>>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is
>>>>>>> also possible to bind unicode strings in to cursors. SQL statements
>>>>>>> and varchar2 and char data are still returned as encoded strings. Note
>>>>>>> that with another new feature in cx_Oracle you can force all strings
>>>>>>> to be returned as unicode if you so desire. Below is an example that
>>>>>>> will run with the data in the test suite:
>>>>>>>
>>>>>>> import cx_Oracle
>>>>>>>
>>>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale):
>>>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR):
>>>>>>> return cursor.var(unicode, size, cursor.arraysize)
>>>>>>>
>>>>>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev")
>>>>>>> connection.outputtypehandler = OutputTypeHandler
>>>>>>> cursor = connection.cursor()
>>>>>>> cursor.execute("select * from teststrings")
>>>>>>> for row in cursor:
>>>>>>> print "Row:", row
>>>>>>>
>>>>>>> If you have questions, let me know. If you have feedback, please
>>>>>>> provide it. Thanks.
>>>>>>>
>>>>>>> Anthony
>>>>>>>
>>>>>>> -------------------------------------------------------------------------
>>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>>> _______________________________________________
>>>>>>> cx-oracle-users mailing list
>>>>>>> cx-...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>>>>> -------------------------------------------------------------------------
>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>> _______________________________________________
>>>>>> cx-oracle-users mailing list
>>>>>> cx-...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>>>>>
>>>>> -------------------------------------------------------------------------
>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>> _______________________________________________
>>>>> cx-oracle-users mailing list
>>>>> cx-...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> cx-oracle-users mailing list
>>>> cx-...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>>>
>>> -------------------------------------------------------------------------
>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>> _______________________________________________
>>> cx-oracle-users mailing list
>>> cx-...@li...
>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> cx-oracle-users mailing list
>> cx-...@li...
>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
|
