Thread: [cx-oracle-users] Unicode support
Brought to you by:
atuining
From: Anthony T. <ant...@gm...> - 2008-09-30 22:12:16
|
All, I just checked in to Subversion support for Unicode. Thanks go to Amaury Forgeot d'Arc for the patch he provided earlier this year and to everyone else for their patience. :-) Since I have little experience with languages other than English and I only have access to little endian machines I'd appreciate those of you with such experience or machines to test the new code for me and let me know the results. Note that now nchar, nvarchar2 data are now returned as unicode. It is also possible to bind unicode strings in to cursors. SQL statements and varchar2 and char data are still returned as encoded strings. Note that with another new feature in cx_Oracle you can force all strings to be returned as unicode if you so desire. Below is an example that will run with the data in the test suite: import cx_Oracle def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): return cursor.var(unicode, size, cursor.arraysize) connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") connection.outputtypehandler = OutputTypeHandler cursor = connection.cursor() cursor.execute("select * from teststrings") for row in cursor: print "Row:", row If you have questions, let me know. If you have feedback, please provide it. Thanks. Anthony |
From: Sébastien S. <sa...@us...> - 2008-10-02 13:23:40
|
Hi Anthony, thank you for this evolution of cx_Oracle! It tested it and it seems to work fine for output, however when doing some input I always get the same kind of error: $ python test_oracle.py Traceback (most recent call last): File "test_oracle.py", line 18, in <module> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here Here is the trivial test I ran: import cx_Oracle def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): return cursor.var(unicode, size, cursor.arraysize) connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') connection.outputtypehandler = OutputTypeHandler cursor = connection.cursor() try: cursor.execute('drop table personne') except cx_Oracle.DatabaseError: pass cursor.execute('create table personne (name nchar(4), first_name nchar(6))') cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) cursor.execute("select * from personne") for row in cursor: print "Row:", row Is there something to configure in order to allow unicode strings input? Thanks in advance -- Sébastien Sablé Anthony Tuininga a écrit : > All, > > I just checked in to Subversion support for Unicode. Thanks go to > Amaury Forgeot d'Arc for the patch he provided earlier this year and > to everyone else for their patience. :-) Since I have little > experience with languages other than English and I only have access to > little endian machines I'd appreciate those of you with such > experience or machines to test the new code for me and let me know the > results. > > Note that now nchar, nvarchar2 data are now returned as unicode. It is > also possible to bind unicode strings in to cursors. SQL statements > and varchar2 and char data are still returned as encoded strings. Note > that with another new feature in cx_Oracle you can force all strings > to be returned as unicode if you so desire. Below is an example that > will run with the data in the test suite: > > import cx_Oracle > > def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): > if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): > return cursor.var(unicode, size, cursor.arraysize) > > connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") > connection.outputtypehandler = OutputTypeHandler > cursor = connection.cursor() > cursor.execute("select * from teststrings") > for row in cursor: > print "Row:", row > > If you have questions, let me know. If you have feedback, please > provide it. Thanks. > > Anthony > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users |
From: Anthony T. <ant...@gm...> - 2008-10-02 14:12:56
|
Hi, I tried the code you provided as is and it worked for me. :-) I checked the error message and the cause claims to be that you cannot use OCI_ATTR_CHAR_COUNT when the server or client character set is variable length. I was using a fixed length character set so I tried a variable length character set but I still didn't get an error. I am using the Oracle 10.2.0.3 instant client on Windows and the full Oracle 11.1.0.6 client on Linux. Neither of these seem to have any difficulty. Can anyone else tell me if they are getting the same behavior? Please specify the Oracle client being used and the value for connection.encoding and connection.encoding as well as the results of select * from nls_database_parameters where instr(parameter, 'CHARACTERSET') != 0; For example, with the Windows instant client: Oracle Instant client version 10.2.0.3 connection.encoding = "WINDOWS-1252" connection.nencoding = "WINDOWS-1252" PARAMETER VALUE ------------------------------ ---------------------------------------- NLS_NCHAR_CHARACTERSET AL16UTF16 NLS_CHARACTERSET WE8MSWIN1252 Anthony On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé <sa...@us...> wrote: > Hi Anthony, > > thank you for this evolution of cx_Oracle! > > It tested it and it seems to work fine for output, however when doing > some input I always get the same kind of error: > > $ python test_oracle.py > Traceback (most recent call last): > File "test_oracle.py", line 18, in <module> > cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) > cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here > > Here is the trivial test I ran: > > import cx_Oracle > > def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): > if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): > return cursor.var(unicode, size, cursor.arraysize) > > connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') > connection.outputtypehandler = OutputTypeHandler > cursor = connection.cursor() > > try: > cursor.execute('drop table personne') > except cx_Oracle.DatabaseError: > pass > > cursor.execute('create table personne (name nchar(4), first_name nchar(6))') > > cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) > > cursor.execute("select * from personne") > for row in cursor: > print "Row:", row > > > > Is there something to configure in order to allow unicode strings input? > Thanks in advance > > -- > Sébastien Sablé > > > Anthony Tuininga a écrit : >> All, >> >> I just checked in to Subversion support for Unicode. Thanks go to >> Amaury Forgeot d'Arc for the patch he provided earlier this year and >> to everyone else for their patience. :-) Since I have little >> experience with languages other than English and I only have access to >> little endian machines I'd appreciate those of you with such >> experience or machines to test the new code for me and let me know the >> results. >> >> Note that now nchar, nvarchar2 data are now returned as unicode. It is >> also possible to bind unicode strings in to cursors. SQL statements >> and varchar2 and char data are still returned as encoded strings. Note >> that with another new feature in cx_Oracle you can force all strings >> to be returned as unicode if you so desire. Below is an example that >> will run with the data in the test suite: >> >> import cx_Oracle >> >> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >> return cursor.var(unicode, size, cursor.arraysize) >> >> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >> connection.outputtypehandler = OutputTypeHandler >> cursor = connection.cursor() >> cursor.execute("select * from teststrings") >> for row in cursor: >> print "Row:", row >> >> If you have questions, let me know. If you have feedback, please >> provide it. Thanks. >> >> Anthony >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
From: Sébastien S. <sa...@us...> - 2008-10-02 16:50:57
|
Hi Anthony, I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on sunos 10. Here are the parameters in my database: $ sqlplus SSA_UTF8/sungard@UTF102NP SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production With the Partitioning, OLAP and Data Mining options SQL> select * from nls_database_parameters where instr(parameter, 'CHARACTERSET') != 0; 2 PARAMETER ------------------------------ VALUE -------------------------------------------------------------------------------- NLS_CHARACTERSET UTF8 NLS_NCHAR_CHARACTERSET UTF8 So this may be linked to the different charset being used. regards -- Sébastien Sablé Anthony Tuininga a écrit : > Hi, > > I tried the code you provided as is and it worked for me. :-) I > checked the error message and the cause claims to be that you cannot > use OCI_ATTR_CHAR_COUNT when the server or client character set is > variable length. I was using a fixed length character set so I tried a > variable length character set but I still didn't get an error. I am > using the Oracle 10.2.0.3 instant client on Windows and the full > Oracle 11.1.0.6 client on Linux. Neither of these seem to have any > difficulty. Can anyone else tell me if they are getting the same > behavior? Please specify the Oracle client being used and the value > for connection.encoding and connection.encoding as well as the results > of > > select * from nls_database_parameters where instr(parameter, > 'CHARACTERSET') != 0; > > For example, with the Windows instant client: > > Oracle Instant client version 10.2.0.3 > connection.encoding = "WINDOWS-1252" > connection.nencoding = "WINDOWS-1252" > > PARAMETER VALUE > ------------------------------ ---------------------------------------- > NLS_NCHAR_CHARACTERSET AL16UTF16 > NLS_CHARACTERSET WE8MSWIN1252 > > Anthony > > On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé > <sa...@us...> wrote: >> Hi Anthony, >> >> thank you for this evolution of cx_Oracle! >> >> It tested it and it seems to work fine for output, however when doing >> some input I always get the same kind of error: >> >> $ python test_oracle.py >> Traceback (most recent call last): >> File "test_oracle.py", line 18, in <module> >> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >> >> Here is the trivial test I ran: >> >> import cx_Oracle >> >> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >> return cursor.var(unicode, size, cursor.arraysize) >> >> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >> connection.outputtypehandler = OutputTypeHandler >> cursor = connection.cursor() >> >> try: >> cursor.execute('drop table personne') >> except cx_Oracle.DatabaseError: >> pass >> >> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >> >> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >> >> cursor.execute("select * from personne") >> for row in cursor: >> print "Row:", row >> >> >> >> Is there something to configure in order to allow unicode strings input? >> Thanks in advance >> >> -- >> Sébastien Sablé >> >> >> Anthony Tuininga a écrit : >>> All, >>> >>> I just checked in to Subversion support for Unicode. Thanks go to >>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>> to everyone else for their patience. :-) Since I have little >>> experience with languages other than English and I only have access to >>> little endian machines I'd appreciate those of you with such >>> experience or machines to test the new code for me and let me know the >>> results. >>> >>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>> also possible to bind unicode strings in to cursors. SQL statements >>> and varchar2 and char data are still returned as encoded strings. Note >>> that with another new feature in cx_Oracle you can force all strings >>> to be returned as unicode if you so desire. Below is an example that >>> will run with the data in the test suite: >>> >>> import cx_Oracle >>> >>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>> return cursor.var(unicode, size, cursor.arraysize) >>> >>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>> connection.outputtypehandler = OutputTypeHandler >>> cursor = connection.cursor() >>> cursor.execute("select * from teststrings") >>> for row in cursor: >>> print "Row:", row >>> >>> If you have questions, let me know. If you have feedback, please >>> provide it. Thanks. >>> >>> Anthony >>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users |
From: Anthony T. <ant...@gm...> - 2008-10-02 17:05:00
|
Hmm, do you have access to a database with a server side fixed width character set? If not, I'll have to try creating a database with a variable width character set like UTF8 and try it -- but my time is stretched right now so it will have to wait if you want me to look into it. :-) Anthony On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé <sa...@us...> wrote: > Hi Anthony, > > I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on > sunos 10. > > Here are the parameters in my database: > > $ sqlplus SSA_UTF8/sungard@UTF102NP > > SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 > > Copyright (c) 1982, 2005, Oracle. All rights reserved. > > > Connected to: > Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production > With the Partitioning, OLAP and Data Mining options > > SQL> select * from nls_database_parameters where instr(parameter, > 'CHARACTERSET') != 0; > 2 > PARAMETER > ------------------------------ > VALUE > -------------------------------------------------------------------------------- > NLS_CHARACTERSET > UTF8 > > NLS_NCHAR_CHARACTERSET > UTF8 > > > So this may be linked to the different charset being used. > regards > > -- > Sébastien Sablé > > > Anthony Tuininga a écrit : >> Hi, >> >> I tried the code you provided as is and it worked for me. :-) I >> checked the error message and the cause claims to be that you cannot >> use OCI_ATTR_CHAR_COUNT when the server or client character set is >> variable length. I was using a fixed length character set so I tried a >> variable length character set but I still didn't get an error. I am >> using the Oracle 10.2.0.3 instant client on Windows and the full >> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any >> difficulty. Can anyone else tell me if they are getting the same >> behavior? Please specify the Oracle client being used and the value >> for connection.encoding and connection.encoding as well as the results >> of >> >> select * from nls_database_parameters where instr(parameter, >> 'CHARACTERSET') != 0; >> >> For example, with the Windows instant client: >> >> Oracle Instant client version 10.2.0.3 >> connection.encoding = "WINDOWS-1252" >> connection.nencoding = "WINDOWS-1252" >> >> PARAMETER VALUE >> ------------------------------ ---------------------------------------- >> NLS_NCHAR_CHARACTERSET AL16UTF16 >> NLS_CHARACTERSET WE8MSWIN1252 >> >> Anthony >> >> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé >> <sa...@us...> wrote: >>> Hi Anthony, >>> >>> thank you for this evolution of cx_Oracle! >>> >>> It tested it and it seems to work fine for output, however when doing >>> some input I always get the same kind of error: >>> >>> $ python test_oracle.py >>> Traceback (most recent call last): >>> File "test_oracle.py", line 18, in <module> >>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >>> >>> Here is the trivial test I ran: >>> >>> import cx_Oracle >>> >>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>> return cursor.var(unicode, size, cursor.arraysize) >>> >>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >>> connection.outputtypehandler = OutputTypeHandler >>> cursor = connection.cursor() >>> >>> try: >>> cursor.execute('drop table personne') >>> except cx_Oracle.DatabaseError: >>> pass >>> >>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >>> >>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>> >>> cursor.execute("select * from personne") >>> for row in cursor: >>> print "Row:", row >>> >>> >>> >>> Is there something to configure in order to allow unicode strings input? >>> Thanks in advance >>> >>> -- >>> Sébastien Sablé >>> >>> >>> Anthony Tuininga a écrit : >>>> All, >>>> >>>> I just checked in to Subversion support for Unicode. Thanks go to >>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>> to everyone else for their patience. :-) Since I have little >>>> experience with languages other than English and I only have access to >>>> little endian machines I'd appreciate those of you with such >>>> experience or machines to test the new code for me and let me know the >>>> results. >>>> >>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>>> also possible to bind unicode strings in to cursors. SQL statements >>>> and varchar2 and char data are still returned as encoded strings. Note >>>> that with another new feature in cx_Oracle you can force all strings >>>> to be returned as unicode if you so desire. Below is an example that >>>> will run with the data in the test suite: >>>> >>>> import cx_Oracle >>>> >>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>> return cursor.var(unicode, size, cursor.arraysize) >>>> >>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>>> connection.outputtypehandler = OutputTypeHandler >>>> cursor = connection.cursor() >>>> cursor.execute("select * from teststrings") >>>> for row in cursor: >>>> print "Row:", row >>>> >>>> If you have questions, let me know. If you have feedback, please >>>> provide it. Thanks. >>>> >>>> Anthony >>>> >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> cx-oracle-users mailing list >>>> cx-...@li... >>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
From: Sébastien S. <sa...@us...> - 2008-10-06 17:26:09
|
Hi Anthony, I did the test with a database using AL16UTF16 and it works fine. So I guess the problem only happens when using variable width character set. I will stick with AL16UTF16 for the moment. Thanks for your help -- Sébastien Sablé Anthony Tuininga a écrit : > Hmm, do you have access to a database with a server side fixed width > character set? If not, I'll have to try creating a database with a > variable width character set like UTF8 and try it -- but my time is > stretched right now so it will have to wait if you want me to look > into it. :-) > > Anthony > > On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé > <sa...@us...> wrote: >> Hi Anthony, >> >> I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on >> sunos 10. >> >> Here are the parameters in my database: >> >> $ sqlplus SSA_UTF8/sungard@UTF102NP >> >> SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 >> >> Copyright (c) 1982, 2005, Oracle. All rights reserved. >> >> >> Connected to: >> Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production >> With the Partitioning, OLAP and Data Mining options >> >> SQL> select * from nls_database_parameters where instr(parameter, >> 'CHARACTERSET') != 0; >> 2 >> PARAMETER >> ------------------------------ >> VALUE >> -------------------------------------------------------------------------------- >> NLS_CHARACTERSET >> UTF8 >> >> NLS_NCHAR_CHARACTERSET >> UTF8 >> >> >> So this may be linked to the different charset being used. >> regards >> >> -- >> Sébastien Sablé >> >> >> Anthony Tuininga a écrit : >>> Hi, >>> >>> I tried the code you provided as is and it worked for me. :-) I >>> checked the error message and the cause claims to be that you cannot >>> use OCI_ATTR_CHAR_COUNT when the server or client character set is >>> variable length. I was using a fixed length character set so I tried a >>> variable length character set but I still didn't get an error. I am >>> using the Oracle 10.2.0.3 instant client on Windows and the full >>> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any >>> difficulty. Can anyone else tell me if they are getting the same >>> behavior? Please specify the Oracle client being used and the value >>> for connection.encoding and connection.encoding as well as the results >>> of >>> >>> select * from nls_database_parameters where instr(parameter, >>> 'CHARACTERSET') != 0; >>> >>> For example, with the Windows instant client: >>> >>> Oracle Instant client version 10.2.0.3 >>> connection.encoding = "WINDOWS-1252" >>> connection.nencoding = "WINDOWS-1252" >>> >>> PARAMETER VALUE >>> ------------------------------ ---------------------------------------- >>> NLS_NCHAR_CHARACTERSET AL16UTF16 >>> NLS_CHARACTERSET WE8MSWIN1252 >>> >>> Anthony >>> >>> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé >>> <sa...@us...> wrote: >>>> Hi Anthony, >>>> >>>> thank you for this evolution of cx_Oracle! >>>> >>>> It tested it and it seems to work fine for output, however when doing >>>> some input I always get the same kind of error: >>>> >>>> $ python test_oracle.py >>>> Traceback (most recent call last): >>>> File "test_oracle.py", line 18, in <module> >>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >>>> >>>> Here is the trivial test I ran: >>>> >>>> import cx_Oracle >>>> >>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>> return cursor.var(unicode, size, cursor.arraysize) >>>> >>>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >>>> connection.outputtypehandler = OutputTypeHandler >>>> cursor = connection.cursor() >>>> >>>> try: >>>> cursor.execute('drop table personne') >>>> except cx_Oracle.DatabaseError: >>>> pass >>>> >>>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >>>> >>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>> >>>> cursor.execute("select * from personne") >>>> for row in cursor: >>>> print "Row:", row >>>> >>>> >>>> >>>> Is there something to configure in order to allow unicode strings input? >>>> Thanks in advance >>>> >>>> -- >>>> Sébastien Sablé >>>> >>>> >>>> Anthony Tuininga a écrit : >>>>> All, >>>>> >>>>> I just checked in to Subversion support for Unicode. Thanks go to >>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>>> to everyone else for their patience. :-) Since I have little >>>>> experience with languages other than English and I only have access to >>>>> little endian machines I'd appreciate those of you with such >>>>> experience or machines to test the new code for me and let me know the >>>>> results. >>>>> >>>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>>>> also possible to bind unicode strings in to cursors. SQL statements >>>>> and varchar2 and char data are still returned as encoded strings. Note >>>>> that with another new feature in cx_Oracle you can force all strings >>>>> to be returned as unicode if you so desire. Below is an example that >>>>> will run with the data in the test suite: >>>>> >>>>> import cx_Oracle >>>>> >>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>> >>>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>>>> connection.outputtypehandler = OutputTypeHandler >>>>> cursor = connection.cursor() >>>>> cursor.execute("select * from teststrings") >>>>> for row in cursor: >>>>> print "Row:", row >>>>> >>>>> If you have questions, let me know. If you have feedback, please >>>>> provide it. Thanks. >>>>> >>>>> Anthony >>>>> >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> cx-oracle-users mailing list >>>>> cx-...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> cx-oracle-users mailing list >>>> cx-...@li... >>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users |
From: Mark H. <mh...@pi...> - 2008-10-06 18:59:58
|
>>>>>> I just checked in to Subversion support for Unicode. Thanks go to >>>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>>>> to everyone else for their patience. :-) Yes, big thanks to everybody involved with this. Internationalization and Unicode are so hard, and having it done "by magic" at the lower level is really nice. After I moved back to the U.S. from working overseas, I told my wife, I want to work at a company that - uses a single language - and preferrably exists in a single timezone! |
From: Anthony T. <ant...@gm...> - 2008-10-14 04:54:16
|
Hi Sebastien, I just committed changes to use byte semantics instead of character semantics and all appears to be well -- but I'm not sure completely so I'd appreciate some testing. Note that I am also in the process of adding "full" unicode support in preparation for a port to Python 3.0. This implies that all strings are both expected in unicode and will be returned in unicode -- but its not complete yet. The test cases currently pass but there may still be some holes. Please advise if you find one. :-) Anthony On Mon, Oct 6, 2008 at 10:21 AM, Sébastien Sablé <sa...@us...> wrote: > Hi Anthony, > > I did the test with a database using AL16UTF16 and it works fine. > So I guess the problem only happens when using variable width character > set. I will stick with AL16UTF16 for the moment. > > Thanks for your help > > -- > Sébastien Sablé > > > > Anthony Tuininga a écrit : >> Hmm, do you have access to a database with a server side fixed width >> character set? If not, I'll have to try creating a database with a >> variable width character set like UTF8 and try it -- but my time is >> stretched right now so it will have to wait if you want me to look >> into it. :-) >> >> Anthony >> >> On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé >> <sa...@us...> wrote: >>> Hi Anthony, >>> >>> I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on >>> sunos 10. >>> >>> Here are the parameters in my database: >>> >>> $ sqlplus SSA_UTF8/sungard@UTF102NP >>> >>> SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 >>> >>> Copyright (c) 1982, 2005, Oracle. All rights reserved. >>> >>> >>> Connected to: >>> Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production >>> With the Partitioning, OLAP and Data Mining options >>> >>> SQL> select * from nls_database_parameters where instr(parameter, >>> 'CHARACTERSET') != 0; >>> 2 >>> PARAMETER >>> ------------------------------ >>> VALUE >>> -------------------------------------------------------------------------------- >>> NLS_CHARACTERSET >>> UTF8 >>> >>> NLS_NCHAR_CHARACTERSET >>> UTF8 >>> >>> >>> So this may be linked to the different charset being used. >>> regards >>> >>> -- >>> Sébastien Sablé >>> >>> >>> Anthony Tuininga a écrit : >>>> Hi, >>>> >>>> I tried the code you provided as is and it worked for me. :-) I >>>> checked the error message and the cause claims to be that you cannot >>>> use OCI_ATTR_CHAR_COUNT when the server or client character set is >>>> variable length. I was using a fixed length character set so I tried a >>>> variable length character set but I still didn't get an error. I am >>>> using the Oracle 10.2.0.3 instant client on Windows and the full >>>> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any >>>> difficulty. Can anyone else tell me if they are getting the same >>>> behavior? Please specify the Oracle client being used and the value >>>> for connection.encoding and connection.encoding as well as the results >>>> of >>>> >>>> select * from nls_database_parameters where instr(parameter, >>>> 'CHARACTERSET') != 0; >>>> >>>> For example, with the Windows instant client: >>>> >>>> Oracle Instant client version 10.2.0.3 >>>> connection.encoding = "WINDOWS-1252" >>>> connection.nencoding = "WINDOWS-1252" >>>> >>>> PARAMETER VALUE >>>> ------------------------------ ---------------------------------------- >>>> NLS_NCHAR_CHARACTERSET AL16UTF16 >>>> NLS_CHARACTERSET WE8MSWIN1252 >>>> >>>> Anthony >>>> >>>> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé >>>> <sa...@us...> wrote: >>>>> Hi Anthony, >>>>> >>>>> thank you for this evolution of cx_Oracle! >>>>> >>>>> It tested it and it seems to work fine for output, however when doing >>>>> some input I always get the same kind of error: >>>>> >>>>> $ python test_oracle.py >>>>> Traceback (most recent call last): >>>>> File "test_oracle.py", line 18, in <module> >>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >>>>> >>>>> Here is the trivial test I ran: >>>>> >>>>> import cx_Oracle >>>>> >>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>> >>>>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >>>>> connection.outputtypehandler = OutputTypeHandler >>>>> cursor = connection.cursor() >>>>> >>>>> try: >>>>> cursor.execute('drop table personne') >>>>> except cx_Oracle.DatabaseError: >>>>> pass >>>>> >>>>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >>>>> >>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>> >>>>> cursor.execute("select * from personne") >>>>> for row in cursor: >>>>> print "Row:", row >>>>> >>>>> >>>>> >>>>> Is there something to configure in order to allow unicode strings input? >>>>> Thanks in advance >>>>> >>>>> -- >>>>> Sébastien Sablé >>>>> >>>>> >>>>> Anthony Tuininga a écrit : >>>>>> All, >>>>>> >>>>>> I just checked in to Subversion support for Unicode. Thanks go to >>>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>>>> to everyone else for their patience. :-) Since I have little >>>>>> experience with languages other than English and I only have access to >>>>>> little endian machines I'd appreciate those of you with such >>>>>> experience or machines to test the new code for me and let me know the >>>>>> results. >>>>>> >>>>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>>>>> also possible to bind unicode strings in to cursors. SQL statements >>>>>> and varchar2 and char data are still returned as encoded strings. Note >>>>>> that with another new feature in cx_Oracle you can force all strings >>>>>> to be returned as unicode if you so desire. Below is an example that >>>>>> will run with the data in the test suite: >>>>>> >>>>>> import cx_Oracle >>>>>> >>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>>> >>>>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>>>>> connection.outputtypehandler = OutputTypeHandler >>>>>> cursor = connection.cursor() >>>>>> cursor.execute("select * from teststrings") >>>>>> for row in cursor: >>>>>> print "Row:", row >>>>>> >>>>>> If you have questions, let me know. If you have feedback, please >>>>>> provide it. Thanks. >>>>>> >>>>>> Anthony >>>>>> >>>>>> ------------------------------------------------------------------------- >>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>> _______________________________________________ >>>>>> cx-oracle-users mailing list >>>>>> cx-...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> cx-oracle-users mailing list >>>>> cx-...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>> >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> cx-oracle-users mailing list >>>> cx-...@li... >>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
From: Michael S. <ms...@co...> - 2008-10-02 15:32:21
|
Anthony Tuininga schrieb: > Hi, > > select * from nls_database_parameters where instr(parameter, > 'CHARACTERSET') != 0; > > For example, with the Windows instant client: > > Oracle Instant client version 10.2.0.3 > connection.encoding = "WINDOWS-1252" > connection.nencoding = "WINDOWS-1252" > > PARAMETER VALUE > ------------------------------ ---------------------------------------- > NLS_NCHAR_CHARACTERSET AL16UTF16 > NLS_CHARACTERSET WE8MSWIN1252 > Another thing that might be interesting is the value for NLS_LENGTH_SEMANTICS, it defaults to Bytes. Wouldn't be surprised if it was set to chars (which is more useful usually, unless you need to really allocate buffers in C). Michael -- Michael Schlenker Software Engineer CONTACT Software GmbH Tel.: +49 (421) 20153-80 Wiener Straße 1-3 Fax: +49 (421) 20153-41 28359 Bremen http://www.contact.de/ E-Mail: ms...@co... Sitz der Gesellschaft: Bremen Geschäftsführer: Karl Heinz Zachries, Ralf Holtgrefe Eingetragen im Handelsregister des Amtsgerichts Bremen unter HRB 13215 |
From: Anthony T. <ant...@gm...> - 2008-10-02 15:45:00
|
On Thu, Oct 2, 2008 at 9:32 AM, Michael Schlenker <ms...@co...> wrote: > Anthony Tuininga schrieb: >> Hi, >> >> select * from nls_database_parameters where instr(parameter, >> 'CHARACTERSET') != 0; >> >> For example, with the Windows instant client: >> >> Oracle Instant client version 10.2.0.3 >> connection.encoding = "WINDOWS-1252" >> connection.nencoding = "WINDOWS-1252" >> >> PARAMETER VALUE >> ------------------------------ ---------------------------------------- >> NLS_NCHAR_CHARACTERSET AL16UTF16 >> NLS_CHARACTERSET WE8MSWIN1252 >> > Another thing that might be interesting is the value for > NLS_LENGTH_SEMANTICS, it defaults to Bytes. Wouldn't be surprised if it was > set to chars (which is more useful usually, unless you need to really > allocate buffers in C). Mine is set to the default value of "BYTES". I am allocating buffers in C but fortunately this setting seems to have no bearing on what happens -- or are you suggesting otherwise? > Michael > > > -- > Michael Schlenker > Software Engineer > > CONTACT Software GmbH Tel.: +49 (421) 20153-80 > Wiener Straße 1-3 Fax: +49 (421) 20153-41 > 28359 Bremen > http://www.contact.de/ E-Mail: ms...@co... > > Sitz der Gesellschaft: Bremen > Geschäftsführer: Karl Heinz Zachries, Ralf Holtgrefe > Eingetragen im Handelsregister des Amtsgerichts Bremen unter HRB 13215 > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
From: Sébastien S. <sa...@us...> - 2008-10-17 10:00:15
|
Hi Anthony, thank you for those improvements! I don't expect to work again on Unicode in my application before a few weeks; but when I will, I will give you some feedback whether it works or not. For your information, I made a prototype of my application running in Unicode (with some Chinese text) for a demo which worked great. I used cx_Oracle revision 142 with a database encoded in AL32UTF8 (AL16UTF16 was working fine also; UTF-8 was not). regards -- Sébastien Sablé Anthony Tuininga a écrit : > Hi Sebastien, > > I just committed changes to use byte semantics instead of character > semantics and all appears to be well -- but I'm not sure completely so > I'd appreciate some testing. Note that I am also in the process of > adding "full" unicode support in preparation for a port to Python 3.0. > This implies that all strings are both expected in unicode and will be > returned in unicode -- but its not complete yet. The test cases > currently pass but there may still be some holes. Please advise if you > find one. :-) > > Anthony > > On Mon, Oct 6, 2008 at 10:21 AM, Sébastien Sablé > <sa...@us...> wrote: >> Hi Anthony, >> >> I did the test with a database using AL16UTF16 and it works fine. >> So I guess the problem only happens when using variable width character >> set. I will stick with AL16UTF16 for the moment. >> >> Thanks for your help >> >> -- >> Sébastien Sablé >> >> >> >> Anthony Tuininga a écrit : >>> Hmm, do you have access to a database with a server side fixed width >>> character set? If not, I'll have to try creating a database with a >>> variable width character set like UTF8 and try it -- but my time is >>> stretched right now so it will have to wait if you want me to look >>> into it. :-) >>> >>> Anthony >>> >>> On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé >>> <sa...@us...> wrote: >>>> Hi Anthony, >>>> >>>> I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on >>>> sunos 10. >>>> >>>> Here are the parameters in my database: >>>> >>>> $ sqlplus SSA_UTF8/sungard@UTF102NP >>>> >>>> SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 >>>> >>>> Copyright (c) 1982, 2005, Oracle. All rights reserved. >>>> >>>> >>>> Connected to: >>>> Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production >>>> With the Partitioning, OLAP and Data Mining options >>>> >>>> SQL> select * from nls_database_parameters where instr(parameter, >>>> 'CHARACTERSET') != 0; >>>> 2 >>>> PARAMETER >>>> ------------------------------ >>>> VALUE >>>> -------------------------------------------------------------------------------- >>>> NLS_CHARACTERSET >>>> UTF8 >>>> >>>> NLS_NCHAR_CHARACTERSET >>>> UTF8 >>>> >>>> >>>> So this may be linked to the different charset being used. >>>> regards >>>> >>>> -- >>>> Sébastien Sablé >>>> >>>> >>>> Anthony Tuininga a écrit : >>>>> Hi, >>>>> >>>>> I tried the code you provided as is and it worked for me. :-) I >>>>> checked the error message and the cause claims to be that you cannot >>>>> use OCI_ATTR_CHAR_COUNT when the server or client character set is >>>>> variable length. I was using a fixed length character set so I tried a >>>>> variable length character set but I still didn't get an error. I am >>>>> using the Oracle 10.2.0.3 instant client on Windows and the full >>>>> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any >>>>> difficulty. Can anyone else tell me if they are getting the same >>>>> behavior? Please specify the Oracle client being used and the value >>>>> for connection.encoding and connection.encoding as well as the results >>>>> of >>>>> >>>>> select * from nls_database_parameters where instr(parameter, >>>>> 'CHARACTERSET') != 0; >>>>> >>>>> For example, with the Windows instant client: >>>>> >>>>> Oracle Instant client version 10.2.0.3 >>>>> connection.encoding = "WINDOWS-1252" >>>>> connection.nencoding = "WINDOWS-1252" >>>>> >>>>> PARAMETER VALUE >>>>> ------------------------------ ---------------------------------------- >>>>> NLS_NCHAR_CHARACTERSET AL16UTF16 >>>>> NLS_CHARACTERSET WE8MSWIN1252 >>>>> >>>>> Anthony >>>>> >>>>> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé >>>>> <sa...@us...> wrote: >>>>>> Hi Anthony, >>>>>> >>>>>> thank you for this evolution of cx_Oracle! >>>>>> >>>>>> It tested it and it seems to work fine for output, however when doing >>>>>> some input I always get the same kind of error: >>>>>> >>>>>> $ python test_oracle.py >>>>>> Traceback (most recent call last): >>>>>> File "test_oracle.py", line 18, in <module> >>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >>>>>> >>>>>> Here is the trivial test I ran: >>>>>> >>>>>> import cx_Oracle >>>>>> >>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>>> >>>>>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >>>>>> connection.outputtypehandler = OutputTypeHandler >>>>>> cursor = connection.cursor() >>>>>> >>>>>> try: >>>>>> cursor.execute('drop table personne') >>>>>> except cx_Oracle.DatabaseError: >>>>>> pass >>>>>> >>>>>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >>>>>> >>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>>> >>>>>> cursor.execute("select * from personne") >>>>>> for row in cursor: >>>>>> print "Row:", row >>>>>> >>>>>> >>>>>> >>>>>> Is there something to configure in order to allow unicode strings input? >>>>>> Thanks in advance >>>>>> >>>>>> -- >>>>>> Sébastien Sablé >>>>>> >>>>>> >>>>>> Anthony Tuininga a écrit : >>>>>>> All, >>>>>>> >>>>>>> I just checked in to Subversion support for Unicode. Thanks go to >>>>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>>>>> to everyone else for their patience. :-) Since I have little >>>>>>> experience with languages other than English and I only have access to >>>>>>> little endian machines I'd appreciate those of you with such >>>>>>> experience or machines to test the new code for me and let me know the >>>>>>> results. >>>>>>> >>>>>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>>>>>> also possible to bind unicode strings in to cursors. SQL statements >>>>>>> and varchar2 and char data are still returned as encoded strings. Note >>>>>>> that with another new feature in cx_Oracle you can force all strings >>>>>>> to be returned as unicode if you so desire. Below is an example that >>>>>>> will run with the data in the test suite: >>>>>>> >>>>>>> import cx_Oracle >>>>>>> >>>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>>>> >>>>>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>>>>>> connection.outputtypehandler = OutputTypeHandler >>>>>>> cursor = connection.cursor() >>>>>>> cursor.execute("select * from teststrings") >>>>>>> for row in cursor: >>>>>>> print "Row:", row >>>>>>> >>>>>>> If you have questions, let me know. If you have feedback, please >>>>>>> provide it. Thanks. >>>>>>> >>>>>>> Anthony >>>>>>> >>>>>>> ------------------------------------------------------------------------- >>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>>> _______________________________________________ >>>>>>> cx-oracle-users mailing list >>>>>>> cx-...@li... >>>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>>> ------------------------------------------------------------------------- >>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>> _______________________________________________ >>>>>> cx-oracle-users mailing list >>>>>> cx-...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>>> >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> cx-oracle-users mailing list >>>>> cx-...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> cx-oracle-users mailing list >>>> cx-...@li... >>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users |
From: Anthony T. <ant...@gm...> - 2008-10-17 13:35:05
|
On Fri, Oct 17, 2008 at 2:59 AM, Sébastien Sablé <sa...@us...> wrote: > Hi Anthony, > > thank you for those improvements! You're welcome. > I don't expect to work again on Unicode in my application before a few > weeks; but when I will, I will give you some feedback whether it works > or not. Thanks. > For your information, I made a prototype of my application running in > Unicode (with some Chinese text) for a demo which worked great. I used > cx_Oracle revision 142 with a database encoded in AL32UTF8 (AL16UTF16 > was working fine also; UTF-8 was not). Please try again when you have a moment with the latest code. I believe UTF-8 encoded databases should work fine now as well but I'd appreciate confirmation of that fact. Thanks. > regards > > -- > Sébastien Sablé > > > > Anthony Tuininga a écrit : >> Hi Sebastien, >> >> I just committed changes to use byte semantics instead of character >> semantics and all appears to be well -- but I'm not sure completely so >> I'd appreciate some testing. Note that I am also in the process of >> adding "full" unicode support in preparation for a port to Python 3.0. >> This implies that all strings are both expected in unicode and will be >> returned in unicode -- but its not complete yet. The test cases >> currently pass but there may still be some holes. Please advise if you >> find one. :-) >> >> Anthony >> >> On Mon, Oct 6, 2008 at 10:21 AM, Sébastien Sablé >> <sa...@us...> wrote: >>> Hi Anthony, >>> >>> I did the test with a database using AL16UTF16 and it works fine. >>> So I guess the problem only happens when using variable width character >>> set. I will stick with AL16UTF16 for the moment. >>> >>> Thanks for your help >>> >>> -- >>> Sébastien Sablé >>> >>> >>> >>> Anthony Tuininga a écrit : >>>> Hmm, do you have access to a database with a server side fixed width >>>> character set? If not, I'll have to try creating a database with a >>>> variable width character set like UTF8 and try it -- but my time is >>>> stretched right now so it will have to wait if you want me to look >>>> into it. :-) >>>> >>>> Anthony >>>> >>>> On Thu, Oct 2, 2008 at 10:50 AM, Sébastien Sablé >>>> <sa...@us...> wrote: >>>>> Hi Anthony, >>>>> >>>>> I use a 10.2.0.1.0 client on linux, connecting to a 10.2.0.3.0 server on >>>>> sunos 10. >>>>> >>>>> Here are the parameters in my database: >>>>> >>>>> $ sqlplus SSA_UTF8/sungard@UTF102NP >>>>> >>>>> SQL*Plus: Release 10.2.0.1.0 - Production on Thu Oct 2 18:38:43 2008 >>>>> >>>>> Copyright (c) 1982, 2005, Oracle. All rights reserved. >>>>> >>>>> >>>>> Connected to: >>>>> Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production >>>>> With the Partitioning, OLAP and Data Mining options >>>>> >>>>> SQL> select * from nls_database_parameters where instr(parameter, >>>>> 'CHARACTERSET') != 0; >>>>> 2 >>>>> PARAMETER >>>>> ------------------------------ >>>>> VALUE >>>>> -------------------------------------------------------------------------------- >>>>> NLS_CHARACTERSET >>>>> UTF8 >>>>> >>>>> NLS_NCHAR_CHARACTERSET >>>>> UTF8 >>>>> >>>>> >>>>> So this may be linked to the different charset being used. >>>>> regards >>>>> >>>>> -- >>>>> Sébastien Sablé >>>>> >>>>> >>>>> Anthony Tuininga a écrit : >>>>>> Hi, >>>>>> >>>>>> I tried the code you provided as is and it worked for me. :-) I >>>>>> checked the error message and the cause claims to be that you cannot >>>>>> use OCI_ATTR_CHAR_COUNT when the server or client character set is >>>>>> variable length. I was using a fixed length character set so I tried a >>>>>> variable length character set but I still didn't get an error. I am >>>>>> using the Oracle 10.2.0.3 instant client on Windows and the full >>>>>> Oracle 11.1.0.6 client on Linux. Neither of these seem to have any >>>>>> difficulty. Can anyone else tell me if they are getting the same >>>>>> behavior? Please specify the Oracle client being used and the value >>>>>> for connection.encoding and connection.encoding as well as the results >>>>>> of >>>>>> >>>>>> select * from nls_database_parameters where instr(parameter, >>>>>> 'CHARACTERSET') != 0; >>>>>> >>>>>> For example, with the Windows instant client: >>>>>> >>>>>> Oracle Instant client version 10.2.0.3 >>>>>> connection.encoding = "WINDOWS-1252" >>>>>> connection.nencoding = "WINDOWS-1252" >>>>>> >>>>>> PARAMETER VALUE >>>>>> ------------------------------ ---------------------------------------- >>>>>> NLS_NCHAR_CHARACTERSET AL16UTF16 >>>>>> NLS_CHARACTERSET WE8MSWIN1252 >>>>>> >>>>>> Anthony >>>>>> >>>>>> On Thu, Oct 2, 2008 at 6:55 AM, Sébastien Sablé >>>>>> <sa...@us...> wrote: >>>>>>> Hi Anthony, >>>>>>> >>>>>>> thank you for this evolution of cx_Oracle! >>>>>>> >>>>>>> It tested it and it seems to work fine for output, however when doing >>>>>>> some input I always get the same kind of error: >>>>>>> >>>>>>> $ python test_oracle.py >>>>>>> Traceback (most recent call last): >>>>>>> File "test_oracle.py", line 18, in <module> >>>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>>>> cx_Oracle.DatabaseError: ORA-24363: measurements in characters illegal here >>>>>>> >>>>>>> Here is the trivial test I ran: >>>>>>> >>>>>>> import cx_Oracle >>>>>>> >>>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>>>> >>>>>>> connection = cx_Oracle.Connection('SSA_UTF8/sungard@UTF102NP') >>>>>>> connection.outputtypehandler = OutputTypeHandler >>>>>>> cursor = connection.cursor() >>>>>>> >>>>>>> try: >>>>>>> cursor.execute('drop table personne') >>>>>>> except cx_Oracle.DatabaseError: >>>>>>> pass >>>>>>> >>>>>>> cursor.execute('create table personne (name nchar(4), first_name nchar(6))') >>>>>>> >>>>>>> cursor.execute('insert into personne values(:0, :1)', [u'abc', u'def']) >>>>>>> >>>>>>> cursor.execute("select * from personne") >>>>>>> for row in cursor: >>>>>>> print "Row:", row >>>>>>> >>>>>>> >>>>>>> >>>>>>> Is there something to configure in order to allow unicode strings input? >>>>>>> Thanks in advance >>>>>>> >>>>>>> -- >>>>>>> Sébastien Sablé >>>>>>> >>>>>>> >>>>>>> Anthony Tuininga a écrit : >>>>>>>> All, >>>>>>>> >>>>>>>> I just checked in to Subversion support for Unicode. Thanks go to >>>>>>>> Amaury Forgeot d'Arc for the patch he provided earlier this year and >>>>>>>> to everyone else for their patience. :-) Since I have little >>>>>>>> experience with languages other than English and I only have access to >>>>>>>> little endian machines I'd appreciate those of you with such >>>>>>>> experience or machines to test the new code for me and let me know the >>>>>>>> results. >>>>>>>> >>>>>>>> Note that now nchar, nvarchar2 data are now returned as unicode. It is >>>>>>>> also possible to bind unicode strings in to cursors. SQL statements >>>>>>>> and varchar2 and char data are still returned as encoded strings. Note >>>>>>>> that with another new feature in cx_Oracle you can force all strings >>>>>>>> to be returned as unicode if you so desire. Below is an example that >>>>>>>> will run with the data in the test suite: >>>>>>>> >>>>>>>> import cx_Oracle >>>>>>>> >>>>>>>> def OutputTypeHandler(cursor, name, defaultType, size, precision, scale): >>>>>>>> if defaultType in (cx_Oracle.STRING, cx_Oracle.FIXED_CHAR): >>>>>>>> return cursor.var(unicode, size, cursor.arraysize) >>>>>>>> >>>>>>>> connection = cx_Oracle.Connection("cx_Oracle/dev@eadev") >>>>>>>> connection.outputtypehandler = OutputTypeHandler >>>>>>>> cursor = connection.cursor() >>>>>>>> cursor.execute("select * from teststrings") >>>>>>>> for row in cursor: >>>>>>>> print "Row:", row >>>>>>>> >>>>>>>> If you have questions, let me know. If you have feedback, please >>>>>>>> provide it. Thanks. >>>>>>>> >>>>>>>> Anthony >>>>>>>> >>>>>>>> ------------------------------------------------------------------------- >>>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>>>> _______________________________________________ >>>>>>>> cx-oracle-users mailing list >>>>>>>> cx-...@li... >>>>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>>>> ------------------------------------------------------------------------- >>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>>> _______________________________________________ >>>>>>> cx-oracle-users mailing list >>>>>>> cx-...@li... >>>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>>>> >>>>>> ------------------------------------------------------------------------- >>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>>> _______________________________________________ >>>>>> cx-oracle-users mailing list >>>>>> cx-...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> cx-oracle-users mailing list >>>>> cx-...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>>>> >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> cx-oracle-users mailing list >>>> cx-...@li... >>>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> cx-oracle-users mailing list >>> cx-...@li... >>> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >>> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >> Build the coolest Linux based applications with Moblin SDK & win great prizes >> Grand prize is a trip for two to an Open Source event anywhere in the world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |