Re: [cx-oracle-users] Problems with UTF8 and unicode version of cx_Oracle

[Anthony T. <ant...@gm...>]

Hi,

First of all, you should be able to use the standard version of
cx_Oracle to bind data for insert as unicode strings. I do that myself
all of the time. If the field is nvarchar2 or nclob you will also get
the data returned as unicode strings. Its only normal strings
(varchar2 and char) that are returned as encoded strings and not
unicode strings. I would suggest you try this first to see if there is
some problem with the unicode mode version of cx_Oracle. In the past
year or so I have discovered a number of problems that suggest that
mode is not particularly well supported by Oracle. I am considering
removing it and simply enabling better unicode support in the normal
version -- just got to find the time to test that and see if it will
work reasonably well.

Beyond that, you need to check the database encodings as Oracle will
feel free to substitute characters if the database encoding does not
support what is coming in.

Anthony

On Wed, Jul 7, 2010 at 6:01 AM, Stefan Dietrich <ste...@de...> wrote:
> Hello,
>
> I´m using the cx_Oracle module for importing data from another database
> (ZODB), but I have problems with the character encoding.
> The data in ZODB is entered through a web form, so the character
> encoding in the ZODB is UTF-8.
>
> Everything is fine until special characters appear or characters from
> foreign languages, e.g. a dash, cyrillic characters or the German letter
> "ß". However, German umlauts (üäö) are not affected.
> If I use "print" to display them, everything is fine.
>
> But if I try to insert them into the Oracle DB, they appear as an
> inverted question mark in the Oracle SQL Developer or SQLPlus.
> If I query my Oracle DB again with cx_Oracle, they appear as inverted
> questionmarks.
>
> Example (trying to insert data with a dash):
>
>     title = conference.getTitle().decode('utf-8')
>     print title
> 2nd Workshop: GISAXS – an advanced scattering method
>     title
> u'2nd Workshop: GISAXS \u2013 an advanced scattering method'
>     cursor.execute(u"INSERT INTO conf (CID, title) VALUES (4, '"
> +title+ "')")
>     conn.commit()
>     cursor.execute(u"SELECT * FROM conf WHERE CID = 4")
>     fetch = cursor.fetchall()
>     fetch[0][1]
> u'2nd Workshop: GISAXS \xbf an advanced scattering method'
>     print fetch[0][1]
> 2nd Workshop: GISAXS ¿ an advanced scattering method
>
>
> If I copy&paste the output of print with the dash and insert the data
> with Oracle SQLDeveloper, everthing is correct. Even cx_Oracle returns
> the dash after a select.
>
> I´m using cx_Oracle 5.0.3 Unicode Version with Python 2.4.3 on
> Scientific Linux 5.3. The field is NVARCHAR2 (happens also with
> VARCHAR), NLS_LANG is set to GERMAN_GERMANY.UTF8 and
> NLS_NCHAR_CHARACTERSET is UTF8.
>
> I´m out of ideas, so I would appreciate any help and thanks in advance!
>
> Regards,
> Stefan
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>