[cx-oracle-users] Extracting data in Native Language: Error
Brought to you by:
atuining
Menu
▾
▴
[cx-oracle-users] Extracting data in Native Language: Error
|
From: Robert L. <rlu...@pi...> - 2011-10-09 11:26:58
|
Check out the Oracle manual Oracle Database Globalization Support Guide http://download.oracle.com/docs/cd/B19306_01/server.102/b14225/toc.htm Also, the manual "Oracle Database SQL Reference" http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/sql_elements0 01.htm#i45694 For performance reasons, Oracle recommends that you use the NLS_LENGTH_SEMANTICS parameter to set length semantics and that you use the BYTE and CHAR qualifiers only when necessary to override the parameter. -----Original Message----- From: cx-...@li... [mailto:cx-...@li...] Sent: Sunday, October 09, 2011 2:51 AM To: cx-...@li... Subject: cx-oracle-users Digest, Vol 64, Issue 2 Send cx-oracle-users mailing list submissions to cx-...@li... To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/cx-oracle-users or, via email, send a message with subject or body 'help' to cx-...@li... You can reach the person managing the list at cx-...@li... When replying, please edit your Subject line so it is more specific than "Re: Contents of cx-oracle-users digest..." Today's Topics: 1. Re: Extracting data in Native Language: Error UnicodeEncodeError (Chris Gould) 2. Re: ORA-24550: signal received (Wong Wah Meng-R32813) ---------------------------------------------------------------------- Message: 1 Date: Thu, 6 Oct 2011 12:43:52 +0100 From: Chris Gould <chr...@to...> Subject: Re: [cx-oracle-users] Extracting data in Native Language: Error UnicodeEncodeError To: cx-...@li... Message-ID: <CA+...@ma...> Content-Type: text/plain; charset="utf-8" Just for reference, if you've using Oracle 10 or later you don't need to use NVARCHAR2 to store multi-byte characters. You can use VARCHAR2 (x CHAR) instead. The "x" here limits the field to a certain number of characters, not bytes. The default for VARCHAR2 is that the length specified is bytes unless you specify the CHAR qualifier. However, you can change this at system-level by setting NLS_LENGTH_SEMANTICS to CHAR. That way, if you declare a field as VARCHAR2(10) say, it will default to being 10 CHAR, not 10 BYTE. Even if you've already set up some VARCHAR2 fields, you can easily change the definitions to switch them all to CHAR. Below is a script to do this which I've used myself on databases. The nice thing about doing this is you can then carry on defining fields in future as VARCHAR2(10) say, and know that it'll hold 10 characters, regardless of how many bytes each character is, and you don't need to bother with NVARCHAR at all. I've used this approach on several systems and I never give any thought these days to what characters are being stored. Note - if you're using partitioning and have already got partition key columns which are VARCHAR2, trying to change from BYTE to CHAR will give an exception. The script picks this up and reports it. -- -- this is to change dictionary definitions for existing cols from BYTE to CHAR -- The database should have already had NLS_LENGTH_SEMANTICS = CHAR set -- This has to be run in each application schema that has tables. (not in any Oracle schemas like SYS or SYSTEM) -- ORA-14060 - cannot change length of partitioning column. -- So this needs to be done before any tables created where partitioning is on an affected (varchar2 or char) col alter session set NLS_LENGTH_SEMANTICS = CHAR; declare v_stmt varchar2(1000); PARTITIONING_COLUMN EXCEPTION; pragma exception_init(PARTITIONING_COLUMN,-14060); begin dbms_output.enable(NULL); for r1 in (select table_name ,column_name ,data_type ,data_length ,char_length from USER_TAB_COLUMNS where column_id is not null /* excludes oracle-generated cols for func-based indexes */ and data_type in ('VARCHAR2','CHAR') and char_used = 'B' /* defined as byte length */ order by table_name asc,column_id asc) loop begin v_stmt := 'alter table '||r1.table_name||' modify '||r1.column_name||' '||r1.data_type||'('||r1.char_length||')'; execute immediate v_stmt; dbms_output.put_line(substr(v_stmt,1,250)); exception when PARTITIONING_COLUMN then dbms_output.put_line('+++ '||r1.table_name||'.'||r1.column_name||' is partitioning col and cannot be changed'); when others then NULL; end; end loop; end; / On 6 October 2011 08:50, Wong Wah Meng-R32813 <r3...@fr...> wrote: > Hello there,**** > > ** ** > > I am migrating my application from python 1.5.2 to 2.7.1, from using > oracledb to cx_oracle. **** > > ** ** > > My database is setup with AL32UTF8 characterset, in order to support > Chinese character entry. **** > > ** ** > > I am in testing stage now. I have never used nVarchar type before as my > column type but now I am using it so that I can be sure exactly the number > of bytes the field can store is predictable regardless it is plain English > or Unicode characters.**** > > ** ** > > When I do insert and update, I do not explicitly pass in some prefix (u for > unicode) onto the data in my SQL statement. However, when I do a SQL query, > the return list tuple from python cx_Oracle seems to attach a prefix u in > front of the data with column type nVarchar. When I want to extract or parse > the data, I encountered an error (expectedly). I am new to this can anyone > tell what I should look at in order to parse native language characters > successfully after getting the raw reply from a cx_Oracle?s query?**** > > ** ** > > E.g. **** > > ** ** > > >>> reply=a.execute("select * from TEST271")**** > > >>> reply**** > > [('\xb2\xe2\xca\xd4', u'\u6d4b\u8bd5')]**** > > >>> for (text1, text2) in reply:**** > > ... print text1, text2**** > > ... **** > > ???? Traceback (most recent call last):**** > > File "<stdin>", line 2, in <module>**** > > File > "/home/r32813/genesis/GEN_DEV_271/Enablers/Python/lib/python2.7/encodings/hp _roman8.py", > line 17, in encode**** > > return codecs.charmap_encode(input,errors,encoding_map)**** > > UnicodeEncodeError: 'charmap' codec can't encode characters in position > 0-1: character maps to <undefined>**** > > >>> **** > > ** ** > > SQL> desc test271**** > > Name Null? Type**** > > ----------------------------------------- -------- > ----------------------------**** > > TEXT1 VARCHAR2(16)**** > > TEXT2 NVARCHAR2(16)**** > > **** > > ** ** > > Regards,**** > > Wah Meng**** > > ** ** > > > -------------------------------------------------------------------------- ---- > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ Message: 2 Date: Sun, 9 Oct 2011 07:50:58 +0000 From: Wong Wah Meng-R32813 <r3...@fr...> Subject: Re: [cx-oracle-users] ORA-24550: signal received To: "cx-...@li..." <cx-...@li...> Message-ID: <02E...@03...> Content-Type: text/plain; charset="us-ascii" Hello Anthony, I am not sure what happened when I first plug in conn = cx_Oracle.connect(<connection string>, threaded=True) it didn't seem to work, however when I tried again it worked. It works until now, even though I have removed the "threaded=True" from the connect() argument list in my source code. I have stopped and started my process multiple times and it still works without failure (no more signal failed error), when threaded=True is omitted. Though I have not seen any difference now by including or omitting it from the argument list, I have decided to include it into my source code. I need to revisit this if on another day bad luck strike me that it gives me that error again. :p Thanks a lot! Regards, Wah Meng -----Original Message----- From: Anthony Tuininga [mailto:ant...@gm...] Sent: Wednesday, October 05, 2011 12:29 PM To: cx-...@li... Subject: Re: [cx-oracle-users] ORA-24550: signal received Hi, I mean threaded = True when creating the connection to the database. Anthony On Tue, Oct 4, 2011 at 9:48 PM, Wong Wah Meng-R32813 <r3...@fr...> wrote: > Hello Anthony, > > Do you mean I can pass in threaded=true when creating the connection to database (cx_Oracle.connect method)? Or do you mean the --enable-thread setting during the build of the python executable? > > Regards, > Wah Meng > > -----Original Message----- > From: Anthony Tuininga [mailto:ant...@gm...] > Sent: Tuesday, October 04, 2011 9:58 PM > To: cx-...@li... > Subject: Re: [cx-oracle-users] ORA-24550: signal received > > Hi, > > If you have multiple threads, make sure that you specify threaded = > True when creating the connection or you will run into difficulties. > The reason this is not default is that there is a performance penalty > that is unnecessary when running single threaded -- which is the most > common mode anyway. > > Hope that helps. > > Anthony > > On Tue, Oct 4, 2011 at 4:27 AM, Wong Wah Meng-R32813 > <r3...@fr...> wrote: >> Hello guys, >> >> >> >> Has anyone seen this befote? >> >> >> >> I am migrating my RMI application from python 1.5.2 (oracledb) to 2.7.1 >> (cx_Oracle). When I start up my DbAccessMgr, previously utilizing oracledb >> to connect to database, I encountered below error. I haven't traced down to >> exactly which code is producing this error however it seems that this is >> generated after I spawn off 4 parallel threads instance for the database >> connection from my application. >> >> >> >> The funnier thing is I am able to launch my DbAccessMgr successfully without >> this error after every 2 failures. My DBA told me this should be application >> specific unless I can show otherwise. Appreciate any input out there. >> Thanks! >> >> >> >> (PS: My platform in HP-UX11.31 and cx_Oracle was compiled against Oracle >> 11.1 library using HP-UX C Compiler) >> >> >> >> ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=2] >> [si_addr=0000000000000000] >> >> >> >> sendsig: useracc failed. 0x9fffffff5feb6200 0x00000000005000 >> >> >> >> Pid 8162 was killed due to failure in writing the signal context - possible >> stack overflow. >> >> >> >> Regards, >> >> Wah Meng >> >> >> >> ------------------------------------------------------------------------- ----- >> All the data continuously generated in your IT infrastructure contains a >> definitive record of customers, application performance, security >> threats, fraudulent activity and more. Splunk takes this data and makes >> sense of it. Business sense. IT sense. Common sense. >> http://p.sf.net/sfu/splunk-d2dcopy1 >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> >> > > -------------------------------------------------------------------------- ---- > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > > > -------------------------------------------------------------------------- ---- > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > ---------------------------------------------------------------------------- -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 _______________________________________________ cx-oracle-users mailing list cx-...@li... https://lists.sourceforge.net/lists/listinfo/cx-oracle-users ------------------------------ ---------------------------------------------------------------------------- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2dcopy2 ------------------------------ _______________________________________________ cx-oracle-users mailing list cx-...@li... https://lists.sourceforge.net/lists/listinfo/cx-oracle-users End of cx-oracle-users Digest, Vol 64, Issue 2 ********************************************** |
