cx-oracle-users Mailing List for cx_Oracle (Page 8)
Brought to you by:
atuining
Menu
▾
▴
cx-oracle-users — Primary discussion list for users (and developers) of cx_Oracle
You can subscribe to this list here.
| 2003 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(5) |
Aug
(9) |
Sep
(8) |
Oct
(12) |
Nov
(4) |
Dec
(8) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2004 |
Jan
(15) |
Feb
(12) |
Mar
(11) |
Apr
(5) |
May
(7) |
Jun
(8) |
Jul
(12) |
Aug
(2) |
Sep
(14) |
Oct
(17) |
Nov
(20) |
Dec
(3) |
| 2005 |
Jan
(16) |
Feb
(9) |
Mar
(22) |
Apr
(21) |
May
(73) |
Jun
(16) |
Jul
(15) |
Aug
(10) |
Sep
(32) |
Oct
(35) |
Nov
(22) |
Dec
(13) |
| 2006 |
Jan
(42) |
Feb
(36) |
Mar
(13) |
Apr
(18) |
May
(8) |
Jun
(17) |
Jul
(24) |
Aug
(30) |
Sep
(35) |
Oct
(33) |
Nov
(33) |
Dec
(11) |
| 2007 |
Jan
(35) |
Feb
(31) |
Mar
(35) |
Apr
(64) |
May
(38) |
Jun
(12) |
Jul
(18) |
Aug
(34) |
Sep
(75) |
Oct
(29) |
Nov
(51) |
Dec
(11) |
| 2008 |
Jan
(27) |
Feb
(46) |
Mar
(48) |
Apr
(36) |
May
(59) |
Jun
(42) |
Jul
(25) |
Aug
(34) |
Sep
(57) |
Oct
(97) |
Nov
(59) |
Dec
(57) |
| 2009 |
Jan
(48) |
Feb
(48) |
Mar
(45) |
Apr
(24) |
May
(46) |
Jun
(52) |
Jul
(52) |
Aug
(37) |
Sep
(27) |
Oct
(40) |
Nov
(37) |
Dec
(13) |
| 2010 |
Jan
(16) |
Feb
(9) |
Mar
(24) |
Apr
(6) |
May
(27) |
Jun
(28) |
Jul
(60) |
Aug
(16) |
Sep
(33) |
Oct
(20) |
Nov
(39) |
Dec
(30) |
| 2011 |
Jan
(23) |
Feb
(43) |
Mar
(16) |
Apr
(29) |
May
(23) |
Jun
(16) |
Jul
(10) |
Aug
(8) |
Sep
(18) |
Oct
(42) |
Nov
(26) |
Dec
(20) |
| 2012 |
Jan
(17) |
Feb
(27) |
Mar
|
Apr
(20) |
May
(18) |
Jun
(7) |
Jul
(24) |
Aug
(21) |
Sep
(23) |
Oct
(18) |
Nov
(12) |
Dec
(5) |
| 2013 |
Jan
(14) |
Feb
(10) |
Mar
(20) |
Apr
(65) |
May
(3) |
Jun
(8) |
Jul
(6) |
Aug
(3) |
Sep
|
Oct
(3) |
Nov
(28) |
Dec
(3) |
| 2014 |
Jan
(3) |
Feb
(9) |
Mar
(4) |
Apr
(7) |
May
(20) |
Jun
(2) |
Jul
(20) |
Aug
(7) |
Sep
(11) |
Oct
(8) |
Nov
(6) |
Dec
(12) |
| 2015 |
Jan
(16) |
Feb
(10) |
Mar
(14) |
Apr
(8) |
May
|
Jun
(8) |
Jul
(15) |
Aug
(7) |
Sep
(1) |
Oct
(33) |
Nov
(8) |
Dec
(5) |
| 2016 |
Jan
(18) |
Feb
(12) |
Mar
(6) |
Apr
(14) |
May
(5) |
Jun
(3) |
Jul
|
Aug
(21) |
Sep
|
Oct
(15) |
Nov
(8) |
Dec
|
| 2017 |
Jan
|
Feb
(14) |
Mar
(21) |
Apr
(9) |
May
(6) |
Jun
(11) |
Jul
(23) |
Aug
(6) |
Sep
(5) |
Oct
(7) |
Nov
(1) |
Dec
(1) |
| 2018 |
Jan
|
Feb
|
Mar
(16) |
Apr
(2) |
May
(1) |
Jun
|
Jul
(2) |
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
(2) |
Feb
(3) |
Mar
(1) |
Apr
(1) |
May
|
Jun
|
Jul
(2) |
Aug
(1) |
Sep
(2) |
Oct
|
Nov
|
Dec
(1) |
| 2020 |
Jan
|
Feb
(4) |
Mar
|
Apr
|
May
(2) |
Jun
(1) |
Jul
(4) |
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
(3) |
| 2021 |
Jan
|
Feb
(5) |
Mar
|
Apr
(7) |
May
(6) |
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
| 2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2023 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Christopher J. <chr...@or...> - 2016-11-09 01:03:52
|
In particular, check you have correct 32 or 64 bit Oracle Client libraries. They need to match Python (and cx_Oracle). Chris On 9/11/2016 10:44 AM, Adam Kerz wrote: > Have you installed the correct version of the Oracle client and is it on > the path? > > eg. > > PATH=C:\oracle\x86-32\product\12.1.0\client_1\bin;... > > That's ALWAYS been my problem when I've gotten that exception from > cx_Oracle. > > Adam > > > On 9/11/2016 2:14 AM, Codiga, Daniel wrote: >> To be a little more complete regarding my final comment, … >> >> >> >> When I create the environment using python 3.4, it appears that >> cx_Oracle 5.2 has been installed successfully (according to Anaconda >> Navigator, at least). Then when I try “import cx_Oracle” the problem is >> not exactly that it can’t find the win32 file; the error message is this: >> >> >> >> ImportError: DLL load failed: %1 is not a valid Win32 application. >> >> >> >> >> >> >> >> *From:*Codiga, Daniel >> *Sent:* Monday, November 07, 2016 3:55 PM >> *To:* 'cx-...@li...' >> *Subject:* anaconda python 3.5 + windows 7 + cx_Oracle 5.2? >> >> >> >> From what I have read in the documentation, it looks like I should be >> able to install anaconda python 3.5 on my windows 7 64-bit PC and >> include cx_Oracle 5.2 in a conda virtual environment. >> >> >> >> However I am encountering problems in trying. >> >> >> >> ( Previously I was using Anaconda python 2.7 with cx_Oracle 5.1.2 in a >> conda environment without trouble. I mention this because it seems >> possible that my problems are associated with not fully/successfully >> uninstalling cx_Oracle 5.1.2? Just a thought.) >> >> >> >> Anyway I do a clean wipe of python from my system and then install >> Miniconda (64 bit windows) and all is well. I then try to create an >> environment “conda env create environment.yml” -- with “python=3.5” and >> “cx_Oracle” both listed as dependencies in the environment.yml file-- >> and get the “specifications were found to be in conflict” error. It says >> “ – cx_oracle -> python 2.7*|3.4*” and “ – python 3.5*”. >> >> >> >> So then, to troubleshoot a little, I try same but with “python=3.4” in >> the environment.yml file. The environment gets created ok. But when I >> try “import cx_Oracle” I get an error saying it couldn’t find the win32 >> file. >> >> >> >> Suggestions? >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Developer Access Program for Intel Xeon Phi Processors >> Access to Intel Xeon Phi processor-based developer platforms. >> With one year of Intel Parallel Studio XE. >> Training and support from Colfax. >> Order your platform today. http://sdm.link/xeonphi >> >> >> >> _______________________________________________ >> cx-oracle-users mailing list >> cx-...@li... >> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users >> > > > ------------------------------------------------------------------------------ > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > -- http://twitter.com/ghrd |
|
From: Adam K. <cx_...@ke...> - 2016-11-09 00:01:49
|
Have you installed the correct version of the Oracle client and is it on the path? eg. PATH=C:\oracle\x86-32\product\12.1.0\client_1\bin;... That's ALWAYS been my problem when I've gotten that exception from cx_Oracle. Adam On 9/11/2016 2:14 AM, Codiga, Daniel wrote: > To be a little more complete regarding my final comment, … > > > > When I create the environment using python 3.4, it appears that > cx_Oracle 5.2 has been installed successfully (according to Anaconda > Navigator, at least). Then when I try “import cx_Oracle” the problem is > not exactly that it can’t find the win32 file; the error message is this: > > > > ImportError: DLL load failed: %1 is not a valid Win32 application. > > > > > > > > *From:*Codiga, Daniel > *Sent:* Monday, November 07, 2016 3:55 PM > *To:* 'cx-...@li...' > *Subject:* anaconda python 3.5 + windows 7 + cx_Oracle 5.2? > > > > From what I have read in the documentation, it looks like I should be > able to install anaconda python 3.5 on my windows 7 64-bit PC and > include cx_Oracle 5.2 in a conda virtual environment. > > > > However I am encountering problems in trying. > > > > ( Previously I was using Anaconda python 2.7 with cx_Oracle 5.1.2 in a > conda environment without trouble. I mention this because it seems > possible that my problems are associated with not fully/successfully > uninstalling cx_Oracle 5.1.2? Just a thought.) > > > > Anyway I do a clean wipe of python from my system and then install > Miniconda (64 bit windows) and all is well. I then try to create an > environment “conda env create environment.yml” -- with “python=3.5” and > “cx_Oracle” both listed as dependencies in the environment.yml file-- > and get the “specifications were found to be in conflict” error. It says > “ – cx_oracle -> python 2.7*|3.4*” and “ – python 3.5*”. > > > > So then, to troubleshoot a little, I try same but with “python=3.4” in > the environment.yml file. The environment gets created ok. But when I > try “import cx_Oracle” I get an error saying it couldn’t find the win32 > file. > > > > Suggestions? > > > > > > > > ------------------------------------------------------------------------------ > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > > > > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
|
From: Codiga, D. <Dan...@mw...> - 2016-11-08 16:14:14
|
To be a little more complete regarding my final comment, ... When I create the environment using python 3.4, it appears that cx_Oracle 5.2 has been installed successfully (according to Anaconda Navigator, at least). Then when I try "import cx_Oracle" the problem is not exactly that it can't find the win32 file; the error message is this: ImportError: DLL load failed: %1 is not a valid Win32 application. From: Codiga, Daniel Sent: Monday, November 07, 2016 3:55 PM To: 'cx-...@li...' Subject: anaconda python 3.5 + windows 7 + cx_Oracle 5.2? >From what I have read in the documentation, it looks like I should be able to install anaconda python 3.5 on my windows 7 64-bit PC and include cx_Oracle 5.2 in a conda virtual environment. However I am encountering problems in trying. ( Previously I was using Anaconda python 2.7 with cx_Oracle 5.1.2 in a conda environment without trouble. I mention this because it seems possible that my problems are associated with not fully/successfully uninstalling cx_Oracle 5.1.2? Just a thought.) Anyway I do a clean wipe of python from my system and then install Miniconda (64 bit windows) and all is well. I then try to create an environment "conda env create environment.yml" -- with "python=3.5" and "cx_Oracle" both listed as dependencies in the environment.yml file-- and get the "specifications were found to be in conflict" error. It says " - cx_oracle -> python 2.7*|3.4*" and " - python 3.5*". So then, to troubleshoot a little, I try same but with "python=3.4" in the environment.yml file. The environment gets created ok. But when I try "import cx_Oracle" I get an error saying it couldn't find the win32 file. Suggestions? |
|
From: Codiga, D. <Dan...@mw...> - 2016-11-07 21:29:14
|
>From what I have read in the documentation, it looks like I should be able to install anaconda python 3.5 on my windows 7 64-bit PC and include cx_Oracle 5.2 in a conda virtual environment. However I am encountering problems in trying. ( Previously I was using Anaconda python 2.7 with cx_Oracle 5.1.2 in a conda environment without trouble. I mention this because it seems possible that my problems are associated with not fully/successfully uninstalling cx_Oracle 5.1.2? Just a thought.) Anyway I do a clean wipe of python from my system and then install Miniconda (64 bit windows) and all is well. I then try to create an environment "conda env create environment.yml" -- with "python=3.5" and "cx_Oracle" both listed as dependencies in the environment.yml file-- and get the "specifications were found to be in conflict" error. It says " - cx_oracle -> python 2.7*|3.4*" and " - python 3.5*". So then, to troubleshoot a little, I try same but with "python=3.4" in the environment.yml file. The environment gets created ok. But when I try "import cx_Oracle" I get an error saying it couldn't find the win32 file. Suggestions? |
|
From: Anthony T. <ant...@gm...> - 2016-10-27 19:40:08
|
You're right, that isn't terribly obvious. Log an issue, here: https://bitbucket.org/anthony_tuininga/cx_oracle/issues?status=new&status=open If you have a solution to the problem, create a pull request. I'll make a note to improve that. :-) Anthony On Thu, Oct 27, 2016 at 11:02 AM, Cjstudios <cjs...@gm...> wrote: > Apologies for the noob question but where does one submit bugs to > CX_oracle? Not obvious from the site. > > > > ------------------------------------------------------------ > ------------------ > The Command Line: Reinvented for Modern Developers > Did the resurgence of CLI tooling catch you by surprise? > Reconnect with the command line and become more productive. > Learn the new .NET and ASP.NET CLI. Get your free copy! > http://sdm.link/telerik > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
|
From: Cjstudios <cjs...@gm...> - 2016-10-27 17:02:49
|
Apologies for the noob question but where does one submit bugs to CX_oracle? Not obvious from the site. |
|
From: James C. M. <Jam...@or...> - 2016-10-25 22:11:10
|
Hi Fred, Great to see that you've found the root cause. When I looked at your stack trace, these lines stood out: #0 0x0000003c9ee0f6ab in raise () from /lib64/libpthread.so.0 #1 <signal handler called> #2 0x00007f5ab2f3ddff in __intel_ssse3_rep_memcpy () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #3 0x00007f5ab2f36cc6 in _intel_fast_memcpy.P () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 because in my experience a segv in such a lowlevel routine indicates that the cpu is being mis-detected. Before I could suggest checking your hardware, you figured it out. James On 26/10/16 07:48 am, Fred Yankowski wrote: > It now appears that my problem was caused by migrating the server from a > VMware 4 host to a VMware 5 host without upgrading the "hardware version" > of the virtual machine. > > I totally forgot that our system admin team had done that about a week > ago. The server otherwise worked exactly the same after that migration. It > may have required a restart of Apache after the migration for the Oracle > connection error to start appearing. > > Our lead sys admin just migrated my server temporarily back to a VMware 4 > host and I can no longer reproduce the error there. > > On a matching dev server running on a VMware 5 host we just upgraded the > virtual hardware version and now I can no longer reproduce the error there. > > So the fix may be as simple as upgrading the virtual machine hardware > version on the production machine, which we plan to do as soon as we can > schedule a little down time. > > > I really hope that works out to be the solution. It's been years since > I've dealt with an application failure this nasty. > > > -- > Fred Yankowski > fc...@im... +1-630-907-5019 > > > P.S. Yes, I should upgrade our Python version to the latest available. > I'll get to that once the server is stable again. > > P.P.S. Thanks for helping out. > > > > > ------------------------------------------------------------------------------ > The Command Line: Reinvented for Modern Developers > Did the resurgence of CLI tooling catch you by surprise? > Reconnect with the command line and become more productive. > Learn the new .NET and ASP.NET CLI. Get your free copy! > http://sdm.link/telerik > > > > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > -- James C. McPherson -- Oracle Systems / Solaris / Core https://www.jmcpdotcom.com/blog |
|
From: Fred Y. <fc...@im...> - 2016-10-25 21:48:15
|
It now appears that my problem was caused by migrating the server from a VMware 4 host to a VMware 5 host without upgrading the "hardware version" of the virtual machine. I totally forgot that our system admin team had done that about a week ago. The server otherwise worked exactly the same after that migration. It may have required a restart of Apache after the migration for the Oracle connection error to start appearing. Our lead sys admin just migrated my server temporarily back to a VMware 4 host and I can no longer reproduce the error there. On a matching dev server running on a VMware 5 host we just upgraded the virtual hardware version and now I can no longer reproduce the error there. So the fix may be as simple as upgrading the virtual machine hardware version on the production machine, which we plan to do as soon as we can schedule a little down time. I really hope that works out to be the solution. It's been years since I've dealt with an application failure this nasty. -- Fred Yankowski fc...@im... +1-630-907-5019 P.S. Yes, I should upgrade our Python version to the latest available. I'll get to that once the server is stable again. P.P.S. Thanks for helping out. |
|
From: Anthony T. <ant...@gm...> - 2016-10-25 18:06:09
|
On Tue, Oct 25, 2016 at 11:52 AM, Fred Yankowski <fc...@im...> wrote:
> Thanks, Anthony.
>
You're welcome.
>
> I have tried running with 'threaded' set to True and that does not help.
>
Didn't think so -- but worth checking. :-)
>
> Various combinations of argument values to my test script cause SIGSEGV in
> different places (or no SIGSEGV at all in some cases), but not correlating
> with any given option value. It's as if minor changes in the call stack
> cause the oracle client code to go off the rails, unrelated to the exact
> arguments passed down to the oracle client API.
>
Yeah. These are the hardest to track down. It will be due to some memory
overwrite somewhere -- but it could be *anything* that is running in your
process that causes the problem. If you're familiar with valgrind and
similar tools you can see if anything turns up using that.
>
> My application connects to Oracle only infrequently, polling for data
> updates on hourly cron jobs. So the complexity of connection pooling did
> not make sense per se. But I'll give it a try to see if it works around
> this problem. I'm not familiar with DRCP.
>
Sure. DRCP stands for "Database Resident Connection Pooling" and is server
side pooling. It is fairly simple to use. I have a sample that will be made
available in a newer version of cx_Oracle. I'll provide the basic here:
In SQL*Plus you need to run these commands:
exec dbms_connection_pool.start_pool() # to start the pool
exec dbms_connection_pool.stop_pool() # to stop the pool
conn = cx_Oracle.Connection("cx_Oracle/dev@localhost/orcl:pooled",
cclass = "PYCLASS", purity = cx_Oracle.ATTR_PURITY_SELF)
Note the addition of the ":pooled" on the end of the connect string (or
this can be included in the TNS entry if you prefer that to the EZ Connect
syntax). Then you need to specify the cclass (connection class).
This makes use of a different API internally (OCISessionGet() vs
OCISessionBegin()) as does the general session pool case so this may or may
not work around your issue.
>
> The oracle database I'm connecting to comes embedded with another product
> (PowerSchool) and so we don't administer it directly, only through admin
> tools provided by PowerSchool. The diagnostics you mention would be on the
> oracle server side, right?
>
No. The diagnostics would be client based as the segfault is occurring
there.
One other thing to check -- the NLS database parameters vs the NLS client
parameters. You can improve performance and possibly eliminate problems by
matching them, if possible.
Good luck!
>
> --
> Fred Yankowski
> fc...@im... +1-630-907-5019
>
> ------------------------------
>
> Hi Fred,
>
> You seem to be experiencing a sort of memory corruption perhaps due to an
> overwrite somewhere. If you were using threads I would have immediately
> told you that you need to ensure threaded mode is enabled -- but I don't
> see multiple threads in use. You may wish to enable it anyway to see if you
> get consistent results, but that shouldn't be a problem.
>
> I have seen similar, highly intermittent, segfaults inside
> OCIServerAttach() myself, but only when using threads -- and they don't
> terminate in the same place.
>
> You could try using a session pool (cx_Oracle.SessionPool) with or without
> DRCP to see if that makes things happier. Not sure how many connections you
> are generally creating but if you are creating a few this would improve
> performance anyway!
>
> You can try enabling diagnostics for Oracle and seeing if interesting
> information shows up. You can also see if you get consistent backtraces.
>
> All of that said, I don't have this problem! Of course I'm not running the
> exact configuration you have. Depending on what you get from backtraces and
> diagnostics you can log a support request and see if they can help you.
>
> Anthony
>
>
> ------------------------------------------------------------
> ------------------
> The Command Line: Reinvented for Modern Developers
> Did the resurgence of CLI tooling catch you by surprise?
> Reconnect with the command line and become more productive.
> Learn the new .NET and ASP.NET CLI. Get your free copy!
> http://sdm.link/telerik
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>
>
|
|
From: Fred Y. <fc...@im...> - 2016-10-25 17:52:07
|
Thanks, Anthony. I have tried running with 'threaded' set to True and that does not help. Various combinations of argument values to my test script cause SIGSEGV in different places (or no SIGSEGV at all in some cases), but not correlating with any given option value. It's as if minor changes in the call stack cause the oracle client code to go off the rails, unrelated to the exact arguments passed down to the oracle client API. My application connects to Oracle only infrequently, polling for data updates on hourly cron jobs. So the complexity of connection pooling did not make sense per se. But I'll give it a try to see if it works around this problem. I'm not familiar with DRCP. The oracle database I'm connecting to comes embedded with another product (PowerSchool) and so we don't administer it directly, only through admin tools provided by PowerSchool. The diagnostics you mention would be on the oracle server side, right? -- Fred Yankowski fc...@im... +1-630-907-5019 ----- Original Message ----- Hi Fred, You seem to be experiencing a sort of memory corruption perhaps due to an overwrite somewhere. If you were using threads I would have immediately told you that you need to ensure threaded mode is enabled -- but I don't see multiple threads in use. You may wish to enable it anyway to see if you get consistent results, but that shouldn't be a problem. I have seen similar, highly intermittent, segfaults inside OCIServerAttach() myself, but only when using threads -- and they don't terminate in the same place. You could try using a session pool (cx_Oracle.SessionPool) with or without DRCP to see if that makes things happier. Not sure how many connections you are generally creating but if you are creating a few this would improve performance anyway! You can try enabling diagnostics for Oracle and seeing if interesting information shows up. You can also see if you get consistent backtraces. All of that said, I don't have this problem! Of course I'm not running the exact configuration you have. Depending on what you get from backtraces and diagnostics you can log a support request and see if they can help you. Anthony |
|
From: Anthony T. <ant...@gm...> - 2016-10-25 17:36:47
|
Hi Fred, You seem to be experiencing a sort of memory corruption perhaps due to an overwrite somewhere. If you were using threads I would have immediately told you that you need to ensure threaded mode is enabled -- but I don't see multiple threads in use. You may wish to enable it anyway to see if you get consistent results, but that shouldn't be a problem. I have seen similar, highly intermittent, segfaults inside OCIServerAttach() myself, but only when using threads -- and they don't terminate in the same place. You could try using a session pool (cx_Oracle.SessionPool) with or without DRCP to see if that makes things happier. Not sure how many connections you are generally creating but if you are creating a few this would improve performance anyway! You can try enabling diagnostics for Oracle and seeing if interesting information shows up. You can also see if you get consistent backtraces. All of that said, I don't have this problem! Of course I'm not running the exact configuration you have. Depending on what you get from backtraces and diagnostics you can log a support request and see if they can help you. Anthony On Tue, Oct 25, 2016 at 10:38 AM, Fred Yankowski <fc...@im...> wrote: > I posted the code at https://gist.github.com/fredcy/ > ebb24602ef9cd94ec5e349affa573cd8 > > The test function is practically the simplest test case I could think of. > When it fails it’s in the call to cx_Oracle.connect. > > It does not always fail. When it does fail the typical pattern is that the > first call to connect works and it’s able to run the query and print the > results; but the second connect then fails. But other times it runs a bunch > of connections successively and they all work. > > I added some code to dump the context and enable faulthandler. Oddly, > enabling those options affects the results! > > 1. > > python gist/testoracle.py --count 20 hangs on second connect; has to > be killed from elsewhere. > 2. > > python gist/testoracle.py --count 20 --fault runs all 20 connections > without problem. > 3. > > python gist/testoracle.py --count 20 --fault --info fails on second > connect and quits. > 4. > > python gist/testoracle.py --count 20 --info runs all 20 connections > without problem. > > The --fault and --info arguments cause conditional imports. I wonder if > some side effect of those imports is changing the context in some subtle > way that hides the problem in some cases and shows it in others. Maybe > something concerning memory placement of the cx_Oracle code? > > Also, when I modify my driver script in ways that don’t directly affect > the cx_Oracle code it can cause changes in how it behaves. > > Here is the result of running with --info: > > > platform: Linux-3.10.10-imsa01-MOD-RHVM-SMP.cos64-x86_64-with-redhat-6.8-Santiago > architecture: ('64bit', 'ELF') > python_build: ('default', 'Jul 22 2014 16:14:39') > python_compiler: GCC 4.4.7 20120313 (Red Hat 4.4.7-4) > python_version: 2.7.8 > libc_ver: ('glibc', '2.2.5') > uname: ('Linux', 'app.devnet.imsa.edu', '3.10.10-imsa01-MOD-RHVM-SMP.cos64', '#4 SMP Thu Sep 5 16:18:19 CDT 2013', 'x86_64', 'x86_64') > > Background > > The problem arose last Friday afternoon. I had within the hour changed our > Apache configuration to use a stronger SSL/TLS configuration. I keep /etc > under source code control and had baselined it earlier that day so I’m > quite sure about what changes I made. Our stack is Web2py 2.14.6 ( > www.web2py.co <http://www.web2py.con>m) on Apache (Apache/2.2.15 (Unix) > mod_wsgi/3.4 Python/2.7.8 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips configured). I > also tweaked Apache to discard proxy headers in the client request. > > Not long after those changes I started seeing errors from cron jobs that > use curl to run requests against the above service. By instrumenting the > python code I found that the connection to Oracle was sometimes failing. On > Monday I temporarily reverted those changes and restarted Apache. The > connection problems continued to happen. > > So I then created a standalone python script (gist link above) to test the > Oracle connections. I ran it using the same python virtualenv that the web > service above uses, and it failed. > > I then tried the same from a development server that runs the same stack > but which was not updated with the SSL changes. And it shows the same > problem. So I’m baffled, since the problem seemed to arise right after the > SSL/TLS config changes but the same problem happens on a similar unmodified > system. > > I then built a new virtualenv on the dev server with just the lastest > version of cx_Oracle (and faulthandler). And that fails the same way. > > It’s possible that the Apache restart on Friday picked up some system > change that had been lurking, but I’ve looked hard at prior configuration > changes and can’t see anything that could matter. So I’m stumped. > > -- > Fred Yankowski > fc...@im... +1-630-907-5019 > > ------------------------------ > > Hi Fred, > > Are you able to post the code that is causing the problem? And of course, > the age old question: what did you change? :-) That may help diagnose why > this has turned up now -- and potentially help resolve the problem for > others as well. > > Anthony > > > ------------------------------------------------------------ > ------------------ > The Command Line: Reinvented for Modern Developers > Did the resurgence of CLI tooling catch you by surprise? > Reconnect with the command line and become more productive. > Learn the new .NET and ASP.NET CLI. Get your free copy! > http://sdm.link/telerik > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > |
|
From: Fred Y. <fc...@im...> - 2016-10-25 17:23:43
|
I was able to get a good backtrace from gdb this time. This is with cx_Oracle 5.2.1. I get a SIGSEGV on the second connection attempt in the test run. Core was generated by `python gist/testoracle.py --fault --count 20 --info'. Program terminated with signal 11, Segmentation fault. #0 0x0000003c9ee0f6ab in raise () from /lib64/libpthread.so.0 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.192.el6.x86_64 libaio-0.3.107-10.el6.x86_64 oracle-instantclient12.1-basic-12.1.0.2.0-1.x86_64 python27-2.7.8-1.x86_64 (gdb) bt #0 0x0000003c9ee0f6ab in raise () from /lib64/libpthread.so.0 #1 <signal handler called> #2 0x00007f5ab2f3ddff in __intel_ssse3_rep_memcpy () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #3 0x00007f5ab2f36cc6 in _intel_fast_memcpy.P () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #4 0x00007f5ab4108828 in nsdo () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #5 0x00007f5ab24cbd6e in nsbasic_sd () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #6 0x00007f5ab24cc024 in nssend () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #7 0x00007f5ab2448915 in nsnasend () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #8 0x00007f5ab2497145 in nacomsn () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #9 0x00007f5ab2490690 in na_client () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #10 0x00007f5ab248a0fa in naconnect () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #11 0x00007f5ab2442a80 in nsnadoconn () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #12 0x00007f5ab243ee20 in nsnaconn () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #13 0x00007f5ab23efef2 in nscall () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #14 0x00007f5ab24700bb in niotns () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #15 0x00007f5ab24e796b in nigcall () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #16 0x00007f5ab2479742 in osncon () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #17 0x00007f5ab2330461 in kpuadef () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #18 0x00007f5ab2314c7a in upiini () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #19 0x00007f5ab2309b31 in upiah0 () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #20 0x00007f5ab232f2f8 in kpuatch () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #21 0x00007f5ab231847f in OCIServerAttach () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #22 0x00007f5ab4dc145d in Connection_Connect (self=0x7f5ab0178c70, args=<value optimized out>, keywordArgs=<value optimized out>) at Connection.c:603 #23 Connection_Init (self=0x7f5ab0178c70, args=<value optimized out>, keywordArgs=<value optimized out>) at Connection.c:914 #24 0x00007f5abb087078 in type_call (type=<value optimized out>, args=0x7f5ab02a9960, kwds=0x7f5ab02d2050) at Objects/typeobject.c:745 #25 0x00007f5abb0241a3 in PyObject_Call (func=0x7f5ab4fcc8e0, arg=<value optimized out>, kw=<value optimized out>) at Objects/abstract.c:2529 #26 0x00007f5abb0d2dfd in do_call (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4251 #27 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4056 #28 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679 #29 0x00007f5abb0d43ae in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4119 #30 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054 #31 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679 #32 0x00007f5abb0d54ae in PyEval_EvalCodeEx (co=0x7f5abaf4cb30, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #33 0x00007f5abb0d55c2 in PyEval_EvalCode (co=<value optimized out>, globals=<value optimized out>, locals=<value optimized out>) at Python/ceval.c:673 #34 0x00007f5abb0f52a0 in run_mod (fp=0x61b5c0, filename=<value optimized out>, start=<value optimized out>, globals=0x7f5abaf6c168, locals=0x7f5abaf6c168, closeit=1, flags=0x7fff44330810) at Python/pythonrun.c:1377 #35 PyRun_FileExFlags (fp=0x61b5c0, filename=<value optimized out>, start=<value optimized out>, globals=0x7f5abaf6c168, locals=0x7f5abaf6c168, closeit=1, flags=0x7fff44330810) at Python/pythonrun.c:1363 #36 0x00007f5abb0f547f in PyRun_SimpleFileExFlags (fp=0x61b5c0, filename=0x7fff44331521 "gist/testoracle.py", closeit=1, flags=0x7fff44330810) at Python/pythonrun.c:955 #37 0x00007f5abb10ac44 in Py_Main (argc=<value optimized out>, argv=<value optimized out>) at Modules/main.c:640 #38 0x0000003c9e61ed1d in __libc_start_main () from /lib64/libc.so.6 #39 0x0000000000400649 in _start () -- Fred Yankowski fc...@im... +1-630-907-5019 |
|
From: Fred Y. <fc...@im...> - 2016-10-25 16:38:52
|
I posted the code at https://gist.github.com/fredcy/ebb24602ef9cd94ec5e349affa573cd8 The test function is practically the simplest test case I could think of. When it fails it’s in the call to cx_Oracle.connect . It does not always fail. When it does fail the typical pattern is that the first call to connect works and it’s able to run the query and print the results; but the second connect then fails. But other times it runs a bunch of connections successively and they all work. I added some code to dump the context and enable faulthandler. Oddly, enabling those options affects the results! 1. python gist/testoracle.py --count 20 hangs on second connect; has to be killed from elsewhere. 2. python gist/testoracle.py --count 20 --fault runs all 20 connections without problem. 3. python gist/testoracle.py --count 20 --fault --info fails on second connect and quits. 4. python gist/testoracle.py --count 20 --info runs all 20 connections without problem. The --fault and --info arguments cause conditional imports. I wonder if some side effect of those imports is changing the context in some subtle way that hides the problem in some cases and shows it in others. Maybe something concerning memory placement of the cx_Oracle code? Also, when I modify my driver script in ways that don’t directly affect the cx_Oracle code it can cause changes in how it behaves. Here is the result of running with --info : platform: Linux-3.10.10-imsa01-MOD-RHVM-SMP.cos64-x86_64-with-redhat-6.8-Santiago architecture: ('64bit', 'ELF') python_build: ('default', 'Jul 22 2014 16:14:39') python_compiler: GCC 4.4.7 20120313 (Red Hat 4.4.7-4) python_version: 2.7.8 libc_ver: ('glibc', '2.2.5') uname: ('Linux', 'app.devnet.imsa.edu', '3.10.10-imsa01-MOD-RHVM-SMP.cos64', '#4 SMP Thu Sep 5 16:18:19 CDT 2013', 'x86_64', 'x86_64') Background The problem arose last Friday afternoon. I had within the hour changed our Apache configuration to use a stronger SSL/TLS configuration. I keep /etc under source code control and had baselined it earlier that day so I’m quite sure about what changes I made. Our stack is Web2py 2.14.6 ( www.web2py.co m) on Apache (Apache/2.2.15 (Unix) mod_wsgi/3.4 Python/2.7.8 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips configured). I also tweaked Apache to discard proxy headers in the client request. Not long after those changes I started seeing errors from cron jobs that use curl to run requests against the above service. By instrumenting the python code I found that the connection to Oracle was sometimes failing. On Monday I temporarily reverted those changes and restarted Apache. The connection problems continued to happen. So I then created a standalone python script (gist link above) to test the Oracle connections. I ran it using the same python virtualenv that the web service above uses, and it failed. I then tried the same from a development server that runs the same stack but which was not updated with the SSL changes. And it shows the same problem. So I’m baffled, since the problem seemed to arise right after the SSL/TLS config changes but the same problem happens on a similar unmodified system. I then built a new virtualenv on the dev server with just the lastest version of cx_Oracle (and faulthandler). And that fails the same way. It’s possible that the Apache restart on Friday picked up some system change that had been lurking, but I’ve looked hard at prior configuration changes and can’t see anything that could matter. So I’m stumped. -- Fred Yankowski fc...@im... +1-630-907-5019 ----- Original Message ----- > Hi Fred, > Are you able to post the code that is causing the problem? And of course, the > age old question: what did you change? :-) That may help diagnose why this > has turned up now -- and potentially help resolve the problem for others as > well. > Anthony |
|
From: Anthony T. <ant...@gm...> - 2016-10-25 03:04:22
|
Hi Fred, Are you able to post the code that is causing the problem? And of course, the age old question: what did you change? :-) That may help diagnose why this has turned up now -- and potentially help resolve the problem for others as well. Anthony On Mon, Oct 24, 2016 at 4:50 PM, Fred Yankowski <fc...@im...> wrote: > Last week I started getting segmentation-violation crashes from cx_Oracle > after using it without problems for years. I cannot figure out what changed > and broke things. I hope you all can help me fix it. > > I'm testing with a fresh installation of cx_Oracle with the latest Oracle > client libs that I could find: > > OS: RHEL 6.8 > > Python: 2.7.8, built from source. > > cx_Oracle: 5.2.1 > > Oracle instantclient libs: 12.1.0.2.0 > > > What I'm seeing now via a simple test driver is that the first call to > cx_Oracle.connect() works. I'm able to get a cursor, execute the query, and > get the results. But when I call connect() a second time I get a SIGSEGV. > When I run the test driver different times I get slightly different errors. > > ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] > [si_int=0] [si_ptr=(nil)] [si_addr=0x870000] > *** glibc detected *** python: free(): invalid pointer: 0x000000000073c150 > *** > > > Or > > > ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] > [si_int=0] [si_ptr=(nil)] [si_addr=0x354] > *** glibc detected *** python: double free or corruption (out): > 0x00000000006f5f40 *** > > > When I enable faulthandler I get a short trace in my test driver python > code that just shows it failing in the connect() call. > > > When I load the core file into gdb I get only a short backtrace, possibly > because debug info is missing from some critical libs: > > > #0 0x0000003c9ee0f6ab in raise () from /lib64/libpthread.so.0 > #1 <signal handler called> > #2 0x00007f4122578e3f in __intel_ssse3_rep_memcpy () from > /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 > #3 0x00007f4122571cc6 in _intel_fast_memcpy.P () from > /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 > #4 0x00007f4121b113f6 in nsc2addr () from /usr/lib/oracle/12.1/client64/ > lib/libclntsh.so.12.1 > #5 0x0000000000000000 in ?? () > > > > Any suggestions on what I should do to fix this? > > > > -- > Fred Yankowski > fc...@im... +1-630-907-5019 > > > ------------------------------------------------------------ > ------------------ > The Command Line: Reinvented for Modern Developers > Did the resurgence of CLI tooling catch you by surprise? > Reconnect with the command line and become more productive. > Learn the new .NET and ASP.NET CLI. Get your free copy! > http://sdm.link/telerik > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > > |
|
From: Glyph L. <gl...@tw...> - 2016-10-25 01:33:32
|
> On Oct 24, 2016, at 15:50, Fred Yankowski <fc...@im...> wrote: > > Last week I started getting segmentation-violation crashes from cx_Oracle after using it without problems for years. I cannot figure out what changed and broke things. I hope you all can help me fix it. > > I'm testing with a fresh installation of cx_Oracle with the latest Oracle client libs that I could find: > > OS: RHEL 6.8 > > Python: 2.7.8, built from source. Sorry for the off-topic reply, but: especially since you're building from source anyway, you should really upgrade to the most recent release of 2.7. At the very least, 2.7.9 makes it possible to make secure HTTPS connections, which previous versions don't. Presumably if this is a program connecting to an Oracle database, it's got interesting enough data in it that you want to protect it from attackers, and keeping up to date on your interpreter versions (or at least, updating more than once a year: <https://www.python.org/downloads/release/python-279/ <https://www.python.org/downloads/release/python-279/>>!) is an important part of that. -glyph |
|
From: Fred Y. <fc...@im...> - 2016-10-25 01:03:41
|
Last week I started getting segmentation-violation crashes from cx_Oracle after using it without problems for years. I cannot figure out what changed and broke things. I hope you all can help me fix it. I'm testing with a fresh installation of cx_Oracle with the latest Oracle client libs that I could find: OS: RHEL 6.8 Python: 2.7.8, built from source. cx_Oracle: 5.2.1 Oracle instantclient libs: 12.1.0.2.0 What I'm seeing now via a simple test driver is that the first call to cx_Oracle.connect() works. I'm able to get a cursor, execute the query, and get the results. But when I call connect() a second time I get a SIGSEGV. When I run the test driver different times I get slightly different errors. ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=0] [si_ptr=(nil)] [si_addr=0x870000] *** glibc detected *** python: free(): invalid pointer: 0x000000000073c150 *** Or ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=0] [si_ptr=(nil)] [si_addr=0x354] *** glibc detected *** python: double free or corruption (out): 0x00000000006f5f40 *** When I enable faulthandler I get a short trace in my test driver python code that just shows it failing in the connect() call. When I load the core file into gdb I get only a short backtrace, possibly because debug info is missing from some critical libs: #0 0x0000003c9ee0f6ab in raise () from /lib64/libpthread.so.0 #1 <signal handler called> #2 0x00007f4122578e3f in __intel_ssse3_rep_memcpy () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #3 0x00007f4122571cc6 in _intel_fast_memcpy.P () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #4 0x00007f4121b113f6 in nsc2addr () from /usr/lib/oracle/12.1/client64/lib/libclntsh.so.12.1 #5 0x0000000000000000 in ?? () Any suggestions on what I should do to fix this? -- Fred Yankowski fc...@im... +1-630-907-5019 |
|
From: Piotr D. <p...@20...> - 2016-10-23 19:48:35
|
Hi Anthony, Thank you very much for your quick reply and showing me how to do this. Best regards, Piotr Dobrogost |
|
From: Anthony T. <ant...@gm...> - 2016-10-19 14:08:50
|
Hi Piotr, There is no built-in way of logging the queries sent to the database....but cx_Oracle *does* have a way that you can do that yourself. Essentially you can subclass the connection and cursor and override the execute() method in order to perform any logging you might want to do before or after the actual execution of the statement takes place. You can see an example in action with a module called cx_OracleEx. You can see it here: https://bitbucket.org/anthony_tuininga/cx_pyoraclelib/src/dda6bfb0217bc48ea7c20cb763b3385e8fc9bfaa/cx_OracleEx.py?at=default&fileviewer=file-view-default Anthony On Wed, Oct 19, 2016 at 3:20 AM, Piotr Dobrogost < p...@20...> wrote: > Hi! > > Is there a way to configure cx_oracle so that it would log queries sent to > db? > I'm particularly interested in logging values of bound parameters. > > > Best regards, > Piotr Dobrogost > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > cx-oracle-users mailing list > cx-...@li... > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users > |
|
From: Piotr D. <p...@20...> - 2016-10-19 09:47:50
|
Hi! Is there a way to configure cx_oracle so that it would log queries sent to db? I'm particularly interested in logging values of bound parameters. Best regards, Piotr Dobrogost |
|
From: Moritz L. <mor...@go...> - 2016-08-31 14:56:29
|
No I was wrong the Oracle docu *SET ARRAY[SIZE] {15 | n}
<https://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm#i2698625>
*is
regarding to sql* Plus and is not refering to python.
So it doesn´t matter in my case. But I have some problems with the aslist()
method for objects.
*My case:*
1.) If I only execute the query (cursor.execute(sql) and print the result
the browser needs around 0,8s
2.) If I do the for-loop this way I need 0,2 seconds:
for id, geometry, classname in cursor:
result.append({"building_nr":id,"geometry":{"type":"
polygon","coordinates":geometry,},"polygon_typ":classname,})
retunr result
3.) But If I create a List and iterate over this list I need 11 seconds!!!:
for id, geometry, classname in cursor:
mo = geometry.SDO_ORDINATES.aslist()
result.append({"building_nr":id,"geometry":{"type":"
polygon","coordinates":zip(*[iter(mo)] * ,},"polygon_typ":classname,})
return json.dumps(result)
But sadly I need the coordinates. Is there a workaround for object.aslist()
?
2016-08-28 22:04 GMT+02:00 Shai Berger <sh...@pl...>:
> Then, perhaps you can just avoid doing that?
>
> If I get it right, the ArraySize does not limit the size of overall
> returned
> results, but only the size returned to the client in one batch.
>
> Add to this that, according to the documentation you linked and quoted
> yourself, 6,000 is not a valid value...
>
> Why don't you just try leaving that value alone?
>
> On Thursday 25 August 2016 12:59:07 Moritz Liebelt wrote:
> > Hi Shai,
> >
> > I did a test query in sql Developer and it seems that Oracle is not
> > responsible for that.
> >
> > But to fetch all the rows from my Database I have to increase the
> arraysize
> > in cx_Oracle (*cursor.arraysize* default=50, my setting = 6.000)
> > Furthermore it is written in the Oracle Docu
> >
> > *SET ARRAY[SIZE] {15 | n}
> > <https://docs.oracle.com/cd/B28359_01/server.111/b31189/
> ch12040.htm#i269862
> > 5>* * Sets the number of rows, called a batch, that SQL*Plus will fetch
> > from the database at one time.Valid values are 1 to 5000. A large value
> > increases the efficiency of queries and subqueries that fetch many rows,
> > but requires more memory.*
> >
> > And In my case under 5000 rows it neeeds some milliseconds and over 5000
> > rows it needs some seconds. So is this depending on the arraysize?
> >
> > 2016-08-23 21:42 GMT+02:00 Shai Berger <sh...@pl...>:
> > > Hi Moritz,
> > >
> > > Two notions come to mind:
> > >
> > > 1) Are you sure the time is wasted in Python-land? Do you have a way to
> > > measure the execution of the query without Python or cx_Oracle being
> > > invoved
> > > (e.g. in SQL*Plus)?
> > >
> > > 2) Assuming it is -- I've seen delays on the order of seconds show up
> > > when you
> > > create a lot of circular references -- the Python GC gets invoked then
> > > and can
> > > cause surprisingly large "lapses". You can check the documentation of
> the
> > > stdlib's gc module for ways to verify and/or eliminate this issue.
> > >
> > > HTH,
> > >
> > > Shai.
> > >
> > > On Tuesday 23 August 2016 18:43:26 Moritz Liebelt wrote:
> > > > Hi,
> > > >
> > > > I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I
> use
> > >
> > > the
> > >
> > > > unreleased version of cx_Oracle! Furthermore I read the sdo_geometry
> > > > objects in the sql-statement and build a json based on this.
> > > >
> > > > For a 500m-Tile with 4634 rows I need 0,326 (measured time in the
> > >
> > > browser=
> > >
> > > > waiting time (DB + Webserver))
> > > > For a 500m-Tile with 5081 rows I need 10,1 second???
> > > > You can see an overview
> > > >
> > > > *500m-Tile_A ( 2311 rows)* *500m-Tile_B (3799 rows)* *400m-Tile
> (4137
> > > >
> > > > rows)* *500m-Tile_E (4634 rows)* *500m-Tile_C (5081 rows)* *500m-Tile
> > > > _D (6241 rows)*
> > > > *sdo_geometry (cx_Oracle)* 0,18s 0,26s 0,27s 0,326s 10,1s 13,77s
> > > > *Size* 729KB 1,2 MB 1,3 MB 1,4MB 1,6 MB 2 MB
> > > >
> > > > Is there anywhere a Threshold? Because the time is increasing and I
> do
> > >
> > > not
> > >
> > > > understand why.
> > > > It does not refer to Browser problems. I think that this is depending
> > > > on the webserver.
> > > >
> > > > Does someone knows this issue?
> > >
> > > ------------------------------------------------------------
> > > ------------------
> > > _______________________________________________
> > > cx-oracle-users mailing list
> > > cx-...@li...
> > > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>
--
*Moritz Liebelt, B. Sc.*
Murgstraße 19
76437 Rastatt
E-Mail: mor...@go...
|
|
From: Shai B. <sh...@pl...> - 2016-08-28 20:21:43
|
Then, perhaps you can just avoid doing that?
If I get it right, the ArraySize does not limit the size of overall returned
results, but only the size returned to the client in one batch.
Add to this that, according to the documentation you linked and quoted
yourself, 6,000 is not a valid value...
Why don't you just try leaving that value alone?
On Thursday 25 August 2016 12:59:07 Moritz Liebelt wrote:
> Hi Shai,
>
> I did a test query in sql Developer and it seems that Oracle is not
> responsible for that.
>
> But to fetch all the rows from my Database I have to increase the arraysize
> in cx_Oracle (*cursor.arraysize* default=50, my setting = 6.000)
> Furthermore it is written in the Oracle Docu
>
> *SET ARRAY[SIZE] {15 | n}
> <https://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm#i269862
> 5>* * Sets the number of rows, called a batch, that SQL*Plus will fetch
> from the database at one time.Valid values are 1 to 5000. A large value
> increases the efficiency of queries and subqueries that fetch many rows,
> but requires more memory.*
>
> And In my case under 5000 rows it neeeds some milliseconds and over 5000
> rows it needs some seconds. So is this depending on the arraysize?
>
> 2016-08-23 21:42 GMT+02:00 Shai Berger <sh...@pl...>:
> > Hi Moritz,
> >
> > Two notions come to mind:
> >
> > 1) Are you sure the time is wasted in Python-land? Do you have a way to
> > measure the execution of the query without Python or cx_Oracle being
> > invoved
> > (e.g. in SQL*Plus)?
> >
> > 2) Assuming it is -- I've seen delays on the order of seconds show up
> > when you
> > create a lot of circular references -- the Python GC gets invoked then
> > and can
> > cause surprisingly large "lapses". You can check the documentation of the
> > stdlib's gc module for ways to verify and/or eliminate this issue.
> >
> > HTH,
> >
> > Shai.
> >
> > On Tuesday 23 August 2016 18:43:26 Moritz Liebelt wrote:
> > > Hi,
> > >
> > > I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I use
> >
> > the
> >
> > > unreleased version of cx_Oracle! Furthermore I read the sdo_geometry
> > > objects in the sql-statement and build a json based on this.
> > >
> > > For a 500m-Tile with 4634 rows I need 0,326 (measured time in the
> >
> > browser=
> >
> > > waiting time (DB + Webserver))
> > > For a 500m-Tile with 5081 rows I need 10,1 second???
> > > You can see an overview
> > >
> > > *500m-Tile_A ( 2311 rows)* *500m-Tile_B (3799 rows)* *400m-Tile (4137
> > >
> > > rows)* *500m-Tile_E (4634 rows)* *500m-Tile_C (5081 rows)* *500m-Tile
> > > _D (6241 rows)*
> > > *sdo_geometry (cx_Oracle)* 0,18s 0,26s 0,27s 0,326s 10,1s 13,77s
> > > *Size* 729KB 1,2 MB 1,3 MB 1,4MB 1,6 MB 2 MB
> > >
> > > Is there anywhere a Threshold? Because the time is increasing and I do
> >
> > not
> >
> > > understand why.
> > > It does not refer to Browser problems. I think that this is depending
> > > on the webserver.
> > >
> > > Does someone knows this issue?
> >
> > ------------------------------------------------------------
> > ------------------
> > _______________________________________________
> > cx-oracle-users mailing list
> > cx-...@li...
> > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
|
|
From: Moritz L. <mor...@go...> - 2016-08-25 09:59:16
|
Hi Shai,
I did a test query in sql Developer and it seems that Oracle is not
responsible for that.
But to fetch all the rows from my Database I have to increase the arraysize
in cx_Oracle (*cursor.arraysize* default=50, my setting = 6.000)
Furthermore it is written in the Oracle Docu
*SET ARRAY[SIZE] {15 | n}
<https://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm#i2698625>*
* Sets the number of rows, called a batch, that SQL*Plus will fetch from
the database at one time.Valid values are 1 to 5000. A large value
increases the efficiency of queries and subqueries that fetch many rows,
but requires more memory.*
And In my case under 5000 rows it neeeds some milliseconds and over 5000
rows it needs some seconds. So is this depending on the arraysize?
2016-08-23 21:42 GMT+02:00 Shai Berger <sh...@pl...>:
> Hi Moritz,
>
> Two notions come to mind:
>
> 1) Are you sure the time is wasted in Python-land? Do you have a way to
> measure the execution of the query without Python or cx_Oracle being
> invoved
> (e.g. in SQL*Plus)?
>
> 2) Assuming it is -- I've seen delays on the order of seconds show up when
> you
> create a lot of circular references -- the Python GC gets invoked then and
> can
> cause surprisingly large "lapses". You can check the documentation of the
> stdlib's gc module for ways to verify and/or eliminate this issue.
>
> HTH,
> Shai.
>
> On Tuesday 23 August 2016 18:43:26 Moritz Liebelt wrote:
> > Hi,
> >
> > I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I use
> the
> > unreleased version of cx_Oracle! Furthermore I read the sdo_geometry
> > objects in the sql-statement and build a json based on this.
> >
> > For a 500m-Tile with 4634 rows I need 0,326 (measured time in the
> browser=
> > waiting time (DB + Webserver))
> > For a 500m-Tile with 5081 rows I need 10,1 second???
> > You can see an overview
> > *500m-Tile_A ( 2311 rows)* *500m-Tile_B (3799 rows)* *400m-Tile (4137
> > rows)* *500m-Tile_E (4634 rows)* *500m-Tile_C (5081 rows)* *500m-Tile _D
> > (6241 rows)*
> > *sdo_geometry (cx_Oracle)* 0,18s 0,26s 0,27s 0,326s 10,1s 13,77s
> > *Size* 729KB 1,2 MB 1,3 MB 1,4MB 1,6 MB 2 MB
> >
> > Is there anywhere a Threshold? Because the time is increasing and I do
> not
> > understand why.
> > It does not refer to Browser problems. I think that this is depending on
> > the webserver.
> >
> > Does someone knows this issue?
>
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>
--
*Moritz Liebelt, B. Sc.*
Murgstraße 19
76437 Rastatt
E-Mail: mor...@go...
|
|
From: Shai B. <sh...@pl...> - 2016-08-23 19:42:24
|
Hi Moritz, Two notions come to mind: 1) Are you sure the time is wasted in Python-land? Do you have a way to measure the execution of the query without Python or cx_Oracle being invoved (e.g. in SQL*Plus)? 2) Assuming it is -- I've seen delays on the order of seconds show up when you create a lot of circular references -- the Python GC gets invoked then and can cause surprisingly large "lapses". You can check the documentation of the stdlib's gc module for ways to verify and/or eliminate this issue. HTH, Shai. On Tuesday 23 August 2016 18:43:26 Moritz Liebelt wrote: > Hi, > > I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I use the > unreleased version of cx_Oracle! Furthermore I read the sdo_geometry > objects in the sql-statement and build a json based on this. > > For a 500m-Tile with 4634 rows I need 0,326 (measured time in the browser= > waiting time (DB + Webserver)) > For a 500m-Tile with 5081 rows I need 10,1 second??? > You can see an overview > *500m-Tile_A ( 2311 rows)* *500m-Tile_B (3799 rows)* *400m-Tile (4137 > rows)* *500m-Tile_E (4634 rows)* *500m-Tile_C (5081 rows)* *500m-Tile _D > (6241 rows)* > *sdo_geometry (cx_Oracle)* 0,18s 0,26s 0,27s 0,326s 10,1s 13,77s > *Size* 729KB 1,2 MB 1,3 MB 1,4MB 1,6 MB 2 MB > > Is there anywhere a Threshold? Because the time is increasing and I do not > understand why. > It does not refer to Browser problems. I think that this is depending on > the webserver. > > Does someone knows this issue? |
|
From: Moritz L. <mor...@go...> - 2016-08-23 15:43:36
|
Hi, I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I use the unreleased version of cx_Oracle! Furthermore I read the sdo_geometry objects in the sql-statement and build a json based on this. For a 500m-Tile with 4634 rows I need 0,326 (measured time in the browser= waiting time (DB + Webserver)) For a 500m-Tile with 5081 rows I need 10,1 second??? You can see an overview *500m-Tile_A ( 2311 rows)* *500m-Tile_B (3799 rows)* *400m-Tile (4137 rows)* *500m-Tile_E (4634 rows)* *500m-Tile_C (5081 rows)* *500m-Tile _D (6241 rows)* *sdo_geometry (cx_Oracle)* 0,18s 0,26s 0,27s 0,326s 10,1s 13,77s *Size* 729KB 1,2 MB 1,3 MB 1,4MB 1,6 MB 2 MB Is there anywhere a Threshold? Because the time is increasing and I do not understand why. It does not refer to Browser problems. I think that this is depending on the webserver. Does someone knows this issue? -- *Moritz Liebelt, B. Sc.* Murgstraße 19 76437 Rastatt E-Mail: mor...@go... |
|
From: Moritz L. <mor...@go...> - 2016-08-23 15:40:04
|
Hi, I fetch rows form an Oracle Locator DB with web.py and cx_Oracle! I use the unreleased version of cx_Oracle! Furthermore I read the sdo_geometry objects in the sql-statement and build a json based on this. For a 500m-Tile with 4634 rows I need 0,326 (measured time in the browser= waiting time (DB + Webserver)) For a 500m-Tile with 5081 rows I need 10,1 second??? You can see an overview in the attachment. Is there anywhere a Threshold? Because the time is increasing and I do not understand why. It does not refer to Browser problems. I think that this is depending on the webserver. Does someone knows this issue? -- *Moritz Liebelt, B. Sc.* Murgstraße 19 76437 Rastatt E-Mail: mor...@go... |
31 messages has been excluded from this view by a project administrator.
