cx-oracle-users — Primary discussion list for users (and developers) of cx_Oracle

Re: [cx-oracle-users] Minimum client requirements for cx_Oracle

[D.R. B. <da...@as...>]

Hoi Nik,

You - and your customers - can use the Oracle Instant Client.
See

   http://www.oracle.com/technology/tech/oci/instantclient/instantclient.html

the "basic" and "sdk" parts of the OIC are sufficient for installing cx_Oracle.
You just have to change the setup.py a bit

   50c50
   <     includeDirs = ["rdbms/demo", "rdbms/public", "network/public", "sdk/include"]
   ---
   >     includeDirs = ["rdbms/demo", "rdbms/public", "network/public"]
   58c58
   <     libDirs = [oracleHome, libPath]
   ---
   >     libDirs = [libPath]


This just appends "sdk/include" to the include searchpath and prepends the value
of the ORACLE_HOME environment variable to the library searchpath.
Hmmm, Anthony, maybe something like this should go into the distributed
cx_Oracle setup.py as well.

This is as "lightweight" as you can get, apart from creating your own
cx_Oracle+Oracle libraries distribution.

Hoping that this helps,

                                                   Danny



On Fri, Oct 21, 2005 at 10:42:02AM +0100, Nik Barron wrote:
> Hi,
> 
> Does cx_Oracle require a full Oracle client installation, or is there a more
> "lightweight" option?
> 
> Currently this hasn't been an issue as our product requires the full client
> anyway, but the question's been raised by one of our support people as some
> customers' IT departments don't like rolling out the Oracle client to all
> users.
> 
> Regards,
> 
> Nick Barron
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users

[cx-oracle-users] Minimum client requirements for cx_Oracle

[Nik B. <Nik...@pe...>]

Hi,

Does cx_Oracle require a full Oracle client installation, or is there a more
"lightweight" option?

Currently this hasn't been an issue as our product requires the full client
anyway, but the question's been raised by one of our support people as some
customers' IT departments don't like rolling out the Oracle client to all
users.

Regards,

Nick Barron

Re: [cx-oracle-users] problems compiling cx_oracle in solaris

[Anthony T. <ant...@gm...>]

You could build gcc in your home directory but that also requires at
least a basic compiler. I thought that Solaris had a default compiler
which was simply very limited but I guess they've taken that away.

Yes, the cx_OracleTools and cx_OracleDBATools do have a binary for the
cx_Oracle package.

On 10/20/05, Gustavo Rahal <gu...@gr...> wrote:
> Hi
>
> When compiling cx_Oracle in Solaris I get the folowing output:
>
> running install
> running build
> running build_ext
> building 'cx_Oracle' extension
> cc -DNDEBUG -O -I/oracle/product/8.1.7/rdbms/demo
> -I/oracle/product/8.1.7/rdbms/public
> -I/oracle/product/8.1.7/network/public
> -I/home/a2i3978/python2.4.1/include/python2.4 -c cx_Oracle.c -o
> build/temp.solaris-2.8-sun4u-2.4/cx_Oracle.o -DBUILD_TIME=3D"October 20,
> 2005 16:37:05"
> /usr/ucb/cc:  language optional software package not installed
> error: command 'cc' failed with exit status 1
>
>
> I did some googling and found out
> (http://www.brandonhutchinson.com/usr_ucb_cc.html) that I need a compiler
> installed. The link suggests  SPROcc (unbundled SPARCworks Professional C
> compiler).
>
> The problem is that I don't have permissions to install packages. Is ther=
e
> any way to install it on my home dir?
>
> Do the cx_OracleDBATools and cx_OracleTools (that have solaries binaries)
> have the cx_Oracle module?
>
>
> Thanks
> Gustavo
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

[cx-oracle-users] problems compiling cx_oracle in solaris

[Gustavo R. <gu...@gr...>]

Hi

When compiling cx_Oracle in Solaris I get the folowing output:

running install
running build
running build_ext
building 'cx_Oracle' extension
cc -DNDEBUG -O -I/oracle/product/8.1.7/rdbms/demo
-I/oracle/product/8.1.7/rdbms/public
-I/oracle/product/8.1.7/network/public
-I/home/a2i3978/python2.4.1/include/python2.4 -c cx_Oracle.c -o
build/temp.solaris-2.8-sun4u-2.4/cx_Oracle.o -DBUILD_TIME="October 20,
2005 16:37:05"
/usr/ucb/cc:  language optional software package not installed
error: command 'cc' failed with exit status 1


I did some googling and found out
(http://www.brandonhutchinson.com/usr_ucb_cc.html) that I need a compiler
installed. The link suggests  SPROcc (unbundled SPARCworks Professional C
compiler).

The problem is that I don't have permissions to install packages. Is there
any way to install it on my home dir?

Do the cx_OracleDBATools and cx_OracleTools (that have solaries binaries)
have the cx_Oracle module?


Thanks
Gustavo

Re: [cx-oracle-users] Paramstyle (RE: cx_Oracle 4.1)

[Luiz C. G. <lui...@gm...>]

Now I saw it...and I've found this 'solution':
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278612

Cheers,
Luiz Carlos Geron


On 10/19/05, Anthony Tuininga <ant...@gm...> wrote:
> You can't truly choose what paramstyle you would like to use. Oracle
> only natively supports named and numeric paramstyles and cx_Oracle
> supports both of those. If you want anything else you will have to
> write code for it yourself. That said, its fairly straightforward to
> translate between qmark and numeric -- they both accept a list of
> arguments. You simply have to convert the SQL that is passed in,
> replacing each ? with a corresponding :<n>. If you create a subclass
> of cx_Oracle.Cursor and override the execute() and prepare() methods
> you can do precisely that.
>
> Note that your desire for choosing the paramstyle is shared by a fair
> number of others. This debate is held on the db-sig mailing list every
> once in a while but no solution ever appears. :-)
>
> On 10/19/05, Luiz Carlos Geron <lui...@gm...> wrote:
> > Hi All,
> > From http://sourceforge.net/mailarchive/message.php?msg_id=3D10645555 :
> >
> > " 13) Added support for binding by position (paramstyle =3D "numeric").=
"
> >
> > I want to choose what paramstyle to use in my code. There is one
> > reason for this: I want to write such a code that can be used with
> > diferent db modules (like kinterbasdb, mssql, whatever) without
> > modifiyng it. The kinterbasdb module simply doesn't support other
> > paramstyle than qmark, then I have to use the same paramstyle on
> > cx_oracle and other possible modules.  How can I choose what
> > paramstyle to use with cx_oracle? If I can't, how can I solve this
> > problem, using the same code to other databases ? (for now, I consider
> > only Firebird)
> >
> > Thanks, and sorry for my bad english
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by:
> > Power Architecture Resource Center: Free content, downloads, discussion=
s,
> > and more. http://solutions.newsforge.com/ibmarch.tmpl
> > _______________________________________________
> > cx-oracle-users mailing list
> > cx-...@li...
> > https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
> >
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

Re: [cx-oracle-users] Paramstyle (RE: cx_Oracle 4.1)

[Anthony T. <ant...@gm...>]

You can't truly choose what paramstyle you would like to use. Oracle
only natively supports named and numeric paramstyles and cx_Oracle
supports both of those. If you want anything else you will have to
write code for it yourself. That said, its fairly straightforward to
translate between qmark and numeric -- they both accept a list of
arguments. You simply have to convert the SQL that is passed in,
replacing each ? with a corresponding :<n>. If you create a subclass
of cx_Oracle.Cursor and override the execute() and prepare() methods
you can do precisely that.

Note that your desire for choosing the paramstyle is shared by a fair
number of others. This debate is held on the db-sig mailing list every
once in a while but no solution ever appears. :-)

On 10/19/05, Luiz Carlos Geron <lui...@gm...> wrote:
> Hi All,
> From http://sourceforge.net/mailarchive/message.php?msg_id=3D10645555 :
>
> " 13) Added support for binding by position (paramstyle =3D "numeric")."
>
> I want to choose what paramstyle to use in my code. There is one
> reason for this: I want to write such a code that can be used with
> diferent db modules (like kinterbasdb, mssql, whatever) without
> modifiyng it. The kinterbasdb module simply doesn't support other
> paramstyle than qmark, then I have to use the same paramstyle on
> cx_oracle and other possible modules.  How can I choose what
> paramstyle to use with cx_oracle? If I can't, how can I solve this
> problem, using the same code to other databases ? (for now, I consider
> only Firebird)
>
> Thanks, and sorry for my bad english
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

[cx-oracle-users] Paramstyle (RE: cx_Oracle 4.1)

[Luiz C. G. <lui...@gm...>]

Hi All,
From http://sourceforge.net/mailarchive/message.php?msg_id=3D10645555 :

" 13) Added support for binding by position (paramstyle =3D "numeric")."

I want to choose what paramstyle to use in my code. There is one
reason for this: I want to write such a code that can be used with
diferent db modules (like kinterbasdb, mssql, whatever) without
modifiyng it. The kinterbasdb module simply doesn't support other
paramstyle than qmark, then I have to use the same paramstyle on
cx_oracle and other possible modules.  How can I choose what
paramstyle to use with cx_oracle? If I can't, how can I solve this
problem, using the same code to other databases ? (for now, I consider
only Firebird)

Thanks, and sorry for my bad english

Re: [cx-oracle-users] Re: python 2.3 + cx_oracle on HP-UX Itanium

[Anthony T. <ant...@gm...>]

Since you can't verify I've added a comment indicating that on HP-UX
Itanium with Oracle 10g, the library "ttsh10" must be linked against.
If anyone can provide me a programmatic way of determing when that
library is necessary I'll replace the comment with the code. Thank
you.

On 10/10/05, Bernard Delm=E9e <bd...@ad...> wrote:
>
> > Could you give me the output from "sys.platform"? I can use that to
> > automatically include in the setup.py the library "ttsh10" in the
> > libraries for that platform -- if you're reasonably certain that this
> > will be true for all users of that platform.
>
> Unfortunately I am really not sure about this - sys.platform simply
> reports "hp-ux11" and does not even mention itanium. I do not have
> access to oracle 10g on HP-PA to check if the problem is there as
> well...
>
> Bernard.
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

[cx-oracle-users] Re: python 2.3 + cx_oracle on HP-UX Itanium

[<bd...@ad...>]

> Could you give me the output from "sys.platform"? I can use that to
> automatically include in the setup.py the library "ttsh10" in the
> libraries for that platform -- if you're reasonably certain that this
> will be true for all users of that platform.

Unfortunately I am really not sure about this - sys.platform simply
reports "hp-ux11" and does not even mention itanium. I do not have
access to oracle 10g on HP-PA to check if the problem is there as
well...

Bernard.

Re: [cx-oracle-users] Re: python 2.3 + cx_oracle on HP-UX Itanium

[Anthony T. <ant...@gm...>]

Could you give me the output from "sys.platform"? I can use that to
automatically include in the setup.py the library "ttsh10" in the
libraries for that platform -- if you're reasonably certain that this
will be true for all users of that platform.

On 10/6/05, Bernard Delm=E9e <bd...@ad...> wrote:
> Better late than never ;-)
>
> David Ripton provided the solution to the puzzle that is getting
> cx_oracle to work on hpux / itanium / ora 10g: one needs to add
> "ttsh10" to the libs line In setup.py. (eg libs=3D["clntsh","ttsh10"])
>
> Bernard.
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

[cx-oracle-users] Re: python 2.3 + cx_oracle on HP-UX Itanium

[<bd...@ad...>]

Better late than never ;-)

David Ripton provided the solution to the puzzle that is getting
cx_oracle to work on hpux / itanium / ora 10g: one needs to add
"ttsh10" to the libs line In setup.py. (eg libs=["clntsh","ttsh10"])

Bernard.

[cx-oracle-users] Workarounds for Cygwin compile - still have a problem

[Dale S. <DS...@Al...>]

Hi there,

I installed cx_Oracle using the Windows installer and had no problems. =
But I'd also like to have cx_Oracle available in Cygwin. I downloaded =
the source and unzipped to /cygdrive/c/Temp. I then ran

python setup.py build --compler=3Dmingw32 install

1.  gcc reported that it could not find cc1. I tried to un-install and =
re-install gcc using the Cygwin setup program, and noticed that cc1.exe =
was definitely being installed. I added the cc1.exe location to my path. =
That solved the first problem.

2.  The linker complained that it couldn't find the file -loci. Of =
course, the file it couldn't find was liboci.a. I used the following web =
page to make liboci.a and placed it in /cygdrive/c/oracle/ora92/bin

http://www.python.org/doc/current/inst/tweak-flags.html

See the section for MingW to create the .a file needed.

3.  I had to symlink libpython2.4.a to libpython2.4.dll.a as the linker =
complained about -lpython2.4.

4.  -lgcc was not found either. I did a find and modified the setup.py =
file so it has

elif sys.platform =3D=3D "cygwin":
    includeDirs =3D ["/usr/include", "rdbms/demo", "rdbms/public", \
            "network/public", "oci/include", "/lib", "/usr/lib", \
                   "/lib/gcc/i686-pc-cygwin/3.4.4", \
                   "/usr/lib/gcc/i686-pc-cygwin/3.4.4"]
    libDirs =3D ["bin", "lib", "/lib", "/usr/lib", \
                   "/lib/gcc/i686-pc-cygwin/3.4.4", \
                   "/usr/lib/gcc/i686-pc-cygwin/3.4.4"]

At that point, everything worked.

My test script (see below) reported a runtime error when calling =
connect:

Unable to acquire Oracle environment handle

ORACLE_HOME is defined and points to the correct place. When connecting =
with sqlplus under Cygwin, I get the message

Error 6 initializing SQL*Plus
Message file sp1<lang>.msb not found
SP2-0750: You may need to set ORACLE_HOME to your Oracle software =
directory

I looked around using Google. I set the appropriate read and execute =
permissions on oracle/ora92 and sub-directories in bash. It appears that =
the NLS_LANG variable needs to be set in the bash shell. I did that =
according to directions in

http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq=
.htm

but it didn't help. I'm still working on this problem, so if anyone has =
information please email me directly.

test1.py:

import os
import sys
import cx_Oracle

uname =3D "dsmith"
passwd =3D "xxx"
tnsentry =3D "yyy"
conn =3D cx_Oracle.connect(uname, passwd, tnsentry)
cursor =3D conn.cursor()
cursor.execute("select count(*) from tbl")
count, =3D cursor.fetchone()
print "rowcount is %d" % count
conn.close()

sys.exit()

Regards,
Dale Smith, Ph.D.
Aladdin Capital Management LLC
Three Landmark Square
Stamford, CT 06901
Phone: 203-487-6768
Fax: 203-326-7902
ds...@al...

Re: [cx-oracle-users] assistance in analyzing memory usage?

[Mark H. <mh...@pi...>]

Hancock, David (DHANCOCK) wrote:
> Sorry to say I haven't got any solution to offer, but just a question: How does your memsnap() function work? I just hacked something like it to help us with other memory leak problems, but I'd like to see another implementation, because it's sure to be better than mine (mine saves sys.argv[0] to a string, and does a "ps auxww | grep 'that string' | grep -v grep"
> 
> Thanks, and
> Cheers!

Heh, yours is probably better than mine coz I did the same and just hard-coded the
executable name.

Here's a nice grep tip:  if you put the first character of the grep pattern
in square brackets you won't need the final grep -v, coz the string "[q]"
doesn't match the regular expression "[q]".

def memsnap(note):
     print '---', note
     os.system("ps auxww|grep '[q]2'")

If you don't mind a linux-only solution, you can open /proc/PID/status
where the memory stats are available in a nicely parsable form.

I've also been pointed to memusage on linux which will produce nice graphs.

HTH!
Mark

-- 
Mark Harrison
Pixar Animation Studio

Re: [cx-oracle-users] assistance in analyzing memory usage?

[Anthony T. <ant...@gm...>]

Hmm, I just tried something similar to your code (attached) which does
not seem to have any memory leak at all. The code for showing memory
usage only happens to work under Linux but there is no memory growth
at all which suggests that cx_Oracle is not the culprit in this
particular case over some 250000 rows. Could you indicate what=20
version of cx_Oracle, platform and Oracle version you are using? I am
using Linux, Oracle 9i and cx_Oracle 4.1.

On 9/28/05, Mark Harrison <mh...@pi...> wrote:
> I'm seeing a memory leak when running this program which just
> loops over a table and fetches its data.
>
> Is there something I should be doing after a fetchmany to
> free up some memory?
>
> Attached below are some cleaned up output and the test program.
> The "job" table has quite a few fields, including clob's.  Should
> I be doing something special with them?
>
> Many TIA!!
> Mark
>
> --
> Mark Harrison
> Pixar Animation Studios
>
>
>
> USER       PID %CPU %MEM   VSZ  RSS
> --- starting
> mh       29770  0.0  0.0 54044 3260
> --- import cx_Oracle
> mh       29770  0.0  0.0 54044 3260
> --- connect
> mh       29770  0.0  0.1 75000 5640
> --- cursor
> mh       29770  0.0  0.2 141904 9860
> --- arraysize
> mh       29770  0.0  0.2 141904 9864
> --- execute
> mh       29770  0.0  0.2 141904 9864
> --- fetchmany
> mh       29770  0.0  0.6 195228 27308
> mh       29770  0.0  2.0 218652 82924
> mh       29770  0.0  2.5 236252 101356
> mh       29770  0.0  2.5 236956 102428
> mh       29770  0.0  2.5 237508 102968
> mh       29770  0.0  2.5 238068 103696
> mh       29770  0.0  2.5 238476 104088
> mh       29770  0.0  2.5 238880 104540
> mh       29770  0.0  2.5 239428 105108
> mh       29770  0.0  2.6 239844 105628
> mh       29770  0.0  2.6 240264 106048
>
>
> import time
> import os
> os.system("ps auxww|head -n 1")
> memsnap('starting')
> memsnap('import cx_Oracle')
> import cx_Oracle
>
> user     =3D 'zzz'
> passwd   =3D 'zzz'
> connurl =3D 'racdb'
>
> os.environ['TNS_ADMIN']=3D'/u0/mh/oracle'
>
> memsnap('connect')
> connection =3D cx_Oracle.connect(user, passwd, connurl)
>
>
> memsnap('cursor')
> cursor =3D connection.cursor() memsnap('arraysize')
> cursor.arraysize =3D 10000
>
> memsnap('execute')
> cursor.execute('select * from job')
> done =3D 0
> tot =3D 0
> print '---'
> while not done:
>      memsnap('fetchmany')
>      result =3D cursor.fetchmany(10000)
>      dir(result)
>      tot +=3D len(result)
>      if len(result) =3D=3D 0:
>          done =3D 1
>      result =3D []
>
> cursor.close()
>
> connection.close()
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

RE: [cx-oracle-users] assistance in analyzing memory usage?

[Hancock, D. \(DHANCOCK\) <DHA...@ar...>]

Sorry to say I haven't got any solution to offer, but just a question: =
How does your memsnap() function work? I just hacked something like it =
to help us with other memory leak problems, but I'd like to see another =
implementation, because it's sure to be better than mine (mine saves =
sys.argv[0] to a string, and does a "ps auxww | grep 'that string' | =
grep -v grep"

Thanks, and
Cheers!
--
David Hancock | dha...@ar...


-----Original Message-----
From: cx-...@li... on behalf of Mark =
Harrison
Sent: Wed 9/28/2005 6:43 PM
To: cx-...@li...
Subject: [cx-oracle-users] assistance in analyzing memory usage?
=20
I'm seeing a memory leak when running this program which just
loops over a table and fetches its data.

Is there something I should be doing after a fetchmany to
free up some memory?

Attached below are some cleaned up output and the test program.
The "job" table has quite a few fields, including clob's.  Should
I be doing something special with them?

Many TIA!!
Mark

--
Mark Harrison
Pixar Animation Studios



USER       PID %CPU %MEM   VSZ  RSS
--- starting
mh       29770  0.0  0.0 54044 3260
--- import cx_Oracle
mh       29770  0.0  0.0 54044 3260
--- connect
mh       29770  0.0  0.1 75000 5640
--- cursor
mh       29770  0.0  0.2 141904 9860
--- arraysize
mh       29770  0.0  0.2 141904 9864
--- execute
mh       29770  0.0  0.2 141904 9864
--- fetchmany
mh       29770  0.0  0.6 195228 27308
mh       29770  0.0  2.0 218652 82924
mh       29770  0.0  2.5 236252 101356
mh       29770  0.0  2.5 236956 102428
mh       29770  0.0  2.5 237508 102968
mh       29770  0.0  2.5 238068 103696
mh       29770  0.0  2.5 238476 104088
mh       29770  0.0  2.5 238880 104540
mh       29770  0.0  2.5 239428 105108
mh       29770  0.0  2.6 239844 105628
mh       29770  0.0  2.6 240264 106048


import time
import os
os.system("ps auxww|head -n 1")
memsnap('starting')
memsnap('import cx_Oracle')
import cx_Oracle

user     =3D 'zzz'
passwd   =3D 'zzz'
connurl =3D 'racdb'

os.environ['TNS_ADMIN']=3D'/u0/mh/oracle'

memsnap('connect')
connection =3D cx_Oracle.connect(user, passwd, connurl)


memsnap('cursor')
cursor =3D connection.cursor() memsnap('arraysize')
cursor.arraysize =3D 10000

memsnap('execute')
cursor.execute('select * from job')
done =3D 0
tot =3D 0
print '---'
while not done:
     memsnap('fetchmany')
     result =3D cursor.fetchmany(10000)
     dir(result)
     tot +=3D len(result)
     if len(result) =3D=3D 0:
         done =3D 1
     result =3D []

cursor.close()

connection.close()


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, =
discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
cx-oracle-users mailing list
cx-...@li...
https://lists.sourceforge.net/lists/listinfo/cx-oracle-users

[cx-oracle-users] assistance in analyzing memory usage?

[Mark H. <mh...@pi...>]

I'm seeing a memory leak when running this program which just
loops over a table and fetches its data.

Is there something I should be doing after a fetchmany to
free up some memory?

Attached below are some cleaned up output and the test program.
The "job" table has quite a few fields, including clob's.  Should
I be doing something special with them?

Many TIA!!
Mark

--
Mark Harrison
Pixar Animation Studios



USER       PID %CPU %MEM   VSZ  RSS
--- starting
mh       29770  0.0  0.0 54044 3260
--- import cx_Oracle
mh       29770  0.0  0.0 54044 3260
--- connect
mh       29770  0.0  0.1 75000 5640
--- cursor
mh       29770  0.0  0.2 141904 9860
--- arraysize
mh       29770  0.0  0.2 141904 9864
--- execute
mh       29770  0.0  0.2 141904 9864
--- fetchmany
mh       29770  0.0  0.6 195228 27308
mh       29770  0.0  2.0 218652 82924
mh       29770  0.0  2.5 236252 101356
mh       29770  0.0  2.5 236956 102428
mh       29770  0.0  2.5 237508 102968
mh       29770  0.0  2.5 238068 103696
mh       29770  0.0  2.5 238476 104088
mh       29770  0.0  2.5 238880 104540
mh       29770  0.0  2.5 239428 105108
mh       29770  0.0  2.6 239844 105628
mh       29770  0.0  2.6 240264 106048


import time
import os
os.system("ps auxww|head -n 1")
memsnap('starting')
memsnap('import cx_Oracle')
import cx_Oracle

user     = 'zzz'
passwd   = 'zzz'
connurl = 'racdb'

os.environ['TNS_ADMIN']='/u0/mh/oracle'

memsnap('connect')
connection = cx_Oracle.connect(user, passwd, connurl)


memsnap('cursor')
cursor = connection.cursor() memsnap('arraysize')
cursor.arraysize = 10000

memsnap('execute')
cursor.execute('select * from job')
done = 0
tot = 0
print '---'
while not done:
     memsnap('fetchmany')
     result = cursor.fetchmany(10000)
     dir(result)
     tot += len(result)
     if len(result) == 0:
         done = 1
     result = []

cursor.close()

connection.close()

RE: [cx-oracle-users] OCI-22054 error using cx_Oracle

[Molina, G. <Ger...@sc...>]

That did the trick!

Thanks,
Gerardo=20

-----Original Message-----
From: cx-...@li...
[mailto:cx-...@li...] On Behalf Of
Anthony Tuininga
Sent: Thursday, September 22, 2005 2:42 PM
To: cx-...@li...
Subject: Re: [cx-oracle-users] OCI-22054 error using cx_Oracle

The reason this is happening is because you are doing the following:

a) calling a procedure with out parameters that specify number
b) passing in numbers which are integers

Of course, Oracle is trying to return a floating point number at this
point and is failing on converting the number back to an integer......
If you change all of your =3D 0 to =3D 0.0 the problem goes away.

On 9/22/05, Molina, Gerardo <Ger...@sc...> wrote:
>
>
> I'm getting the following error:
>
> Traceback (most recent call last):
>   File "./pydbconn.py", line 113, in ?
>     (bid, eid,
> cx_Oracle.DatabaseError: OCI-22054: underflow error
>
> Google didn't turn up much on this error.
>
> Any help would be greatly appreciated.
>
> TIA,
> Gerardo
>
>
> Here is program listing:
>
> #!/usr/bin/env python
>
> # pydbconn
>
> # test sql connect
>
> import cx_Oracle
>
> user =3D 'perfstat'
> pword =3D 'itsnotthis'
> dbstring =3D 'TESTDB'
>
> connection =3D cx_Oracle.connect(user,pword,dbstring)
>
> bid =3D 11
> eid =3D 12
> dbid =3D 4275102739
> inst_num =3D 1
> para =3D 'NO'
>
> lhtr =3D 0
> bfwt =3D 0
> tran =3D 0
> chng =3D 0
> ucal =3D 0
> urol =3D 0
> rsiz =3D 0
> phyr =3D 0
> phyrd =3D 0
> phyrdl =3D 0
> phyw =3D 0
> ucom =3D 0
> prse =3D 0
> hprs =3D 0
> recr =3D 0
> gets =3D 0
> rlsr =3D 0
> rent =3D 0
> srtm =3D 0
> srtd =3D 0
> srtr =3D 0
> strn =3D 0
> lhr =3D 0
> #bc =3D ''
> sp =3D ''
> lb =3D ''
> bs =3D ''
> twt =3D 0
> logc =3D 0
> prscpu =3D 0
> tcpu =3D 0
> exe =3D 0
> prsela =3D 0
> bspm =3D 0
> espm =3D 0
> bfrm =3D 0
> efrm =3D 0
> blog =3D 0
> elog =3D 0
> bocur =3D 0
> eocur =3D 0
> dmsd =3D 0
> dmfc =3D 0
> dmsi =3D 0
> pmrv =3D 0
> pmpt =3D 0
> npmrv =3D 0
> npmpt =3D 0
> dbfr =3D 0
> dpms =3D 0
> dnpms =3D 0
> glsg =3D 0
> glag =3D 0
> glgt =3D 0
> glsc =3D 0
> glac =3D 0
> glct =3D 0
> glrl =3D 0
> gcdfr =3D 0
> gcge =3D 0
> gcgt =3D 0
> gccv =3D 0
> gcct =3D 0
> gccrrv =3D 0
> gccrrt =3D 0
> gccurv =3D 0
> gccurt =3D 0
> gccrsv =3D 0
> gccrbt =3D 0
> gccrft =3D 0
> gccrst =3D 0
> gccusv =3D 0
> gccupt =3D 0
> gccuft =3D 0
> gccust =3D 0
> msgsq =3D 0
> msgsqt =3D 0
> msgsqk =3D 0
> msgsqtk =3D 0
> msgrq =3D 0
> msgrqt =3D 0
>
> # Call proc statspack.stat_changes
>
> cursor2 =3D connection.cursor()
>
> outbcVar =3D cursor2.var(cx_Oracle.STRING) outspVar =3D=20
> cursor2.var(cx_Oracle.STRING) outlbVar =3D =
cursor2.var(cx_Oracle.STRING)

> outbsVar =3D cursor2.var(cx_Oracle.STRING)
>
> cursor2.callproc('statspack.stat_changes',
>     (bid, eid,
>      dbid, inst_num,
>      para,
>      lhtr, bfwt,
>      tran, chng,
>      ucal, urol,
>      rsiz,
>      phyr, phyrd,
>      phyrdl,
>      phyw, ucom,
>      prse, hprs,
>      recr, gets,
>      rlsr, rent,
>      srtm, srtd,
>      srtr, strn,
>      lhr,  outbcVar,
>      outspVar,   outlbVar,
>      outbsVar,   twt,
>      logc, prscpu,
>      tcpu, exe,
>      prsela,
>      bspm, espm,
>      bfrm, efrm,
>      blog, elog,
>      bocur, eocur,
>      dmsd, dmfc,
>      dmsi,
>      pmrv, pmpt,
>      npmrv, npmpt,
>      dbfr,
>      dpms, dnpms,
>      glsg, glag,
>      glgt, glsc,
>      glac, glct,
>      glrl, gcdfr,
>      gcge, gcgt,
>      gccv, gcct,
>      gccrrv, gccrrt, gccurv, gccurt,
>      gccrsv,
>      gccrbt, gccrft,
>      gccrst, gccusv,
>      gccupt, gccuft,
>      gccust,
>      msgsq, msgsqt,
>      msgsqk, msgsqtk,
>      msgrq, msgrqt ))


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your
very own Sony(tm)PSP.  Click here to play:
http://sourceforge.net/geronimo.php
_______________________________________________
cx-oracle-users mailing list
cx-...@li...
https://lists.sourceforge.net/lists/listinfo/cx-oracle-users

[cx-oracle-users] AW: confirmation of subscription -- request 771604

[<Rei...@t-...>]

-----Urspr=FCngliche Nachricht-----
Von: cx-...@li... =
[mailto:cx-...@li...] Im Auftrag von =
cx-...@li...
Gesendet: Montag, 26. September 2005 05:16
An: cx-...@li...
Betreff: cx-oracle-users digest, Vol 1 #147 - 1 msg


Send cx-oracle-users mailing list submissions to
	cx-...@li...

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
or, via email, send a message with subject or body 'help' to
	cx-...@li...

You can reach the person managing the list at
	cx-...@li...

When replying, please edit your Subject line so it is more specific than =
"Re: Contents of cx-oracle-users digest..."


Today's Topics:

   1. re: cx-oracle-users -- confirmation of subscription -- request =
771604 (Chimezie Ogbuji)

--__--__--

Message: 1
Date: Sun, 25 Sep 2005 20:09:20 -0400 (EDT)
From: Chimezie Ogbuji <og...@bi...>
To: cx-...@li...
Subject: [cx-oracle-users] re: cx-oracle-users -- confirmation of =
subscription -- request 771604
Reply-To: cx-...@li...

confirm 771604



--__--__--

_______________________________________________
cx-oracle-users mailing list cx-...@li...
https://lists.sourceforge.net/lists/listinfo/cx-oracle-users


End of cx-oracle-users Digest

[cx-oracle-users] re: cx-oracle-users -- confirmation of subscription -- request 771604

[Chimezie O. <og...@bi...>]

confirm 771604

Re: [cx-oracle-users] OCI-22054 error using cx_Oracle

[Anthony T. <ant...@gm...>]

The reason this is happening is because you are doing the following:

a) calling a procedure with out parameters that specify number
b) passing in numbers which are integers

Of course, Oracle is trying to return a floating point number at this
point and is failing on converting the number back to an integer......
If you change all of your =3D 0 to =3D 0.0 the problem goes away.

On 9/22/05, Molina, Gerardo <Ger...@sc...> wrote:
>
>
> I'm getting the following error:
>
> Traceback (most recent call last):
>   File "./pydbconn.py", line 113, in ?
>     (bid, eid,
> cx_Oracle.DatabaseError: OCI-22054: underflow error
>
> Google didn't turn up much on this error.
>
> Any help would be greatly appreciated.
>
> TIA,
> Gerardo
>
>
> Here is program listing:
>
> #!/usr/bin/env python
>
> # pydbconn
>
> # test sql connect
>
> import cx_Oracle
>
> user =3D 'perfstat'
> pword =3D 'itsnotthis'
> dbstring =3D 'TESTDB'
>
> connection =3D cx_Oracle.connect(user,pword,dbstring)
>
> bid =3D 11
> eid =3D 12
> dbid =3D 4275102739
> inst_num =3D 1
> para =3D 'NO'
>
> lhtr =3D 0
> bfwt =3D 0
> tran =3D 0
> chng =3D 0
> ucal =3D 0
> urol =3D 0
> rsiz =3D 0
> phyr =3D 0
> phyrd =3D 0
> phyrdl =3D 0
> phyw =3D 0
> ucom =3D 0
> prse =3D 0
> hprs =3D 0
> recr =3D 0
> gets =3D 0
> rlsr =3D 0
> rent =3D 0
> srtm =3D 0
> srtd =3D 0
> srtr =3D 0
> strn =3D 0
> lhr =3D 0
> #bc =3D ''
> sp =3D ''
> lb =3D ''
> bs =3D ''
> twt =3D 0
> logc =3D 0
> prscpu =3D 0
> tcpu =3D 0
> exe =3D 0
> prsela =3D 0
> bspm =3D 0
> espm =3D 0
> bfrm =3D 0
> efrm =3D 0
> blog =3D 0
> elog =3D 0
> bocur =3D 0
> eocur =3D 0
> dmsd =3D 0
> dmfc =3D 0
> dmsi =3D 0
> pmrv =3D 0
> pmpt =3D 0
> npmrv =3D 0
> npmpt =3D 0
> dbfr =3D 0
> dpms =3D 0
> dnpms =3D 0
> glsg =3D 0
> glag =3D 0
> glgt =3D 0
> glsc =3D 0
> glac =3D 0
> glct =3D 0
> glrl =3D 0
> gcdfr =3D 0
> gcge =3D 0
> gcgt =3D 0
> gccv =3D 0
> gcct =3D 0
> gccrrv =3D 0
> gccrrt =3D 0
> gccurv =3D 0
> gccurt =3D 0
> gccrsv =3D 0
> gccrbt =3D 0
> gccrft =3D 0
> gccrst =3D 0
> gccusv =3D 0
> gccupt =3D 0
> gccuft =3D 0
> gccust =3D 0
> msgsq =3D 0
> msgsqt =3D 0
> msgsqk =3D 0
> msgsqtk =3D 0
> msgrq =3D 0
> msgrqt =3D 0
>
> # Call proc statspack.stat_changes
>
> cursor2 =3D connection.cursor()
>
> outbcVar =3D cursor2.var(cx_Oracle.STRING)
> outspVar =3D cursor2.var(cx_Oracle.STRING)
> outlbVar =3D cursor2.var(cx_Oracle.STRING)
> outbsVar =3D cursor2.var(cx_Oracle.STRING)
>
> cursor2.callproc('statspack.stat_changes',
>     (bid, eid,
>      dbid, inst_num,
>      para,
>      lhtr, bfwt,
>      tran, chng,
>      ucal, urol,
>      rsiz,
>      phyr, phyrd,
>      phyrdl,
>      phyw, ucom,
>      prse, hprs,
>      recr, gets,
>      rlsr, rent,
>      srtm, srtd,
>      srtr, strn,
>      lhr,  outbcVar,
>      outspVar,   outlbVar,
>      outbsVar,   twt,
>      logc, prscpu,
>      tcpu, exe,
>      prsela,
>      bspm, espm,
>      bfrm, efrm,
>      blog, elog,
>      bocur, eocur,
>      dmsd, dmfc,
>      dmsi,
>      pmrv, pmpt,
>      npmrv, npmpt,
>      dbfr,
>      dpms, dnpms,
>      glsg, glag,
>      glgt, glsc,
>      glac, glct,
>      glrl, gcdfr,
>      gcge, gcgt,
>      gccv, gcct,
>      gccrrv, gccrrt, gccurv, gccurt,
>      gccrsv,
>      gccrbt, gccrft,
>      gccrst, gccusv,
>      gccupt, gccuft,
>      gccust,
>      msgsq, msgsqt,
>      msgsqk, msgsqtk,
>      msgrq, msgrqt ))

Re: [cx-oracle-users] cx_Oracle equivalent of MySQL's DictCursor?

[Anthony T. <ant...@gm...>]

Not directly but that doesn't mean you can't do it yourself. Subclass
from the Cursor class and have at it.... There are a number of ways of
doing this depending on whether simplicity of code or performance is
more important to you.

On 9/23/05, Mark Harrison <mh...@pi...> wrote:
> I've just received this query.  Does anyone know of such a thing?
>
> The MySQL-python-1.2.0 doc describes it as:
>
>      DictCursor
>          Like ``Cursor`` except it returns rows as dictionaries.
>
> > We use MySQL's DictCursor as a cursor for queries. It looks like this
> > cursor will present results as a dict. Do you know if the Oracle obj
> > supports this type of cursor?
>
> Many TIA!
> Mark
>
> --
> Mark Harrison
> Pixar Animation Studio
> Emeryville, CA
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server. Downl=
oad
> it for free - -and be entered to win a 42" plasma tv or your very own
> Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>

[cx-oracle-users] cx_Oracle equivalent of MySQL's DictCursor?

[Mark H. <mh...@pi...>]

I've just received this query.  Does anyone know of such a thing?

The MySQL-python-1.2.0 doc describes it as:

     DictCursor
         Like ``Cursor`` except it returns rows as dictionaries.

> We use MySQL's DictCursor as a cursor for queries. It looks like this
> cursor will present results as a dict. Do you know if the Oracle obj
> supports this type of cursor?

Many TIA!
Mark

-- 
Mark Harrison
Pixar Animation Studio
Emeryville, CA

[cx-oracle-users] OCI-22054 error using cx_Oracle

[Molina, G. <Ger...@sc...>]

I'm getting the following error:

Traceback (most recent call last):
  File "./pydbconn.py", line 113, in ?
    (bid, eid,
cx_Oracle.DatabaseError: OCI-22054: underflow error

Google didn't turn up much on this error.

Any help would be greatly appreciated.

TIA,
Gerardo


Here is program listing:

#!/usr/bin/env python

# pydbconn

# test sql connect

import cx_Oracle

user =3D 'perfstat'
pword =3D 'itsnotthis'
dbstring =3D 'TESTDB'

connection =3D cx_Oracle.connect(user,pword,dbstring)

bid =3D 11
eid =3D 12
dbid =3D 4275102739
inst_num =3D 1
para =3D 'NO'

lhtr =3D 0
bfwt =3D 0
tran =3D 0
chng =3D 0
ucal =3D 0
urol =3D 0
rsiz =3D 0
phyr =3D 0
phyrd =3D 0
phyrdl =3D 0=20
phyw =3D 0
ucom =3D 0
prse =3D 0
hprs =3D 0
recr =3D 0
gets =3D 0
rlsr =3D 0
rent =3D 0
srtm =3D 0
srtd =3D 0
srtr =3D 0
strn =3D 0
lhr =3D 0
#bc =3D ''
sp =3D ''
lb =3D ''
bs =3D ''
twt =3D 0
logc =3D 0
prscpu =3D 0
tcpu =3D 0
exe =3D 0
prsela =3D 0
bspm =3D 0
espm =3D 0
bfrm =3D 0
efrm =3D 0
blog =3D 0
elog =3D 0
bocur =3D 0
eocur =3D 0
dmsd =3D 0
dmfc =3D 0
dmsi =3D 0
pmrv =3D 0
pmpt =3D 0
npmrv =3D 0
npmpt =3D 0
dbfr =3D 0
dpms =3D 0
dnpms =3D 0
glsg =3D 0
glag =3D 0
glgt =3D 0
glsc =3D 0
glac =3D 0
glct =3D 0
glrl =3D 0
gcdfr =3D 0
gcge =3D 0
gcgt =3D 0
gccv =3D 0
gcct =3D 0
gccrrv =3D 0
gccrrt =3D 0
gccurv =3D 0
gccurt =3D 0
gccrsv =3D 0
gccrbt =3D 0
gccrft =3D 0
gccrst =3D 0
gccusv =3D 0
gccupt =3D 0
gccuft =3D 0
gccust =3D 0
msgsq =3D 0
msgsqt =3D 0
msgsqk =3D 0
msgsqtk =3D 0
msgrq =3D 0
msgrqt =3D 0

# Call proc statspack.stat_changes

cursor2 =3D connection.cursor()

outbcVar =3D cursor2.var(cx_Oracle.STRING)
outspVar =3D cursor2.var(cx_Oracle.STRING)
outlbVar =3D cursor2.var(cx_Oracle.STRING)
outbsVar =3D cursor2.var(cx_Oracle.STRING)

cursor2.callproc('statspack.stat_changes',
    (bid, eid,
     dbid, inst_num,
     para,
     lhtr, bfwt,
     tran, chng,
     ucal, urol,
     rsiz,
     phyr, phyrd,
     phyrdl,
     phyw, ucom,
     prse, hprs,
     recr, gets,
     rlsr, rent,
     srtm, srtd,
     srtr, strn,
     lhr,  outbcVar,
     outspVar,   outlbVar,
     outbsVar,   twt,
     logc, prscpu,
     tcpu, exe,
     prsela,
     bspm, espm,
     bfrm, efrm,
     blog, elog,
     bocur, eocur,
     dmsd, dmfc,
     dmsi,
     pmrv, pmpt,
     npmrv, npmpt,
     dbfr,
     dpms, dnpms,
     glsg, glag,
     glgt, glsc,
     glac, glct,
     glrl, gcdfr,
     gcge, gcgt,
     gccv, gcct,
     gccrrv, gccrrt, gccurv, gccurt,
     gccrsv,
     gccrbt, gccrft,
     gccrst, gccusv,
     gccupt, gccuft,
     gccust,
     msgsq, msgsqt,
     msgsqk, msgsqtk,
     msgrq, msgrqt ))

Re: [cx-oracle-users] is SessionPool thread safe?

[Forest W. <mo...@ti...>]

Yes, I understand now, thanks.  Too bad Oracle's docs aren't more 
explicit.  :)

Anthony Tuininga <ant...@gm...> wrote:

> Ok, I'll try again. :-) As far as I know the session pool acquire
> method is thread safe. There is no Oracle documentation that says
> otherwise and the description of OCI_THREADED mode seems to imply that
> it is thread safe. We have also done precisely what you are doing
> (acquiring a connection from a pool in a worker thread) in our own
> product and have never run into a problem. If we forget to set
> "threaded =3D True" in the session pool constructor, though, all kinds
> of trouble occurs. :-) Does that answer your question sufficiently
> now?

Re: [cx-oracle-users] is SessionPool thread safe?

[Anthony T. <ant...@gm...>]

Ok, I'll try again. :-) As far as I know the session pool acquire
method is thread safe. There is no Oracle documentation that says
otherwise and the description of OCI_THREADED mode seems to imply that
it is thread safe. We have also done precisely what you are doing
(acquiring a connection from a pool in a worker thread) in our own
product and have never run into a problem. If we forget to set
"threaded =3D True" in the session pool constructor, though, all kinds
of trouble occurs. :-) Does that answer your question sufficiently
now?

On 9/15/05, Forest Wilkinson <mo...@ti...> wrote:
> Thanks for the response, but I'm afraid it doesn't explain the matter.  I=
t
> tells me how Connection objects behave when shared among threads, but it
> does not tell me how SessionPool objects behave when shared among threads=
.
>=20
> One interpretation of what you wrote (and of the documentation) is that
> SessionPool.acquire() returns a thread-safe object when it was constructe=
d
> with threaded=3DTrue, but the SessionPool.acquire() call itself is not
> thread safe.  If that is the case, a program's main thread could safely
> acquire() Connections and then pass them to worker threads, but multiple
> worker threads could not safely call acquire() on their own.
>=20
> My program's worker threads have to acquire() Connections on an as-needed
> basis, rather than being handed a Connection when they start up.  The
> answer to my question will determine whether I have to write my own
> synchronization wrapper for SessionPool.acquire().
>=20
> Anthony Tuininga <ant...@gm...> wrote:
>=20
> > If you specify "threaded =3D3D True" in the constructor of the session
> > pool, Oracle guarantees thread safety provided you use one thread for
> > each connection you use. If you want more than one thread to access a
> > connection, you will have to synchronize access. Does that explain the
> > matter sufficiently?
> >
> > On 9/12/05, Forest Wilkinson <mo...@ti...> wrote:
> >
> >>I'm using cx_Oracle SessionPool objects in a threaded http server.  Sin=
ce
> >>I didn't see an explicit statement in the docs that SessionPool.acquire=
()
> >>and SessionPool.release() are thread safe, I'm currently synchronizing
> >>access to those methods with a threading.Condition object.  Is this
> >>necessary?  If SessionPool does its own method synchronization, I'll ge=
t
> >>rid of my wrapper class and perhaps get better performance.
> >>=3D20
> >>=3D20
>=20
>=20
>=20
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server. Downl=
oad
> it for free - -and be entered to win a 42" plasma tv or your very own
> Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> cx-oracle-users mailing list
> cx-...@li...
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>