#19 Bad memory leak

closed
nobody
None
5
2006-02-13
2006-01-26
Anonymous
No

We were using c3p0 v0.8.5.2 with Oracle Thick driver in
a project using Oracle database. We were noticing
growth in the Java VM size, which eventually used to
exceed 2GB limit imposed on processes running in a 32
bit OS and crash soon thereafter. After several months
of investigation, we finally tracked the growth in
memory down to the fore-mentioned version of the c3p0.
The problem disappeared after downgrading to v0.8.5
downloaded from this web-site. So looks this version
had a bad memory leak. We are also noticing the leak
and the growth in size also in later versions
including v0.9.x. I think this should be looked up
carefully as other users may face the same problem.

Are c3p0 parameters were:
initialPoolSize=10
maxPoolSize=40^M
acquireRetryAttempts=60
maxIdleTime=0
maxStatements=180
user=nai_user
idleConnectionTestPeriod=300
password=nai_user
minPoolSize=10

Thanks
Manoj Ahluwalia
manoj_ahluwalia@mcafee.com

Discussion

  • Steve Waldman
    Steve Waldman
    2006-01-26

    Logged In: YES
    user_id=175530

    Hi.

    1) If you could gather some heap information, that would be helpful. I can't
    reproduce this leak (though I'm not using Oracle thick).

    2) Are you seeing OutOfMemoryErrors, or a VM crash? If the latter, that's a VM
    and/or JNI bug. Are you sure garbage collection is happening as it ought to?
    Do you have the maximum heap size set to below 2GB, to force garbage
    collection before the process runs agains OS limits? Please do try that if not.

    I'm suspecting something relating to the native part of the Oracle thick driver.
    But, if some versions of c3p0 tweak the problem and others don't, I'd like to
    understand why.

    Thanks, and good luck!

     
  • Logged In: NO

    Thanks for your reply.

    1. There is this nice "perfmon" tool in Windows that
    monitors the size (total address space), working set
    (portion of the address space in main memory) etc. I'm
    attaching the two logs... one taken with v0.8.5 and the
    other with v0.8.5.2. Trust me there is no other difference
    in the application exception c3p0*.jar file. Note the
    variation in the fourth column
    (\\BIG-MOMMA\Process(java#1)\Virtual Bytes Peak). It is
    static with v0.8.5 and keeps on growing with v0.8.5.2. This
    column is total address space of the process which is
    contrained by the 2GB limit (and can be extended to 3GB
    using OS /3GB switch).

    2. The heap is capped at 768 MB (with -mx argument). We are
    not seeing any OutOfMemoryErrors in the application. The VM
    only crashes when "Virtual Bytes" exceed /2GB (or /3GB
    depending on the OS setting).

    We also doubted Oracle's OCI thick driver all these months
    but we are quite positive after our breakthrough this week
    that this is caused by the c3P0... actually most versions.
    Only v0.8.5 does the magic. There was some reference in the
    c3P0 change log with regards to some BAD memory fix related
    to statement caching.

    Actually I just noticed I can't find a way to send files. If
    you send me an email at manoj_ahluwalia@mcafee.com, I can
    mail you the perfmon log files.

    Thanks again for your help.

    Manoj Ahluwalia

     
  • Steve Waldman
    Steve Waldman
    2006-01-27

    Logged In: YES
    user_id=175530

    Manoj,

    Send along what you have, but do note that while c3p0 may be implicated in
    the issue, there is more going on here than c3p0. If your VM is not throwing
    OutOfMemoryErrors when you're exceeding max heap, that's a problem.
    Probably the cause is that the memory being used is held by unmanaged
    Oracle JNI code, which must manage there own memory, and it could well be
    the case that one version of c3p0 ends up cleaning up some JNI resource
    more quickly than some others, leading to resource build-up.

    The mostly likely change in v0.8.5.2 had to do with the Statement cache,
    where PreparedStatements in the cache were being prematurely closed. From
    your perspective, this means PreparedStatement resources were released
    earlier than they are now when Statement is turned on (as it is in your setup).
    If you can, please try a more recent c3p0 with maxStatements (and
    maxStatementsPerConnection) left at 0. If you don't observe the resource
    leak, try an absurdly small number (say 5) for maxStatements, and see if the
    leak reappears.

    Anyway, send me what you have, especially if it includes a description of the
    heap (objects on heap, threads that created them).

     
  • Steve Waldman
    Steve Waldman
    2006-02-13

    • status: open --> closed
     
  • Steve Waldman
    Steve Waldman
    2006-02-13

    Logged In: YES
    user_id=175530

    I think this was resolved via e-mail to the reporter's satisfaction. (?)