Re: [Wrapper-user] JVM hang - Xms / Xmx

[Leif M. <le...@ta...>]

Paul,
    Here is Java memory as I understand it. :-]

    As I understand it, the only reason to set the init memory value to 
anything other than
the default is if you know that your application will always need a 
certain amount of
memory on startup.  This will cause the JVM to allocate all of that 
memory in one large
chunk rather than bit by bit as the application loads and initializes 
all of its data.  This
can improve performance a bit.

    The JVM starts out with it allocated memory size set to the init 
memory size.   If at
any point, the JVM needs more memory than it has allocated then it first 
attempts to
do a GC.   If that fails then it will check to see if the allocated size 
is less than the max
size.  If it is then another block of memory is allocated, thus 
increasing the allocated
memory.  If the max size is hit then you will get an out of memory 
exception.

    This amount of real memory on the system does not cause OutOfMemory 
exceptions
unless the max size set when launching the JVM is larger than the total 
Virtual memory
on the machine. If you start using swap memory however then performance 
will be
severely degraded (not an issue with you)

    Each time a GC is performed, old memory will be freed up from the 
allocated
memory.   As the JVM runs, its free memory will slowly decrease until 
the above is
repeated.

    From what I have seen, once a block of memory is allocated from the 
system it is
never returned.  So the allocated memory will stick at the largest size 
since the JVM
was started.   (The exception appears to be 1.4.2  I have noticed that 
it seems to
give memory back to the system?)

    Now, remember what I said about how the the JVM uses free memory 
until it
is exhausted.  Well if that free memory is very large then the time 
between GCs will
be long.  But the size of each GC will be quite large.  This appears to 
cause a larger
hiccup in JVM performance.   For this reason, I usually try to keep the 
init memory
small so that the free memory block is never any larger than is 
absolutely needed.
This also makes sure that I don't use any more system memory than is 
really needed.


    So what happens if you don't set the initmemory at all.  (Ie, leave 
it at 3MB)
Your application should still work but it may not start up quite as 
fast.  I am
wondering how much memory the application takes on startup.  Then what
happens to the memory over time.   It you have set the init memory to 1400
but it is still growing to 1600, this means that the application is 
really requiring
that much memory.   This could be because of a leak or because you actually
need more than 1600MB.  What happens if you set it to 2000MB

    Are you familiar with the memory methods in the Runtime class?   Try 
setting
up a thread in your program which logs the Allocated Memory, Free Memory,
and then via a little math, the in use memory to the console once per 
minute or
so.   This will give you a much more accurate view of what is happening 
memory
wise than you will get from watching the memory usage of Java in the Task
Manager.
    You may also want to call System.gc just before taking the memory
measurements to make sure you are looking at the memory usage without any
garbage in memory.

    I always use the Instrumentation package that I wrote for the Apache 
Avalon
Excalibur project to monitor memory and other data over time.   It lets 
you view
data like the images I attached.  Much easier to understand than log 
output.  That
project is quite stable and well tested but at the moment it is lacking in
documentation.  Something I need to get around to doing.
http://avalon.apache.org/excalibur/index.html

More below.

Paul Casanova wrote:

>Thanks Leif.
>
>We have 6.3GB of RAM in a Win2K Advanced Server box - usually around 3GB
>free, so there's no chance of running out of physical RAM.
>
>We were using:
># Initial Java Heap Size (in MB)
>wrapper.java.initmemory=1600
>
># Maximum Java Heap Size (in MB)
>wrapper.java.maxmemory=1600
>
>
>Now using:
># Initial Java Heap Size (in MB)
>wrapper.java.initmemory=1400
>
># Maximum Java Heap Size (in MB)
>wrapper.java.maxmemory=1600
>
>When first trying the change, I tried initmemory=1200, then 1300, now 1400
>just to make sure it was functioning ok.  In the Windows Task Manager the
>Java process always peaked closed to each of these figures - never exceeded
>1600MB as it did previously (used to get up to 1780MB: 1600MB + JVM
>overhead I assume).
>
>I've seen this hotspot error once before since switching to JSW (months
>back) - before hand it probably would have just been lost (svrany - MS
>product) - yay for JSW I say.
>
>Once again I don't seem to be able to find anything concrete about this
>error in Sun's Java forums, other than suggestions that native font dll's
>may have become corrupt - but then one would expect the error to occur
>regularly.  It happened twice in one day last week, and then today, with
>varying durations between the java.lang.OutOfMemoryError's and the JVM
>crash.  They've both been there each time though.
>  
>
I am wondering if 1600MB is too small or if your application is leaking 
memory?   It
would not be fun profiling an app that large. :-/

>We had always previously set the min and max heap values to the same.  I
>couldn't find much Sun info on how Xms and Xmx alter the behaviour of
>garbage collection (or any other behaviour for that matter) - only in
>http://java.sun.com/docs/hotspot/gc/.  I just gave it a whirl and it seems
>to have made a difference to the JVM hanging.
>  
>
What changes did you make?

>I also read in one of the forums that issues with native calls from the JVM
>were rectified in 1.4.1_02 +.  Have you also heard this?
>  
>
I had not heard of any problems with earlier versions.

Cheers,
Leif