[Wrapper-user] JVM appears hung: VMware, verbose garbage collection

[Nathan W. <wr...@de...>]

Hi;

I have an application being deployed onto JBoss on Windows under VMware.
JBoss is running as a service under wrapper.
Although the application has run elsewhere for long periods of time with no
issues, we're seeing a problem under wrapper.
Once the app has been running for 1-2 hours, we see:

ERROR  | wrapper  | 2007/10/26 16:25:17 | JVM appears hung: Timed out
waiting for signal from JVM.
ERROR  | wrapper  | 2007/10/26 16:25:18 | JVM did not exit on request,
terminated

Our app is deployed as an EAR file, and if the JBoss server process doesn't
shut down cleanly it won't come back up without deleting the tmp, work and
data folders.  So once wrapper has gunned the server it's dead until someone
manually intervenes.

Using the default ticks timer we see messages like this periodically:

 The timer fell behind the system clock by 14600ms.

Our server is a VMware instance, so it is possible that the VM could be
getting swapped out or starved.  This would probably appear to an
application as though the clock and timers were jumping out of sync in ways
that normally are impossible.
Is anyone successfully using wrapper under VMware?

Also we're running with verbose garbage collection, which I understand can
hang the server momentarily.  Are there known issues with verbose garbage
collection under wrapper?

Looking through the code it doesn't appear that wrapper tries to shut down
the server once it decides there is a hang; it looks like it goes right into
WRAPPER_JSTATE_KILLING rather than WRAPPER_JSTATE_STOPPING; so that the
shutdownTimeout parameter does not come into play.  Would it be possible to
attempt a clean shutdown before killing the server?


Selected settings below.  Note right now we're trying to address the hard
kill that leaves the server unusable, we will eventually bump up the ping
timeout to try and avoid the kill in the first place.

# Number of seconds between Wrapper ping requests to the JVM.
wrapper.ping.interval=5

#Number of seconds to allow between the wrapper pinging the JVM and the
response.
wrapper.ping.timeout=10

#Number of seconds between JVM reports stopped and the JVM actually
terminates.
wrapper.jvm_exit.timeout=60
wrapper.shutdown.timeout=60
wrapper.restart.delay=60

wrapper.timer_fast_threshold=0
wrapper.timer_slow_threshold=10

# Force reload of wrapper config on JVM restart
wrapper.restart.reload_configuration=TRUE

Thanks for any insight you can bring.

Nathan

-- 
Nathan Wray