[Wrapper-user] JVM appears hung: Timed out waiting for signal from JVM

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

<div>Hello, The purpose of this post is to determine the most appropriate course of action given our current situation. Background: The application is running the 3.3.0 x86 release of the Java Service Wrapper and using the WrapperSimpleApp main class at startup.&nbsp; The supporting hardware has 8 CPUs running 4 instances of the application.&nbsp; The processing for this service is quite CPU and memory intensive.&nbsp; It is not uncommon for the CPU utilization on the box to reach 100% for 3-5 minutes at a time, and if four large files arrived in quick succession, the CPU utilization could potentially be pegged for as long as 15 minutes. Symptom: The Java Service Wrapper is reporting that the JVM appears to be hung and that the timeout expired while waiting on a signal from the JVM.&nbsp; The Java Service Wrapper then shuts down the JVM even though we believe the application is processing normally.&nbsp; Note:&nbsp; We have explicitly disabled the automatic restart functionality as we would like our existing monitoring software to detect and report the problem with the instance of the application.&nbsp; The following snippet is from the wrapper.log file: ERROR&nbsp; | wrapper&nbsp; | 2008/07/21 16:15:53 | JVM appears hung: Timed out waiting for signal from JVM. STATUS | wrapper&nbsp; | 2008/07/21 16:15:54 | JVM Restarts disabled.&nbsp; Shutting down. STATUS | wrapper&nbsp; | 2008/07/21 16:15:54 | &lt;-- Wrapper Stopped The wrapper configuration is as follows: # ----------------------------------------------------------------- # This file contains the configuration for the Java Service Wrapper # ----------------------------------------------------------------- # Set the name of the service. wrapper.name=VPC Gen2 Capture Service 1 # Set the location of the wrapper log wrapper.logfile=../logs/wrapper.1.log # Set the java command line options. wrapper.java.additional.1=-DFILE_PREFIX=//localhost/data wrapper.java.additional.2=-Dlog4j.configuration=log4j.1.xml wrapper.java.additional.3=-DCAPTURE_SERVICES_LOG_DIR=D:\vpc\capture-services\ wrapper.java.additional.4=-Dcom.sun.management.jmxremote.authenticate=false wrapper.java.additional.5=-Dcom.sun.management.jmxremote.port=1151 wrapper.java.additional.6=-Dcom.sun.management.jmxremote.ssl=false wrapper.java.additional.7=-Xmx1150m wrapper.java.additional.8=-Xms512m wrapper.java.additional.9=-XX:MaxPermSize=192m wrapper.java.additional.10=-server # Set the application classpath. # The application classpath must contain wrapper.jar! wrapper.java.classpath.1=../lib/wrapper.jar wrapper.java.classpath.2=../../lib/vpc-framework-1.1.3.jar wrapper.java.classpath.3=../../lib/capture-services-1.1.3.jar # ----------------------------------------------------------------- # The following properties should not need to be modified. # ----------------------------------------------------------------- # Set the application classname and optionally any # command-line parameters that must be passed. wrapper.app.parameter.1=com.jpmc.vpc.core.StartApplication # Set the Java Library Path. wrapper.java.library.path.1=../lib # Set the Java command. wrapper.java.command=%JAVA_HOME%/bin/java # Set the Java Service Wrapper main class. wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp # The following property instructs the Java Service Wrapper # to append the system PATH to the java.library.path at # runtime.&nbsp; This is used in order to load secondary DLLs. wrapper.java.library.path.append_system_path=true # Set the debug flag. wrapper.debug=false # Allow 5 minutes between pings wrapper.ping.interval=300 # Do NOT try to restart the JVM wrapper.disable_restarts=TRUE # Number of seconds to wait for the ping to timeout wrapper.ping.timeout=900 Our current thought is that we could set the value of the wrapper.ping.timeout property to be zero.&nbsp; This should prevent the Java Service Wrapper from attempting to ping the WrapperSimpleApp and ultimately prevent the false detection of a hung JVM which was causing the JVM to be shut down in the middle of normal processing.&nbsp; Does that sound about accurate? Has anyone else out there experienced this or a similar problem?&nbsp; If so, what was your solution?&nbsp; Is there an alternate property that I should be focused on instead of the wrapper.ping.timeout property?&nbsp; I am aware of the wrapper.cpu.timeout property, but I have not seen any messages in the log to indicate that the wrapper has not received any CPU time for N seconds. I would appreciate any input that you may have to offer. <div> Patrick M. Farrell, PMP, CSM Consultant Mentor Quick Solutions, Inc. A four time INC 500 winner! <a href="mailto:pfa...@qu...">pfa...@qu...</a> </div></div>