Menu
▾
▴
Re: [Wrapper-user] JVM appears hung: Timed out waiting for signalfrom JVM. problem
|
From: Simon C. \(Funmobile\) <sim...@fu...> - 2007-06-28 02:46:04
|
Hi leif, for the log file, it is ended and the process seem stopped and wrapper version is 3.23. I have one strange stuff when start wrapper and I don't know whether it is since reason of wrapper hang. It is when I start wrapper, then it says /root/.bachrc permission deny. the run user I use in wrapper is nobody. do u have any idea why wrapper hang? Regards Simon. ----- Original Message ----- From: "Leif Mortenson" <le...@ta...> To: <wra...@li...> Sent: Thursday, June 28, 2007 9:56 AM Subject: Re: [Wrapper-user] JVM appears hung: Timed out waiting for signalfrom JVM. problem > Simon, > Thanks for the log. I need to confirm as it makes a big difference > how I look into this. Is what you sent the very END of your log file > or does it continue. You had mentioned that the Wrapper process > was hanging and not restarting the JVM. I want to verify, A) that the > Wrapper process is hanging, and B) that the log you posted is indeed > the last output before it does so. > Also, from your configuration file properties, it looks like you are > using version 3.2.3. Is that correct? > > Cheers, > Leif > > > Simon Chan wrote: > > Hi Leif, > > The following is the error after add the value that u mentioned before: > > STATUS | wrapper | 2007/06/27 01:13:45 | Ticks=0008ce46, > > WrapperState=STARTED, JVMState=STARTED JVMStateTimeoutTicks=0008ce55 > > (1s), Exit=false, Restart=false > > STATUS | wrapper | 2007/06/27 01:13:45 | Loop: handle wrapper state: > > STARTED > > STATUS | wrapper | 2007/06/27 01:13:46 | Loop: handle jvm state: STARTED > > STATUS | wrapper | 2007/06/27 01:13:46 | Loop: sleep > > STATUS | wrapper | 2007/06/27 01:13:46 | Loop: maintain logger > > STATUS | wrapper | 2007/06/27 01:13:46 | Loop: process jvm output > > STATUS | wrapper | 2007/06/27 01:13:46 | Loop: process socket > > STATUS | wrapper | 2007/06/27 01:13:47 | Loop: maintain logger(2) > > STATUS | wrapper | 2007/06/27 01:13:47 | Ticks=0008ce52, > > WrapperState=STARTED, JVMState=STARTED JVMStateTimeoutTicks=0008ce55 > > (0s), Exit=false, Restart=false > > STATUS | wrapper | 2007/06/27 01:13:47 | Loop: handle wrapper state: > > STARTED > > STATUS | wrapper | 2007/06/27 01:13:47 | Loop: handle jvm state: STARTED > > *ERROR | wrapper | 2007/06/27 01:13:47 | JVM appears hung: Timed out > > waiting for signal from JVM. > > STATUS | wrapper | 2007/06/27 01:13:47 | Set Java State STARTED -> KILLING > > STATUS | wrapper | 2007/06/27 01:13:47 | Set Java State KILLING > > Timeout 0008ce55 -> 0008ce52 > > STATUS | wrapper | 2007/06/27 01:13:47 | Loop: sleep > > STATUS | wrapper | 2007/06/27 01:13:48 | Loop: maintain logger > > STATUS | wrapper | 2007/06/27 01:13:48 | Loop: process jvm output > > STATUS | wrapper | 2007/06/27 01:13:48 | Loop: process socket > > STATUS | wrapper | 2007/06/27 01:13:48 | Loop: maintain logger(2)* > > ** > > do u have any idea on it? > > For server, our server has 2G RAM and running mod_jk + tomcat. we are > > running 2 x tomcat now. > > Regards > > Simon. > > ----- Original Message ----- > > From: "Leif Mortenson" <le...@ta... > > <mailto:le...@ta...>> > > To: <wra...@li... > > <mailto:wra...@li...>> > > Sent: Tuesday, June 26, 2007 2:09 PM > > Subject: Re: [Wrapper-user] JVM appears hung: Timed out waiting for > > signalfrom JVM. problem > > > > > Simon, > > > Your configuration is forcing the JVM to allocate 768MB of memory > > > on startup? How much memory does your system have available? > > > Are you sure the JVM is able to run without any disk swapping taking > > > place? Java gets VERY slow when its memory is being swapped to disk. > > > We are talking about 100-1000x slower. This can cause the JVM to > > > appear to be frozen for long periods of time. > > > > > > The log looks like the JVM is freezing up for some reason. The > > > Wrapper detects that and then should be attempting a restart. > > > Extending the ping timeout would make the wrapper take longer to > > > decide that the JVM is frozen, but it will not solve the problem if it > > > really is frozen. > > > > > > Is what you posted the end of your wrapper.log file? I would > > > expect more output after that point. If that is the end of your > > > log file, try adding the following property. It greatly increases the > > > output but will show whether or not the main loop of the wrapper > > > is frozen. > > > > > > wrapper.state_output=TRUE > > > > > > If that freezes, then adding the following as well will provide > > > even more information about the precise location of the freeze. > > > > > > wrapper.loop_output=TRUE > > > > > > Cheers, > > > Leif > > > > > > Simon Chan wrote: > > >> Hi leif, > > >> have any news on it? > > >> I tried to change wrapper.ping.timeout to 180 and seem still hang. > > >> is it CPU 100% fully use or no memory? > > >> Regards > > >> Simon. > > >> > > >> ----- Original Message ----- > > >> *From:* Simon Chan (Funmobile) <mailto:sim...@fu...> > > >> *To:* wra...@li... > > <mailto:wra...@li...> > > >> <mailto:wra...@li...> > > >> *Sent:* Sunday, June 24, 2007 9:55 AM > > >> *Subject:* Re: [Wrapper-user] JVM appears hung: Timed out waiting > > >> for signalfrom JVM. problem > > >> > > >> Hi Leif, > > >> the following is the wrapper.conf: > > >> #******************************************************************** > > >> # TestWrapper Properties > > >> # > > >> # NOTE - Please use src/conf/wrapper.conf.in as a template for your > > >> # own application rather than the values used for the > > >> # TestWrapper sample. > > >> #******************************************************************** > > >> # Java Application > > >> wrapper.java.command=/usr/java/jdk1.5.0_07/bin/java > > >> # Java Main class. This class must implement the WrapperListener > > >> interface > > >> # or guarantee that the WrapperManager class is initialized. Helper > > >> # classes are provided to do this for you. See the Integration section > > >> # of the documentation for details. > > >> #wrapper.java.mainclass=org.tanukisoftware.wrapper.test.Main > > >> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperStartStopApp > > >> # Java Classpath (include wrapper.jar) Add class path elements as > > >> # needed starting from 1 > > >> # wrapper.java.classpath.1=../lib/wrappertest.jar > > >> # wrapper.java.classpath.2=../lib/wrapper.jar > > >> > > wrapper.java.classpath.1=/usr/local/wrapper-linux-x86-32-3.2.3/lib/wrapper.j ar > > >> wrapper.java.classpath.2=/usr/apache-tomcat-5.5.17/bin/bootstrap.jar > > >> > > wrapper.java.classpath.3=/usr/apache-tomcat-5.5.17/bin/commons-logging-api.j ar > > >> wrapper.java.classpath.4=/usr/java/jdk1.5.0_07/lib/tools.jar > > >> # Java Library Path (location of Wrapper.DLL or libwrapper.so) > > >> # wrapper.java.library.path.1=../lib > > >> wrapper.java.library.path.1=/usr/local/wrapper-linux-x86-32-3.2.3/lib > > >> # Java Additional Parameters > > >> #wrapper.java.additional.1= > > >> > > wrapper.java.additional.1=-Djava.endorsed.dirs=/usr/apache-tomcat-5.5.17/com mon/endorsed > > >> wrapper.java.additional.2=-Dcatalina.base=/usr/apache-tomcat-5.5.17 > > >> wrapper.java.additional.3=-Dcatalina.home=/usr/apache-tomcat-5.5.17 > > >> > > wrapper.java.additional.4=-Djava.io.tmpdir=/usr/apache-tomcat-5.5.17/temp > > >> wrapper.java.additional.5=-XX:MaxPermSize=64m > > >> # Initial Java Heap Size (in MB) > > >> #wrapper.java.initmemory=3 > > >> wrapper.java.initmemory=768 > > >> # Maximum Java Heap Size (in MB) > > >> #wrapper.java.maxmemory=64 > > >> wrapper.java.maxmemory=768 > > >> # Application parameters. Add parameters as needed starting from 1 > > >> #wrapper.app.parameter.1= > > >> wrapper.app.parameter.1=org.apache.catalina.startup.Bootstrap > > >> wrapper.app.parameter.2=1 > > >> wrapper.app.parameter.3=start > > >> wrapper.app.parameter.4=org.apache.catalina.startup.Bootstrap > > >> wrapper.app.parameter.5=true > > >> wrapper.app.parameter.6=1 > > >> wrapper.app.parameter.7=stop > > >> #******************************************************************** > > >> # Wrapper Logging Properties > > >> #******************************************************************** > > >> # Format of output for the console. (See docs for formats) > > >> wrapper.console.format=PM > > >> # Log Level for console output. (See docs for log levels) > > >> wrapper.console.loglevel=INFO > > >> # Log file to use for wrapper output logging. > > >> wrapper.logfile=/var/log/tomcat/tomcat-1.log > > >> # Format of output for the log file. (See docs for formats) > > >> wrapper.logfile.format=LPTM > > >> # Log Level for log file output. (See docs for log levels) > > >> wrapper.logfile.loglevel=DEBUG > > >> # Maximum size that the log file will be allowed to grow to before > > >> # the log is rolled. Size is specified in bytes. The default value > > >> # of 0, disables log rolling. May abbreviate with the 'k' (kb) or > > >> # 'm' (mb) suffix. For example: 10m = 10 megabytes. > > >> wrapper.logfile.maxsize=0 > > >> # Maximum number of rolled log files which will be allowed before old > > >> # files are deleted. The default value of 0 implies no limit. > > >> wrapper.logfile.maxfiles=0 > > >> # Log Level for sys/event log output. (See docs for log levels) > > >> wrapper.syslog.loglevel=DEBUG > > >> wrapper.ping.timeout=60 > > >> wrapper.statusfile=/var/run/tomcat/wrapper_1.status > > >> wrapper.filter.trigger.1=java.lang.OutOfMemoryError > > >> wrapper.filter.action.1=RESTART > > >> and we use it for monitor tomcat 5.5.17. and the following is the > > >> debug log: > > >> DEBUG | wrapperp | 2007/06/23 16:47:17 | read a packet PING : ok > > >> DEBUG | wrapper | 2007/06/23 16:47:17 | Got ping response from JVM > > >> DEBUG | wrapperp | 2007/06/23 16:47:22 | send a packet PING : ping > > >> INFO | jvm 2 | 2007/06/23 16:47:22 | Received a packet PING : ping > > >> INFO | jvm 2 | 2007/06/23 16:47:22 | Send a packet PING : ok > > >> DEBUG | wrapperp | 2007/06/23 16:47:22 | read a packet PING : ok > > >> DEBUG | wrapper | 2007/06/23 16:47:22 | Got ping response from JVM > > >> DEBUG | wrapperp | 2007/06/23 16:47:26 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:31 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:35 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:40 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:44 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:49 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:53 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:47:58 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:02 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:07 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:11 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:16 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:20 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:25 | send a packet PING : ping > > >> DEBUG | wrapperp | 2007/06/23 16:48:29 | send a packet PING : ping > > >> *ERROR | wrapper | 2007/06/23 16:48:32 | JVM appears hung: Timed > > >> out waiting for signal from JVM.* > > >> ** > > >> wrapper seeems hang after this and cannot try to restart JVM. it > > >> is so strange? > > >> I need to try to increase the value of ping timeout? > > >> Regards > > >> Simon. > > >> ----- Original Message ----- > > >> From: "Leif Mortenson" <le...@ta... > > <mailto:le...@ta...> > > >> <mailto:le...@ta...>> > > >> To: <wra...@li... > > <mailto:wra...@li...> > > >> <mailto:wra...@li...>> > > >> Sent: Sunday, June 24, 2007 8:45 AM > > >> Subject: Re: [Wrapper-user] JVM appears hung: Timed out waiting > > >> for signal from JVM. problem > > >> > > >> > Simon, > > >> > Can you post your wrapper.conf file? Also if you are able to > > >> > reproduce this, can you set the wrapper.debug=true property > > >> > and post the resulting wrapper.conf file as well? > > >> > > > >> > >From what you posted, it looks like the Wrapper is detecting > > >> > that the JVM is hung, at which point it should be killing and > > >> > restarting the JVM process. I am not clear exactly what you > > >> > are see at that point however. > > >> > > > >> > Cheers, > > >> > Leif > > >> > > > >> > Simon Chan (Funmobile) wrote: > > >> > > hi all, > > >> > > I am using wrapper to monitor my tomcat 5.5.17 (use java 1.5) now. > > >> > > but I see that it is hung and show the following error: > > >> > > INFO | jvm 1 | 2007/06/22 18:34:25 | Send a packet PING : ok > > >> > > ERROR | wrapper | 2007/06/22 18:35:36 | JVM appears hung: > > >> Timed out > > >> > > waiting for signal from JVM. > > >> > > wrapper seems don't try to restart it and just leave it and make > > >> > > tomcat die. > > >> > > do u have any idea? > > >> > > Regards > > >> > > Simon. > > >> > > > > >> > > ------------------------------------------------------------------------ > > >> > > > > >> > > > > >> > > ------------------------------------------------------------------------- > > >> > > This SF.net email is sponsored by DB2 Express > > >> > > Download DB2 Express C - the FREE version of DB2 express and take > > >> > > control of your XML. No limits. Just data. Click to get it now. > > >> > > http://sourceforge.net/powerbar/db2/ > > >> > > > > >> > > ------------------------------------------------------------------------ > > >> > > > > >> > > _______________________________________________ > > >> > > Wrapper-user mailing list > > >> > > Wra...@li... > > <mailto:Wra...@li...> > > >> <mailto:Wra...@li...> > > >> > > https://lists.sourceforge.net/lists/listinfo/wrapper-user > > >> > > > > >> > > > >> > > > >> > > > >> > > ------------------------------------------------------------------------ > > >> > > > >> > > ------------------------------------------------------------------------- > > >> > This SF.net email is sponsored by DB2 Express > > >> > Download DB2 Express C - the FREE version of DB2 express and take > > >> > control of your XML. No limits. Just data. Click to get it now. > > >> > http://sourceforge.net/powerbar/db2/ > > >> > > ------------------------------------------------------------------------ > > >> > _______________________________________________ > > >> > Wrapper-user mailing list > > >> > Wra...@li... > > <mailto:Wra...@li...> > > >> <mailto:Wra...@li...> > > >> > https://lists.sourceforge.net/lists/listinfo/wrapper-user > > >> > > > >> > > ------------------------------------------------------------------------ > > >> No virus found in this incoming message. > > >> Checked by AVG Free Edition. > > >> Version: 7.5.472 / Virus Database: 269.9.6/862 - Release Date: > > >> 6/22/2007 3:04 PM > > >> > > >> > > ------------------------------------------------------------------------ > > >> > > ------------------------------------------------------------------------- > > >> This SF.net email is sponsored by DB2 Express > > >> Download DB2 Express C - the FREE version of DB2 express and take > > >> control of your XML. No limits. Just data. Click to get it now. > > >> http://sourceforge.net/powerbar/db2/ > > >> > > >> > > ------------------------------------------------------------------------ > > >> _______________________________________________ > > >> Wrapper-user mailing list > > >> Wra...@li... > > <mailto:Wra...@li...> > > >> https://lists.sourceforge.net/lists/listinfo/wrapper-user > > >> > > >> > > ------------------------------------------------------------------------ > > >> > > >> Anti-virus and anti-spam by FortiGate > > >> > > >> > > ------------------------------------------------------------------------ > > >> > > >> > > ------------------------------------------------------------------------- > > >> This SF.net email is sponsored by DB2 Express > > >> Download DB2 Express C - the FREE version of DB2 express and take > > >> control of your XML. No limits. Just data. Click to get it now. > > >> http://sourceforge.net/powerbar/db2/ > > >> > > ------------------------------------------------------------------------ > > >> > > >> _______________________________________________ > > >> Wrapper-user mailing list > > >> Wra...@li... > > <mailto:Wra...@li...> > > >> https://lists.sourceforge.net/lists/listinfo/wrapper-user > > >> > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by DB2 Express > > > Download DB2 Express C - the FREE version of DB2 express and take > > > control of your XML. No limits. Just data. Click to get it now. > > > http://sourceforge.net/powerbar/db2/ > > ------------------------------------------------------------------------ > > > _______________________________________________ > > > Wrapper-user mailing list > > > Wra...@li... > > <mailto:Wra...@li...> > > > https://lists.sourceforge.net/lists/listinfo/wrapper-user > > > > > ------------------------------------------------------------------------ > > > > Anti-virus and anti-spam by FortiGate > > > > ------------------------------------------------------------------------ > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Wrapper-user mailing list > > Wra...@li... > > https://lists.sourceforge.net/lists/listinfo/wrapper-user > > > > > ---------------------------------------------------------------------------- ---- > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ ---------------------------------------------------------------------------- ---- > _______________________________________________ > Wrapper-user mailing list > Wra...@li... > https://lists.sourceforge.net/lists/listinfo/wrapper-user > ---------------------------------------------------------------------------- ---- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.472 / Virus Database: 269.9.9/870 - Release Date: 6/26/2007 10:07 AM |
