Re: [Wrapper-user] wrapper hangs after logfile rotation

[Leif M. <lei...@ta...>]

Arno,
I am not aware of anything that would cause the system itself to hang.

There was a deadlock problem that we discovered and fixed in the 3.5.6
release.  From the release notes:
---
Fix a deadlock in the code that is used to send data to the Java
process. It was only possible if debug level output was enabled and
log file rolling was enabled. Introduced in 3.3.7.
---

As it happens when the rolling took place, this could be the cause.
I see that you also have debug output enabled so all of the
requirements to reproduce this problem are there.

When the Wrapper process got into this state, the JVM would stop
receiving pings from the Wrapper.  It is designed to shut itself down
after 3x the wrapper.ping.timeout has expired.  This might be what you
were seeing.

Could you please try out 3.5.6 and then let me know how it works for you.

Cheers,
Leif

On Thu, Nov 11, 2010 at 9:32 PM,  <wra...@ar...> wrote:
> The wrapper and the wrapped application are running fine so far.
> The Logfile size is set to 64m. 'Sometimes' (is not predictable
> and reproducable) after logfile rotation the system hangs.
> The new logfile is created, but no new logs are written.
>
> The wrapped java process is still running some time, but after
> a while the wrapped web application is not reacting anymore.
>
> Any hints?
>
>
>
>
> MY ENVIRONMENT
>
> o Wrapper version: Java Service Wrapper Community Edition 64-bit 3.5.4
> o Linux 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux
> o Also detected on a SUSE 64bit Enterprise Server
> o wrapped application is a Jira 4.1.1 on apache-tomcat-6.0.28
> o java-6-sun-1.6.0.22
>
>
>
> WRAPPER SETUP
>
>
> #encoding=UTF-8
> # Configuration files must begin with a line specifying the encoding
> #  of the the file.
>
> #********************************************************************
> # Wrapper License Properties (Ignored by Community Edition)
> #********************************************************************
> # Professional and Standard Editions of the Wrapper require a valid
> #  License Key to start.  Licenses can be purchased or a trial license
> #  requested on the following pages:
> # http://wrapper.tanukisoftware.com/purchase
> # http://wrapper.tanukisoftware.com/trial
>
> # Include file problems can be debugged by removing the first '#'
> #  from the following line:
> ##include.debug
>
> # The Wrapper will look for either of the following optional files for a
> #  valid License Key.  License Key properties can optionally be included
> #  directly in this configuration file.
> #include ../conf/wrapper-license.conf
> #include ../conf/wrapper-license-%WRAPPER_HOST_NAME%.conf
>
> # The following property will output information about which License Key(s)
> #  are being found, and can aid in resolving any licensing problems.
> #wrapper.license.debug=TRUE
>
> #********************************************************************
> # Wrapper Localization
> #********************************************************************
> # Specify the locale which the Wrapper should use.  By default the system
> #  locale is used.
> wrapper.lang=en_US # en_US or ja_JP
>
> # Specify the location of the Wrapper's language resources.  If these are
> #  missing, the Wrapper will default to the en_US locale.
> #wrapper.lang.folder=../lang
>
> #********************************************************************
> # Wrapper Java Properties
> #********************************************************************
> # Java Application
> #  Locate the java binary on the system PATH:
> #wrapper.java.command=java
> #  Specify a specific java binary:
>
> set.SI_HOME=/srv/jira4
>
> set.CATALINA_BASE=%SI_HOME%
> set.CATALINA_HOME=%SI_HOME%/tomcat/current
> set.JAVA_HOME=%SI_HOME%/jvm/current
>
> wrapper.java.command=%JAVA_HOME%/bin/java
>
> # Tell the Wrapper to log the full generated Java command line.
> wrapper.java.command.loglevel=INFO
>
> # Java Main class.  This class must implement the WrapperListener interface
> #  or guarantee that the WrapperManager class is initialized.  Helper
> #  classes are provided to do this for you.  See the Integration section
> #  of the documentation for details.
> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp
>
> # Java Classpath (include wrapper.jar)  Add class path elements as
> #  needed starting from 1
> wrapper.java.classpath.1=%CATALINA_BASE%/lib/wrapper.jar
> wrapper.java.classpath.2=%CATALINA_HOME%/bin/bootstrap.jar
>
> # Java Library Path (location of Wrapper.DLL or libwrapper.so)
> wrapper.java.library.path.1=%CATALINA_BASE%/lib
>
> # Java Bits.  On applicable platforms, tells the JVM to run in 32 or 64-bit mode.
> wrapper.java.additional.auto_bits=TRUE
>
> # Java Additional Parameters
> wrapper.java.additional.1=-Dcom.sun.management.jmxremote
> wrapper.java.additional.2=-Dcom.sun.management.jmxremote.ssl=false
> wrapper.java.additional.3=-Dcom.sun.management.jmxremote.authenticate=true
> wrapper.java.additional.4=-Dcom.sun.management.jmxremote.port=98079
> wrapper.java.additional.5=-Dcom.sun.management.jmxremote.password.file=%CATALINA_BASE%/conf/jmxremote.password
> wrapper.java.additional.6=-Dcom.sun.management.jmxremote.access.file=%CATALINA_BASE%/conf/jmxremote.access
> wrapper.java.additional.7=-DVOID
> wrapper.java.additional.8=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> wrapper.java.additional.9=-Djava.util.logging.config.file=%CATALINA_BASE%/conf/logging.properties
> wrapper.java.additional.10=-Djava.endorsed.dirs=%CATALINA_HOME%/common/endorsed
> wrapper.java.additional.11=-Dcatalina.base=%CATALINA_BASE%
> wrapper.java.additional.12=-Dcatalina.home=%CATALINA_HOME%
> wrapper.java.additional.13=-Djava.io.tmpdir=%CATALINA_BASE%/temp
> wrapper.java.additional.14=-Djava.awt.headless=true
> wrapper.java.additional.15=-Dmail.mime.decodeparameters=true
> wrapper.java.additional.16=-Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true
> wrapper.java.additional.17=-Dfile.encoding=UTF-8
> wrapper.java.additional.18=-Dclient.encoding.override=UTF-8
> wrapper.java.additional.19=-verbose:gc
> wrapper.java.additional.20=-XX:+PrintGCDetails
> wrapper.java.additional.21=-XX:+PrintGCTimeStamps
> wrapper.java.additional.22=-XX:+PrintHeapAtGC
> wrapper.java.additional.23=-XX:+PrintClassHistogram
> wrapper.java.additional.24=-XX:+PrintTenuringDistribution
> wrapper.java.additional.25=-XX:+HeapDumpOnOutOfMemoryError
> wrapper.java.additional.26=-Xloggc:%CATALINA_BASE%/logs/gclog.log
> wrapper.java.additional.27=-XX:PermSize=192m
> wrapper.java.additional.28=-XX:MaxPermSize=192m
> wrapper.java.additional.29=-XX:HeapDumpPath=%CATALINA_BASE%/logs/
> #wrapper.java.additional.30=-DGROOVYSCRIPTS_HOME=%CATALINA_BASE%/apps/jira/current/WEB-INF/groovy
> wrapper.java.additional.30=-DGROOVYSCRIPTS_HOME=/srv/svn/jira-deliver/modules/scripts/runtime/src/main/groovy
> wrapper.java.additional.31=-DGROOVYSCRIPTS_LOG_LEVEL=INFO
>
> # Initial Java Heap Size (in MB)
> wrapper.java.initmemory=512
>
> # Maximum Java Heap Size (in MB)
> wrapper.java.maxmemory=1024
>
> # Application parameters.  Add parameters as needed starting from 1
> wrapper.app.parameter.1=org.apache.catalina.startup.Bootstrap
> wrapper.app.parameter.2=start
>
> #********************************************************************
> # Wrapper Logging Properties
> #********************************************************************
> # Enables Debug output from the Wrapper.
> wrapper.debug=TRUE
>
> # Format of output for the console.  (See docs for formats)
> wrapper.console.format=PM
>
> # Log Level for console output.  (See docs for log levels)
> wrapper.console.loglevel=INFO
>
> # Log file to use for wrapper output logging.
> wrapper.logfile=%CATALINA_BASE%/logs/wrapper.log
>
> # Format of output for the log file.  (See docs for formats)
> wrapper.logfile.format=LPTM
>
> # Log Level for log file output.  (See docs for log levels)
> wrapper.logfile.loglevel=INFO
>
> # Maximum size that the log file will be allowed to grow to before
> #  the log is rolled. Size is specified in bytes.  The default value
> #  of 0, disables log rolling.  May abbreviate with the 'k' (kb) or
> #  'm' (mb) suffix.  For example: 10m = 10 megabytes.
> wrapper.logfile.maxsize=64m
>
> # Maximum number of rolled log files which will be allowed before old
> #  files are deleted.  The default value of 0 implies no limit.
> wrapper.logfile.maxfiles=50
>
> # Log Level for sys/event log output.  (See docs for log levels)
> wrapper.syslog.loglevel=NONE
>
> #********************************************************************
> # Wrapper General Properties
> #********************************************************************
> # Allow for the use of non-contiguous numbered properties
> wrapper.ignore_sequence_gaps=TRUE
>
> # Title to use when running as a console
> wra...@ap...@
>
> #********************************************************************
> # Wrapper JVM Checks
> #********************************************************************
> # Detect DeadLocked Threads in the JVM. (Requires Standard Edition)
> wrapper.check.deadlock=TRUE
> wrapper.check.deadlock.interval=60
> wrapper.check.deadlock.action=RESTART
> wrapper.check.deadlock.output=FULL
>
> # Out Of Memory detection.
> wrapper.filter.trigger.1000=java.lang.OutOfMemoryError
> wrapper.filter.action.1000=RESTART
> wrapper.filter.message.1000=The JVM has run out of memory.
>
> #********************************************************************
> # Wrapper Email Notifications. (Requires Professional Edition)
> #********************************************************************
> # Common Event Email settings.
> #wrapper.event.default.email.debug=TRUE
> #wrapper.event.default.email.smtp.host=<SMTP_Host>
> #wrapper.event.default.email.smtp.port=25
> #wrapper.event.default.email.subject=[%WRAPPER_HOSTNAME%:%WRAPPER_NAME%:%WRAPPER_EVENT_NAME%] Event Notification
> #wrapper.event.default.email.sender=<Sender email>
> #wrapper.event.default.email.recipient=<Recipient email>
>
> # Configure the log attached to event emails.
> #wrapper.event.default.email.attach_log=TRUE
> #wrapper.event.default.email.maillog.lines=50
> #wrapper.event.default.email.maillog.format=LPTM
> #wrapper.event.default.email.maillog.loglevel=INFO
>
> # Enable specific event emails.
> #wrapper.event.wrapper_start.email=TRUE
> #wrapper.event.jvm_prelaunch.email=TRUE
> #wrapper.event.jvm_start.email=TRUE
> #wrapper.event.jvm_started.email=TRUE
> #wrapper.event.jvm_deadlock.email=TRUE
> #wrapper.event.jvm_stop.email=TRUE
> #wrapper.event.jvm_stopped.email=TRUE
> #wrapper.event.jvm_restart.email=TRUE
> #wrapper.event.jvm_failed_invocation.email=TRUE
> #wrapper.event.jvm_max_failed_invocations.email=TRUE
> #wrapper.event.jvm_kill.email=TRUE
> #wrapper.event.jvm_killed.email=TRUE
> #wrapper.event.jvm_unexpected_exit.email=TRUE
> #wrapper.event.wrapper_stop.email=TRUE
>
> # Specify custom mail content
> wrapper.event.jvm_restart.email.body=The JVM was restarted.\n\nPlease check on its status.\n
>
> #********************************************************************
> # Wrapper Windows NT/2000/XP Service Properties
> #********************************************************************
> # WARNING - Do not modify any of these properties when an application
> #  using this configuration file has been installed as a service.
> #  Please uninstall the service before modifying this section.  The
> #  service can then be reinstalled.
>
> # Name of the service
> wra...@ap...@
>
> # Display name of the service
> wra...@ap...@
>
> # Description of the service
> wra...@ap...@
>
> # Service dependencies.  Add dependencies as needed starting from 1
> wrapper.ntservice.dependency.1=
>
> # Mode in which the service is installed.  AUTO_START, DELAY_START or DEMAND_START
> wrapper.ntservice.starttype=AUTO_START
>
> # Allow the service to interact with the desktop.
> wrapper.ntservice.interactive=false