Re: [Wrapper-user] Service shutdown problem

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Mark,
Sorry for the trouble on this.   Extending the wrapper.ping.timeout to
a value longer than the JVM is going without CPU should do the trick.
If it is still being terminated after 30 minutes (1800 sec) then most
likely the JVM is being 100% starved of CPU for even longer.

What is the memory situation on this system?  Normally processes to do
not take 100% of CPU quite so completely.  The exception is when there
is massive amounts of disk swapping going on.  Java gets VERY VERY
slow when this happens.  On the order of 1/1000th speed in many cases.

It is strange that the Wrapper is not restarting your JVM after it has
been terminated however:
---
ERROR  | wrapper  | 2010/04/25 17:18:32 | JVM did not exit on request,
terminated
STATUS | wrapper  | 2010/04/25 17:18:37 | Launching a JVM...
INFO   | jvm 2    | 2010/04/25 17:18:37 | WrapperManager: Initializing...
STATUS | wrapper  | 2010/04/25 17:18:40 | <-- Wrapper Stopped
---

Would it be possible for you to set the wrapper.debug=true property
and let this happen once more so I can take a look at the detailed
wrapper.log and see what is happening?   Feel free to send that to me
directly as it would be be a bit large for the list.

If the system is very heavy still after being restarted, it may be
necessary to also set the startup timeout so that the JVM has time to
be launched.
wrapper.startup.timeout=600

Cheers,
Leif

On Tue, Apr 27, 2010 at 4:34 AM, Ludwig, Mark <mar...@us...> wrote:
> Recently I started having trouble with wrapper services going down in a
> couple of scenarios. Here’s the first scenario.
>
> I have a machine (a partition or virtual server actually) with about a dozen
> Eclipse infocenters set up with wrappers so that they can be stopped,
> started, or left running as Windows services. A couple of months ago, after
> many months of pretty much error-free operation, several of the services
> would go down. I would have to end the javaw.exe process for the service
> that went down and then I could restart the service.
>
> I eventually figured out that the virus scan update program was hogging the
> CPU for up to 20 or 30 minutes each day. The IT folks tweaked the scheduling
> and config of the virus scan update some and then I added in the following
> parameters for each service. This has greatly reduced the number of times
> the services go down, but it still happens (one service goes down every 2 or
> 3 days instead of 3 or 4 services going down every day):
>
> wrapper.ping.timeout=1800
>
> wrapper.ping.interval=10
>
> I also tried changing the logging levels to get more detail. Unfortunately,
> I could never get the services to crash when the logging was at TRACE and I
> didn’t want to worry about the log files getting bigger and bigger.
> Occasionally I would see a reference to Service Control Code 4.
>
> Log files have things like the following when the service goes down:
>
> ERROR  | wrapper  | 2010/04/25 17:18:31 | JVM appears hung: Timed out
> waiting for signal from JVM.
>
> ERROR  | wrapper  | 2010/04/25 17:18:32 | JVM did not exit on request,
> terminated
>
> STATUS | wrapper  | 2010/04/25 17:18:37 | Launching a JVM...
>
> INFO   | jvm 2    | 2010/04/25 17:18:37 | WrapperManager: Initializing...
>
> STATUS | wrapper  | 2010/04/25 17:18:40 | <-- Wrapper Stopped
>
>
>
> So my questions at this point are the following:
>
> What would be the best possibilities for configuration parameters in an
> occasional CPU starvation scenario?
>
> What would be the best parameters to set to get troubleshooting info?
>
> Would it help to upgrade to newer wrapper (I’m on 3.3.1)?
>
>
>
> Other basic info:
>
> Windows 2003 server
>
> One processor
>
> 2 GB RAM
>
> Server is set up with a 10GB C: drive (2 GB free) and a 30GB E: drive (10 GB
> free). All the files for the infocenters are on the E: drive.
>
>
>
> Sample config file for one of the wrapper services:
>
> Recently I started having trouble with wrapper services going down in a
> couple of scenarios. Here’s the first scenario.
>
> I have a machine (a partition or virtual server actually) with about a dozen
> Eclipse infocenters set up with wrappers so that they can be stopped,
> started, or left running as Windows services. A couple of months ago, after
> many months of pretty much error-free operation, several of the services
> would go down. I would have to end the javaw.exe process for the service
> that went down and then I could restart the service.
>
> I eventually figured out that the virus update program was hogging the CPU
> for up to 20 or 30 minutes each day. The IT folks tweaked the scheduling of
> the update some and then I added in the following parameters for each
> service, which has greatly reduced the number of times the services go down,
> but it still happens (one service goes down every 2 or 3 days instead of 3
> or 4 services going down every day):
>
> wrapper.ping.timeout=1800
>
> wrapper.ping.interval=10
>
> I also tried changing the logging levels to get more detail. Unfortunately,
> I could never get the services to crash when the logging was at TRACE and I
> didn’t want to worry about the log files getting bigger and bigger.
> Occasionally I would see a reference to Service Control Code 4.
>
> Log files have things like the following when the service goes down:
>
> ERROR  | wrapper  | 2010/04/25 17:18:31 | JVM appears hung: Timed out
> waiting for signal from JVM.
>
> ERROR  | wrapper  | 2010/04/25 17:18:32 | JVM did not exit on request,
> terminated
>
> STATUS | wrapper  | 2010/04/25 17:18:37 | Launching a JVM...
>
> INFO   | jvm 2    | 2010/04/25 17:18:37 | WrapperManager: Initializing...
>
> STATUS | wrapper  | 2010/04/25 17:18:40 | <-- Wrapper Stopped
>
>
>
> So my questions at this point are the following:
>
> What would be the best possibilities for configuration parameters in an
> occasional CPU starvation scenario?
>
> What would be the best parameters to set to get troubleshooting info?
>
> Would it help to upgrade to newer wrapper (I’m on 3.3.1)?
>
>
>
> Other basic info:
>
> Windows 2003 server
>
> One processor
>
> 2 GB RAM
>
> Server is set up with a 10GB C: drive (2 GB free) and a 30GB E: drive (10 GB
> free). All the files for the infocenters are on the E: drive.
>
>
>
> Sample config file for one of the wrapper services:
>
> Recently I started having trouble with wrapper services going down in a
> couple of scenarios. Here’s the first scenario.
>
> I have a machine (a partition or virtual server actually) with about a dozen
> Eclipse infocenters set up with wrappers so that they can be stopped,
> started, or left running as Windows services. A couple of months ago, after
> many months of pretty much error-free operation, several of the services
> would go down. I would have to end the javaw.exe process for the service
> that went down and then I could restart the service.
>
> I eventually figured out that the virus update program was hogging the CPU
> for up to 20 or 30 minutes each day. The IT folks tweaked the scheduling of
> the update some and then I added in the following parameters for each
> service, which has greatly reduced the number of times the services go down,
> but it still happens (one service goes down every 2 or 3 days instead of 3
> or 4 services going down every day):
>
> wrapper.ping.timeout=1800
>
> wrapper.ping.interval=10
>
> I also tried changing the logging levels to get more detail. Unfortunately,
> I could never get the services to crash when the logging was at TRACE and I
> didn’t want to worry about the log files getting bigger and bigger.
> Occasionally I would see a reference to Service Control Code 4.
>
> Log files have things like the following when the service goes down:
>
> ERROR  | wrapper  | 2010/04/25 17:18:31 | JVM appears hung: Timed out
> waiting for signal from JVM.
>
> ERROR  | wrapper  | 2010/04/25 17:18:32 | JVM did not exit on request,
> terminated
>
> STATUS | wrapper  | 2010/04/25 17:18:37 | Launching a JVM...
>
> INFO   | jvm 2    | 2010/04/25 17:18:37 | WrapperManager: Initializing...
>
> STATUS | wrapper  | 2010/04/25 17:18:40 | <-- Wrapper Stopped
>
>
>
> So my questions at this point are the following:
>
> What would be the best possibilities for configuration parameters in an
> occasional CPU starvation scenario?
>
> What would be the best parameters to set to get troubleshooting info?
>
> Would it help to upgrade to newer wrapper (I’m on 3.3.1)?
>
>
>
> Other basic info:
>
> Windows 2003 server
>
> One processor
>
> 2 GB RAM
>
> Server is set up with a 10GB C: drive (2 GB free) and a 30GB E: drive (10 GB
> free). All the files for the infocenters are on the E: drive.
>
> Sample config file for one of the wrapper services:
>
> wrapper.java.command=java
>
> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp
>
> wrapper.java.classpath.1=../lib/wrapper.jar
>
> wrapper.java.classpath.2=E:/eclipse/plugins/org.eclipse.help.base_*.jar
>
> wrapper.java.library.path.1=../lib
>
> wrapper.port=32000
>
> wrapper.app.parameter.1=org.eclipse.help.standalone.Infocenter
>
> wrapper.app.parameter.2=-command
>
> wrapper.app.parameter.3=start
>
> wrapper.app.parameter.4=-eclipsehome
>
> wrapper.app.parameter.5=E:/eclipse
>
> wrapper.app.parameter.6=-port
>
> wrapper.app.parameter.7=7121
>
> wrapper.app.parameter.8=-noexec
>
> wrapper.app.parameter.9=-data
>
> wrapper.app.parameter.10=E:/infocenter/7120_7129_LmrkLibrary/7121_LmrkWinSQL/infocenter
>
> wrapper.app.parameter.11=-product
>
> wrapper.app.parameter.12=com.lawson.help.brand.branding
>
> wrapper.app.parameter.13=-configuration
>
> wrapper.app.parameter.14=E:/infocenter/7120_7129_LmrkLibrary/7121_LmrkWinSQL/configuration
>
>
>
> #********************************************************************
>
> # Wrapper Logging Properties
>
> #********************************************************************
>
> wrapper.console.format=PM
>
> wrapper.console.loglevel=INFO
>
>
>
> wrapper.logfile=../logs/LmrkWinSQL.log
>
> wrapper.logfile.format=LPTM
>
> wrapper.logfile.loglevel=INFO
>
> wrapper.logfile.maxsize=0
>
> wrapper.logfile.maxfiles=0
>
> # Log Level for sys/event log output.  (See docs for log levels)
>
> wrapper.syslog.loglevel=NONE
>
> wrapper.ping.timeout=30
>
> wrapper.ping.interval=10
>
> #********************************************************************
>
> # Wrapper Windows Properties
>
> #********************************************************************
>
> wrapper.console.title=Test Wrapper Sample Application
>
> #********************************************************************
>
> # Wrapper Windows NT/2000/XP Service Properties
>
> #********************************************************************
>
> wrapper.ntservice.name=LmrkWinSQL
>
> wrapper.ntservice.displayname=Infocenter - LmrkWinSQL
>
> wrapper.ntservice.description=Infocenter for Landmark Library for Windows/MS
> SQL
>
> wrapper.ntservice.dependency.1=
>
> wrapper.ntservice.starttype=AUTO_START
>
> wrapper.ntservice.interactive=false

-- 
Leif Mortenson
Tanuki Software, Ltd.
6-16-7-1001 Nishi-Kasai, Edogawa-ku
Tokyo 134-0088 Japan
Tel/Fax: +81-3-3878-3211
http://www.tanukisoftware.com
lei...@ta...