Menu
▾
▴
wrapper-user — User issues for Wrapper. (Support questions, etc.)
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(5) |
Oct
(13) |
Nov
(16) |
Dec
(29) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(38) |
Feb
(51) |
Mar
(51) |
Apr
(115) |
May
(82) |
Jun
(30) |
Jul
(50) |
Aug
(68) |
Sep
(57) |
Oct
(160) |
Nov
(80) |
Dec
(78) |
| 2004 |
Jan
(71) |
Feb
(75) |
Mar
(108) |
Apr
(87) |
May
(79) |
Jun
(70) |
Jul
(69) |
Aug
(39) |
Sep
(52) |
Oct
(47) |
Nov
(50) |
Dec
(32) |
| 2005 |
Jan
(22) |
Feb
(122) |
Mar
(46) |
Apr
(76) |
May
(31) |
Jun
(51) |
Jul
(61) |
Aug
(70) |
Sep
(37) |
Oct
(46) |
Nov
(57) |
Dec
(83) |
| 2006 |
Jan
(55) |
Feb
(81) |
Mar
(51) |
Apr
(67) |
May
(77) |
Jun
(43) |
Jul
(106) |
Aug
(64) |
Sep
(47) |
Oct
(64) |
Nov
(60) |
Dec
(12) |
| 2007 |
Jan
(50) |
Feb
(93) |
Mar
(49) |
Apr
(56) |
May
(40) |
Jun
(63) |
Jul
(40) |
Aug
(47) |
Sep
(54) |
Oct
(37) |
Nov
(54) |
Dec
(37) |
| 2008 |
Jan
(35) |
Feb
(39) |
Mar
(26) |
Apr
(14) |
May
(23) |
Jun
(51) |
Jul
(43) |
Aug
(26) |
Sep
(29) |
Oct
(31) |
Nov
(24) |
Dec
(16) |
| 2009 |
Jan
(21) |
Feb
(30) |
Mar
(74) |
Apr
(26) |
May
(26) |
Jun
(43) |
Jul
(23) |
Aug
(23) |
Sep
(15) |
Oct
(27) |
Nov
(37) |
Dec
(10) |
| 2010 |
Jan
(16) |
Feb
(28) |
Mar
(16) |
Apr
(45) |
May
(8) |
Jun
(68) |
Jul
(45) |
Aug
(44) |
Sep
(51) |
Oct
(7) |
Nov
(20) |
Dec
(21) |
| 2011 |
Jan
(14) |
Feb
(17) |
Mar
(7) |
Apr
(7) |
May
(48) |
Jun
(23) |
Jul
(5) |
Aug
(33) |
Sep
(22) |
Oct
(14) |
Nov
(14) |
Dec
(5) |
| 2012 |
Jan
|
Feb
(10) |
Mar
(12) |
Apr
(51) |
May
(10) |
Jun
(8) |
Jul
(14) |
Aug
(22) |
Sep
(9) |
Oct
(24) |
Nov
(14) |
Dec
(13) |
| 2013 |
Jan
(12) |
Feb
(4) |
Mar
(14) |
Apr
(19) |
May
(2) |
Jun
(5) |
Jul
(13) |
Aug
(10) |
Sep
(4) |
Oct
(11) |
Nov
(13) |
Dec
(2) |
| 2014 |
Jan
(3) |
Feb
(14) |
Mar
(5) |
Apr
(10) |
May
(10) |
Jun
(11) |
Jul
(10) |
Aug
(3) |
Sep
(13) |
Oct
(22) |
Nov
(14) |
Dec
(32) |
| 2015 |
Jan
(8) |
Feb
(2) |
Mar
(17) |
Apr
(1) |
May
(24) |
Jun
|
Jul
(4) |
Aug
|
Sep
(9) |
Oct
(9) |
Nov
(5) |
Dec
(2) |
| 2016 |
Jan
(8) |
Feb
(6) |
Mar
(6) |
Apr
(9) |
May
(3) |
Jun
(2) |
Jul
(7) |
Aug
(6) |
Sep
|
Oct
|
Nov
(1) |
Dec
(6) |
| 2017 |
Jan
(9) |
Feb
(8) |
Mar
(6) |
Apr
|
May
|
Jun
(3) |
Jul
(13) |
Aug
(10) |
Sep
(8) |
Oct
|
Nov
(6) |
Dec
|
| 2018 |
Jan
|
Feb
(5) |
Mar
(7) |
Apr
(2) |
May
|
Jun
|
Jul
(3) |
Aug
(2) |
Sep
(9) |
Oct
(1) |
Nov
(1) |
Dec
(1) |
| 2019 |
Jan
(9) |
Feb
|
Mar
|
Apr
(10) |
May
(3) |
Jun
|
Jul
(7) |
Aug
(1) |
Sep
|
Oct
(2) |
Nov
|
Dec
|
| 2020 |
Jan
(1) |
Feb
(1) |
Mar
(1) |
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
| 2021 |
Jan
(1) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
(3) |
| 2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
| 2023 |
Jan
(3) |
Feb
|
Mar
(1) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2024 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
(1) |
Jun
(1) |
Jul
(1) |
Aug
|
Sep
(1) |
Oct
|
Nov
(1) |
Dec
|
| 2025 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
(1) |
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
| 2026 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Lars S. <Lar...@if...> - 2010-12-29 13:02:06
|
Hi My Java application that I am starting with the Java Service Wrapper uses the network. I noticed that on Windows 7 it sometimes takes more time before the network is ready. Is there a way to define dependencies so that my application first is started when the network is up and running? Thanks in advance! Lars |
|
From: David H. <dho...@gm...> - 2010-12-28 11:12:21
|
Lief, Thanks for the detailed response and the HOWTO document. I had two separate folder structures each with bin/lib/config folders. The only differences I had from what you describe here is that my libwrapper.so was installed in the bin folder instead of the lib. (And the log was written to bin as well.) The only other thing I can think of that I may have did wrong is that I tried to configure each program to start on reboot...I followed some instructions on how to do this and may have gotten this wrong. I see you now have an install command that does this functionality via the script...nice I will try again following your HOWTO. Once again, thanks very much. -Dave On Mon, Dec 27, 2010 at 11:34 PM, Leif Mortenson < lei...@ta...> wrote: > Dave, > Where do you have the shell scripts and wrapper binaries installed? > If they are stored with your applications (in different folders) then > the two Wrappers should work without any conflicts. They will both > automatically store state files in their own directories and then > allocate their necessary backend socket ports on their own. > > When you start and stop the Wrapper, the shell script will store and > then reference a PID file to keep track of the Wrapper instance that > the script is managing. This script file needs to be unique for each > application. It will be if they are in their own directories. If > you want to put them in the same directory however, then you will need > to configure the shell scripts to avoid conflicts. > > I have written up a quick little HOWTO talking about this here: > > http://wrapper.tanukisoftware.com/doc/english/howto-multiple-wrappers-unix.html > > Please let me know if you still find anything confusing or have any > issues getting it working, so we can help further. > > Cheers, > Leif > > On Tue, Dec 28, 2010 at 9:14 AM, David Hoffer <dho...@gm...> wrote: > > This is probably a JSW/Linux newbie question. > > > > I want to deploy two (or more) applications to a Linux box and need to > > control each of them separately with JSW. That is, I will have two > > completely separate application folders each with its own wrapper and > > configuration. > > > > Years ago I tried this with JSW and it didn't work, what happened is that > > starting, stopping one of the wrappers effected both of the > applications. I > > had no idea why this happened. > > > > How do I configure for two applications on the same Linux box? Was this > a > > bug/problem with older versions or is there something I need to do to > > configure this? > > > > Thanks, > > -Dave > > > ------------------------------------------------------------------------------ > Learn how Oracle Real Application Clusters (RAC) One Node allows customers > to consolidate database storage, standardize their database environment, > and, > should the need arise, upgrade to a full multi-node Oracle RAC database > without downtime or disruption > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > Wrapper-user mailing list > Wra...@li... > https://lists.sourceforge.net/lists/listinfo/wrapper-user > |
|
From: Leif M. <lei...@ta...> - 2010-12-28 06:34:22
|
Dave, Where do you have the shell scripts and wrapper binaries installed? If they are stored with your applications (in different folders) then the two Wrappers should work without any conflicts. They will both automatically store state files in their own directories and then allocate their necessary backend socket ports on their own. When you start and stop the Wrapper, the shell script will store and then reference a PID file to keep track of the Wrapper instance that the script is managing. This script file needs to be unique for each application. It will be if they are in their own directories. If you want to put them in the same directory however, then you will need to configure the shell scripts to avoid conflicts. I have written up a quick little HOWTO talking about this here: http://wrapper.tanukisoftware.com/doc/english/howto-multiple-wrappers-unix.html Please let me know if you still find anything confusing or have any issues getting it working, so we can help further. Cheers, Leif On Tue, Dec 28, 2010 at 9:14 AM, David Hoffer <dho...@gm...> wrote: > This is probably a JSW/Linux newbie question. > > I want to deploy two (or more) applications to a Linux box and need to > control each of them separately with JSW. That is, I will have two > completely separate application folders each with its own wrapper and > configuration. > > Years ago I tried this with JSW and it didn't work, what happened is that > starting, stopping one of the wrappers effected both of the applications. I > had no idea why this happened. > > How do I configure for two applications on the same Linux box? Was this a > bug/problem with older versions or is there something I need to do to > configure this? > > Thanks, > -Dave |
|
From: David H. <dho...@gm...> - 2010-12-28 02:38:09
|
Being a Java programmer and not a Linux guru the best way I can explain the prior behavior I observed is that JSW behaved statically instead of per instance. However the only thing the same between the two instances, that I am aware of, is that the name of the wrapper executable and conf files are the same among instances. I assume there is a way to configure JSW as separate instances on Linux? -Dave On Mon, Dec 27, 2010 at 5:14 PM, David Hoffer <dho...@gm...> wrote: > This is probably a JSW/Linux newbie question. > > I want to deploy two (or more) applications to a Linux box and need to > control each of them separately with JSW. That is, I will have two > completely separate application folders each with its own wrapper and > configuration. > > Years ago I tried this with JSW and it didn't work, what happened is that > starting, stopping one of the wrappers effected both of the applications. I > had no idea why this happened. > > How do I configure for two applications on the same Linux box? Was this a > bug/problem with older versions or is there something I need to do to > configure this? > > Thanks, > -Dave > |
|
From: David H. <dho...@gm...> - 2010-12-28 01:15:03
|
I just sent a message after subscribing but did not get an email copy. Please let me know if the group is getting this email. Thanks, -Dave |
|
From: David H. <dho...@gm...> - 2010-12-28 00:15:04
|
This is probably a JSW/Linux newbie question. I want to deploy two (or more) applications to a Linux box and need to control each of them separately with JSW. That is, I will have two completely separate application folders each with its own wrapper and configuration. Years ago I tried this with JSW and it didn't work, what happened is that starting, stopping one of the wrappers effected both of the applications. I had no idea why this happened. How do I configure for two applications on the same Linux box? Was this a bug/problem with older versions or is there something I need to do to configure this? Thanks, -Dave |
|
From: Leif M. <lei...@ta...> - 2010-12-20 18:02:20
|
Lars, When you set the java command to the following, it will attempt to locate Java on the PATH: wrapper.java.command=java Many systems define a JAVA_HOME variable which will let you do the following: wrapper.java.command=%JAVA_HOME%/bin/java If you application ships with its own JRE, then you can also do something like the following: wrapper.java.command=../jre/bin/java Cheers, Leif On Fri, Dec 17, 2010 at 8:33 PM, Lars Schnoor <Lar...@if...> wrote: > Hi Leif > > Thanks for the reply, it seems like I had a mistake in my wrapper.conf, > so the wrapper.debug=TRUE helped a lot. > Now that I can see the command that is used I was wondering why: > wrapper.java.command=java > in my wrapper.conf results in a command like: > "c:\windows\system32\java.exe > My problem is that my application now works when I enter the correct > path to java.exe as wrapper.java.command, but since the location can be > different from system to system I would like to avoid to hard code it, > is there away to get the correct path without hard coding it in the > wrapper.conf? > Thanks > > Lars > > On 17-12-2010 09:51, Leif Mortenson wrote: >> Lars, >> Could you please set the wrapper.debug=TRUE property and then post >> your wrapper.log and wrapper.conf as attachments? I am not sure what >> would be causing the errors you mentioned, but I should be able to see >> from the logs. >> >> The message about the "public static" is that the WrapperSimpleApp >> requires that the main class you specify is a public class, and that >> it defines a public static main method. If either of those is not >> true then you will not be able to access it. >> >> Cheers, >> Leif >> >> On Thu, Dec 16, 2010 at 10:54 PM, Lars Schnoor<Lar...@if...> wrote: >>> Hi again >>> >>> I am again trying the mentioned scenario, but with the parameters as: >>> wrapper.java.mainclass=special.start.class >>> wrapper.app.parameter.1=-"parameter to special start class" >>> wrapper.app.parameter.2=myStartClass >>> >>> where myStartClass implements WrapperListener. >>> This works fine on Linux (Red Hat Enterprise Linux 5.5 64-bit with Java >>> 1.6.0 Update 22 64-bit). >>> On Windows 7 (64-bit with Java 1.6.0 Update 21 32-bit) it does not work, >>> I get a lot of error messages in the log file about a timeout in waiting >>> for a signal from the JVM. >>> Any help would be greatly appreciated. >>> >>> When I try Integration method #1 I get an exception about that the >>> WrapperSimpleApp can not access a member with modifiers "public static" >>> in the special.start.class. >>> Thanks >>> >>> Lars >>> >>> On 06-11-2009 16:40, Leif Mortenson wrote: >>>> Lars, >>>> If the WrapperManager is not initialized, then the Wrapper will kill >>>> the JVM by default after 30 seconds or so. Have you disabled any of >>>> the default timeouts? If so, all of the freeze detection code will >>>> not be working. >>>> >>>> Have you tried using Integration method #1? It lets you do codeless >>>> integration. If your special.start.class is designed to run as a >>>> normal standalone Java main class then following will work: >>>> >>>> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp >>>> wrapper.app.parameter.1=special.start.class >>>> wrapper.app.parameter.2=-"parameter to special start class" >>>> wrapper.app.parameter.3=myStartClass >>>> >>>> See the following documentation for more details: >>>> http://wrapper.tanukisoftware.org/doc/english/integrate-simple-win.html >>>> >>>> Cheers, >>>> Leif >>>> >>>> On Fri, Nov 6, 2009 at 11:37 PM, Lars Schnoor<Lar...@if...> wrote: >>>>> Hi Leif >>>>> Thanks for the quick reply. I changed the wrapper parameters like you >>>>> suggested and now it works. >>>>> The special.start.class does not implement WrapperListener or initialize the >>>>> WrapperManager, but it works anyway. >>>>> I did not make the special.start.class myself, so I can't make implement the >>>>> WrapperListener. >>>>> >>>>> Lars >>>>> >>>>> Leif Mortenson wrote: >>>>> >>>>> Lars, >>>>> All versions of the Wrapper actually expect that you correctly break >>>>> the parameters into individual parameters. On Windows they are all >>>>> reconstructed into a single line so it works, but on UNIX, the command >>>>> is broken up into the individual components and passed to the system >>>>> as an array. >>>>> >>>>> You need to do the following and it will work on all platforms: >>>>> >>>>> wrapper.java.mainclass=special.start.class >>>>> wrapper.app.parameter.1=-"parameter to special start class" >>>>> wrapper.app.parameter.1=myStartClass >>>>> >>>>> The class you specify for the wrapper.java.mainclass must implement >>>>> the WrapperListener and initialize the WrapperManager class directly >>>>> or indirectly. You are using what we call Integration Method #3. >>>>> Please read over the following page and let me know if you have any >>>>> additional questions. >>>>> http://wrapper.tanukisoftware.org/doc/english/integrate-listener.html >>>>> >>>>> Cheers, >>>>> Leif >>>>> >>>>> >>>>> On Fri, Nov 6, 2009 at 10:18 PM, Lars Schnoor<Lar...@if...> wrote: >>>>> >>>>> >>>>> Hi >>>>> I have a Java application that is started in a special way. I have a main >>>>> class in my application that implements the WrapperListener interface. >>>>> from the command prompt I can start my application with: >>>>> java -cp myApplication.jar special.start.class -"parameter to special start >>>>> class" myStartClass >>>>> I can start my application by the above line on both Windows and Linux. >>>>> >>>>> On Windows I put the following line in my wrapper.conf: >>>>> wrapper.java.mainclass=special.start.class -"parameter to special start >>>>> class" myStartClass >>>>> And it works fine. >>>>> >>>>> On Linux (Red Hat Enterprise Linux 5 64-bit) I tried the same, putting the >>>>> same line in the wrapper.conf: >>>>> wrapper.java.mainclass=special.start.class -"parameter to special start >>>>> class" myStartClass >>>>> Here it does not work, I get an ClassNotFoundException for the class: >>>>> special.start.class -"parameter to special start class" myStartClass >>>>> For me it seems as if the wrapper on Linux sees special.start.class >>>>> -"parameter to special start class" myStartClass as one class, where the >>>>> wrapper on Windows starts the special.start.class with -"parameter to >>>>> special start class" myStartClass as parameters. >>>>> I tried putting: >>>>> wrapper.java.mainclass=special.start.class >>>>> wrapper.app.parameter.1=-"parameter to special start class" myStartClass >>>>> in the wrapper.conf and with this the wrapper starts, but since the >>>>> special.start.class does not implement the WrapperListener interface, the >>>>> wrapper shuts down after five tries. >>>>> Any idea how I can get it to work on Linux, I am using version 3.2.3 of the >>>>> wrapper? >>>>> Thanks in advance! >>>>> >>>>> Lars |
|
From: Lars S. <Lar...@if...> - 2010-12-17 11:33:23
|
Hi Leif Thanks for the reply, it seems like I had a mistake in my wrapper.conf, so the wrapper.debug=TRUE helped a lot. Now that I can see the command that is used I was wondering why: wrapper.java.command=java in my wrapper.conf results in a command like: "c:\windows\system32\java.exe My problem is that my application now works when I enter the correct path to java.exe as wrapper.java.command, but since the location can be different from system to system I would like to avoid to hard code it, is there away to get the correct path without hard coding it in the wrapper.conf? Thanks Lars On 17-12-2010 09:51, Leif Mortenson wrote: > Lars, > Could you please set the wrapper.debug=TRUE property and then post > your wrapper.log and wrapper.conf as attachments? I am not sure what > would be causing the errors you mentioned, but I should be able to see > from the logs. > > The message about the "public static" is that the WrapperSimpleApp > requires that the main class you specify is a public class, and that > it defines a public static main method. If either of those is not > true then you will not be able to access it. > > Cheers, > Leif > > On Thu, Dec 16, 2010 at 10:54 PM, Lars Schnoor<Lar...@if...> wrote: >> Hi again >> >> I am again trying the mentioned scenario, but with the parameters as: >> wrapper.java.mainclass=special.start.class >> wrapper.app.parameter.1=-"parameter to special start class" >> wrapper.app.parameter.2=myStartClass >> >> where myStartClass implements WrapperListener. >> This works fine on Linux (Red Hat Enterprise Linux 5.5 64-bit with Java >> 1.6.0 Update 22 64-bit). >> On Windows 7 (64-bit with Java 1.6.0 Update 21 32-bit) it does not work, >> I get a lot of error messages in the log file about a timeout in waiting >> for a signal from the JVM. >> Any help would be greatly appreciated. >> >> When I try Integration method #1 I get an exception about that the >> WrapperSimpleApp can not access a member with modifiers "public static" >> in the special.start.class. >> Thanks >> >> Lars >> >> On 06-11-2009 16:40, Leif Mortenson wrote: >>> Lars, >>> If the WrapperManager is not initialized, then the Wrapper will kill >>> the JVM by default after 30 seconds or so. Have you disabled any of >>> the default timeouts? If so, all of the freeze detection code will >>> not be working. >>> >>> Have you tried using Integration method #1? It lets you do codeless >>> integration. If your special.start.class is designed to run as a >>> normal standalone Java main class then following will work: >>> >>> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp >>> wrapper.app.parameter.1=special.start.class >>> wrapper.app.parameter.2=-"parameter to special start class" >>> wrapper.app.parameter.3=myStartClass >>> >>> See the following documentation for more details: >>> http://wrapper.tanukisoftware.org/doc/english/integrate-simple-win.html >>> >>> Cheers, >>> Leif >>> >>> On Fri, Nov 6, 2009 at 11:37 PM, Lars Schnoor<Lar...@if...> wrote: >>>> Hi Leif >>>> Thanks for the quick reply. I changed the wrapper parameters like you >>>> suggested and now it works. >>>> The special.start.class does not implement WrapperListener or initialize the >>>> WrapperManager, but it works anyway. >>>> I did not make the special.start.class myself, so I can't make implement the >>>> WrapperListener. >>>> >>>> Lars >>>> >>>> Leif Mortenson wrote: >>>> >>>> Lars, >>>> All versions of the Wrapper actually expect that you correctly break >>>> the parameters into individual parameters. On Windows they are all >>>> reconstructed into a single line so it works, but on UNIX, the command >>>> is broken up into the individual components and passed to the system >>>> as an array. >>>> >>>> You need to do the following and it will work on all platforms: >>>> >>>> wrapper.java.mainclass=special.start.class >>>> wrapper.app.parameter.1=-"parameter to special start class" >>>> wrapper.app.parameter.1=myStartClass >>>> >>>> The class you specify for the wrapper.java.mainclass must implement >>>> the WrapperListener and initialize the WrapperManager class directly >>>> or indirectly. You are using what we call Integration Method #3. >>>> Please read over the following page and let me know if you have any >>>> additional questions. >>>> http://wrapper.tanukisoftware.org/doc/english/integrate-listener.html >>>> >>>> Cheers, >>>> Leif >>>> >>>> >>>> On Fri, Nov 6, 2009 at 10:18 PM, Lars Schnoor<Lar...@if...> wrote: >>>> >>>> >>>> Hi >>>> I have a Java application that is started in a special way. I have a main >>>> class in my application that implements the WrapperListener interface. >>>> from the command prompt I can start my application with: >>>> java -cp myApplication.jar special.start.class -"parameter to special start >>>> class" myStartClass >>>> I can start my application by the above line on both Windows and Linux. >>>> >>>> On Windows I put the following line in my wrapper.conf: >>>> wrapper.java.mainclass=special.start.class -"parameter to special start >>>> class" myStartClass >>>> And it works fine. >>>> >>>> On Linux (Red Hat Enterprise Linux 5 64-bit) I tried the same, putting the >>>> same line in the wrapper.conf: >>>> wrapper.java.mainclass=special.start.class -"parameter to special start >>>> class" myStartClass >>>> Here it does not work, I get an ClassNotFoundException for the class: >>>> special.start.class -"parameter to special start class" myStartClass >>>> For me it seems as if the wrapper on Linux sees special.start.class >>>> -"parameter to special start class" myStartClass as one class, where the >>>> wrapper on Windows starts the special.start.class with -"parameter to >>>> special start class" myStartClass as parameters. >>>> I tried putting: >>>> wrapper.java.mainclass=special.start.class >>>> wrapper.app.parameter.1=-"parameter to special start class" myStartClass >>>> in the wrapper.conf and with this the wrapper starts, but since the >>>> special.start.class does not implement the WrapperListener interface, the >>>> wrapper shuts down after five tries. >>>> Any idea how I can get it to work on Linux, I am using version 3.2.3 of the >>>> wrapper? >>>> Thanks in advance! >>>> >>>> Lars > ------------------------------------------------------------------------------ > Lotusphere 2011 > Register now for Lotusphere 2011 and learn how > to connect the dots, take your collaborative environment > to the next level, and enter the era of Social Business. > http://p.sf.net/sfu/lotusphere-d2d > _______________________________________________ > Wrapper-user mailing list > Wra...@li... > https://lists.sourceforge.net/lists/listinfo/wrapper-user |
|
From: Leif M. <lei...@ta...> - 2010-12-17 08:51:25
|
Lars, Could you please set the wrapper.debug=TRUE property and then post your wrapper.log and wrapper.conf as attachments? I am not sure what would be causing the errors you mentioned, but I should be able to see from the logs. The message about the "public static" is that the WrapperSimpleApp requires that the main class you specify is a public class, and that it defines a public static main method. If either of those is not true then you will not be able to access it. Cheers, Leif On Thu, Dec 16, 2010 at 10:54 PM, Lars Schnoor <Lar...@if...> wrote: > Hi again > > I am again trying the mentioned scenario, but with the parameters as: > wrapper.java.mainclass=special.start.class > wrapper.app.parameter.1=-"parameter to special start class" > wrapper.app.parameter.2=myStartClass > > where myStartClass implements WrapperListener. > This works fine on Linux (Red Hat Enterprise Linux 5.5 64-bit with Java > 1.6.0 Update 22 64-bit). > On Windows 7 (64-bit with Java 1.6.0 Update 21 32-bit) it does not work, > I get a lot of error messages in the log file about a timeout in waiting > for a signal from the JVM. > Any help would be greatly appreciated. > > When I try Integration method #1 I get an exception about that the > WrapperSimpleApp can not access a member with modifiers "public static" > in the special.start.class. > Thanks > > Lars > > On 06-11-2009 16:40, Leif Mortenson wrote: >> Lars, >> If the WrapperManager is not initialized, then the Wrapper will kill >> the JVM by default after 30 seconds or so. Have you disabled any of >> the default timeouts? If so, all of the freeze detection code will >> not be working. >> >> Have you tried using Integration method #1? It lets you do codeless >> integration. If your special.start.class is designed to run as a >> normal standalone Java main class then following will work: >> >> wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp >> wrapper.app.parameter.1=special.start.class >> wrapper.app.parameter.2=-"parameter to special start class" >> wrapper.app.parameter.3=myStartClass >> >> See the following documentation for more details: >> http://wrapper.tanukisoftware.org/doc/english/integrate-simple-win.html >> >> Cheers, >> Leif >> >> On Fri, Nov 6, 2009 at 11:37 PM, Lars Schnoor<Lar...@if...> wrote: >>> Hi Leif >>> Thanks for the quick reply. I changed the wrapper parameters like you >>> suggested and now it works. >>> The special.start.class does not implement WrapperListener or initialize the >>> WrapperManager, but it works anyway. >>> I did not make the special.start.class myself, so I can't make implement the >>> WrapperListener. >>> >>> Lars >>> >>> Leif Mortenson wrote: >>> >>> Lars, >>> All versions of the Wrapper actually expect that you correctly break >>> the parameters into individual parameters. On Windows they are all >>> reconstructed into a single line so it works, but on UNIX, the command >>> is broken up into the individual components and passed to the system >>> as an array. >>> >>> You need to do the following and it will work on all platforms: >>> >>> wrapper.java.mainclass=special.start.class >>> wrapper.app.parameter.1=-"parameter to special start class" >>> wrapper.app.parameter.1=myStartClass >>> >>> The class you specify for the wrapper.java.mainclass must implement >>> the WrapperListener and initialize the WrapperManager class directly >>> or indirectly. You are using what we call Integration Method #3. >>> Please read over the following page and let me know if you have any >>> additional questions. >>> http://wrapper.tanukisoftware.org/doc/english/integrate-listener.html >>> >>> Cheers, >>> Leif >>> >>> >>> On Fri, Nov 6, 2009 at 10:18 PM, Lars Schnoor<Lar...@if...> wrote: >>> >>> >>> Hi >>> I have a Java application that is started in a special way. I have a main >>> class in my application that implements the WrapperListener interface. >>> from the command prompt I can start my application with: >>> java -cp myApplication.jar special.start.class -"parameter to special start >>> class" myStartClass >>> I can start my application by the above line on both Windows and Linux. >>> >>> On Windows I put the following line in my wrapper.conf: >>> wrapper.java.mainclass=special.start.class -"parameter to special start >>> class" myStartClass >>> And it works fine. >>> >>> On Linux (Red Hat Enterprise Linux 5 64-bit) I tried the same, putting the >>> same line in the wrapper.conf: >>> wrapper.java.mainclass=special.start.class -"parameter to special start >>> class" myStartClass >>> Here it does not work, I get an ClassNotFoundException for the class: >>> special.start.class -"parameter to special start class" myStartClass >>> For me it seems as if the wrapper on Linux sees special.start.class >>> -"parameter to special start class" myStartClass as one class, where the >>> wrapper on Windows starts the special.start.class with -"parameter to >>> special start class" myStartClass as parameters. >>> I tried putting: >>> wrapper.java.mainclass=special.start.class >>> wrapper.app.parameter.1=-"parameter to special start class" myStartClass >>> in the wrapper.conf and with this the wrapper starts, but since the >>> special.start.class does not implement the WrapperListener interface, the >>> wrapper shuts down after five tries. >>> Any idea how I can get it to work on Linux, I am using version 3.2.3 of the >>> wrapper? >>> Thanks in advance! >>> >>> Lars |
|
From: Lars S. <Lar...@if...> - 2010-12-16 14:15:16
|
Hi again I am again trying the mentioned scenario, but with the parameters as: wrapper.java.mainclass=special.start.class wrapper.app.parameter.1=-"parameter to special start class" wrapper.app.parameter.2=myStartClass where myStartClass implements WrapperListener. This works fine on Linux (Red Hat Enterprise Linux 5.5 64-bit with Java 1.6.0 Update 22 64-bit). On Windows 7 (64-bit with Java 1.6.0 Update 21 32-bit) it does not work, I get a lot of error messages in the log file about a timeout in waiting for a signal from the JVM. Any help would be greatly appreciated. When I try Integration method #1 I get an exception about that the WrapperSimpleApp can not access a member with modifiers "public static" in the special.start.class. Thanks Lars On 06-11-2009 16:40, Leif Mortenson wrote: > Lars, > If the WrapperManager is not initialized, then the Wrapper will kill > the JVM by default after 30 seconds or so. Have you disabled any of > the default timeouts? If so, all of the freeze detection code will > not be working. > > Have you tried using Integration method #1? It lets you do codeless > integration. If your special.start.class is designed to run as a > normal standalone Java main class then following will work: > > wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp > wrapper.app.parameter.1=special.start.class > wrapper.app.parameter.2=-"parameter to special start class" > wrapper.app.parameter.3=myStartClass > > See the following documentation for more details: > http://wrapper.tanukisoftware.org/doc/english/integrate-simple-win.html > > Cheers, > Leif > > On Fri, Nov 6, 2009 at 11:37 PM, Lars Schnoor<Lar...@if...> wrote: >> Hi Leif >> Thanks for the quick reply. I changed the wrapper parameters like you >> suggested and now it works. >> The special.start.class does not implement WrapperListener or initialize the >> WrapperManager, but it works anyway. >> I did not make the special.start.class myself, so I can't make implement the >> WrapperListener. >> >> Lars >> >> Leif Mortenson wrote: >> >> Lars, >> All versions of the Wrapper actually expect that you correctly break >> the parameters into individual parameters. On Windows they are all >> reconstructed into a single line so it works, but on UNIX, the command >> is broken up into the individual components and passed to the system >> as an array. >> >> You need to do the following and it will work on all platforms: >> >> wrapper.java.mainclass=special.start.class >> wrapper.app.parameter.1=-"parameter to special start class" >> wrapper.app.parameter.1=myStartClass >> >> The class you specify for the wrapper.java.mainclass must implement >> the WrapperListener and initialize the WrapperManager class directly >> or indirectly. You are using what we call Integration Method #3. >> Please read over the following page and let me know if you have any >> additional questions. >> http://wrapper.tanukisoftware.org/doc/english/integrate-listener.html >> >> Cheers, >> Leif >> >> >> On Fri, Nov 6, 2009 at 10:18 PM, Lars Schnoor<Lar...@if...> wrote: >> >> >> Hi >> I have a Java application that is started in a special way. I have a main >> class in my application that implements the WrapperListener interface. >> from the command prompt I can start my application with: >> java -cp myApplication.jar special.start.class -"parameter to special start >> class" myStartClass >> I can start my application by the above line on both Windows and Linux. >> >> On Windows I put the following line in my wrapper.conf: >> wrapper.java.mainclass=special.start.class -"parameter to special start >> class" myStartClass >> And it works fine. >> >> On Linux (Red Hat Enterprise Linux 5 64-bit) I tried the same, putting the >> same line in the wrapper.conf: >> wrapper.java.mainclass=special.start.class -"parameter to special start >> class" myStartClass >> Here it does not work, I get an ClassNotFoundException for the class: >> special.start.class -"parameter to special start class" myStartClass >> For me it seems as if the wrapper on Linux sees special.start.class >> -"parameter to special start class" myStartClass as one class, where the >> wrapper on Windows starts the special.start.class with -"parameter to >> special start class" myStartClass as parameters. >> I tried putting: >> wrapper.java.mainclass=special.start.class >> wrapper.app.parameter.1=-"parameter to special start class" myStartClass >> in the wrapper.conf and with this the wrapper starts, but since the >> special.start.class does not implement the WrapperListener interface, the >> wrapper shuts down after five tries. >> Any idea how I can get it to work on Linux, I am using version 3.2.3 of the >> wrapper? >> Thanks in advance! >> >> Lars > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Wrapper-user mailing list > Wra...@li... > https://lists.sourceforge.net/lists/listinfo/wrapper-user |
|
From: Leif M. <le...@ta...> - 2010-12-09 04:05:02
|
Johannes, Thank you for the update. I am glad you got things working. Did he by chance note how far off the times were? Also what is the versions of ESX that you are running? This is something that could affect other users and I would like to reproduce it and see if we can find a way to at least log a warning if not avoid the problem all together. Thanks in advance, Sincerely, Leif Mortenson Tanuki Software, Ltd. On Thu, Dec 9, 2010 at 4:45 AM, Johannes Nicolai <jni...@co...> wrote: > Hi Leif, > > After our admins have resynched the ESX server with the virtual guests (time was off so much that ntp did not adjust any more), the problem does not seem to exist anymore. > Thanks for your help! > > Best, Johannes > > -----Original Message----- > From: Leif Mortenson [mailto:le...@ta...] > Sent: Tuesday, November 30, 2010 7:04 PM > To: Johannes Nicolai > Cc: wra...@li... > Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in wrapperSleep after some seconds > > Johannes, > A few more questions and testing ideas. > > 1) What is the version of vmware ESX that you were using? > > 2) We have been looking into how nanosleep works on CentOS. All Linux platforms use the CLOCK_MONOTONIC clock rather than the CLOCK_REALTIME, so they should be fine. Even if they were using CLOCK_REALTIME, the nanosleep system call is supposed to take system time changes into account and always use a relative time. > http://www.kernel.org/doc/man-pages/online/pages/man2/clock_settime.2.html > > BUT. There is a note at the bottom of the above page which says that each CPU in an SMP system could be using a different clock source. > This can result in them being slightly out of sync and have a slightly different frequency. > One problem that I have noticed with vmware in the past is that when you are not using ntp, the guest system time can drift drastically if the guest is being starved of CPU because of other processes on the > host. I have seem drifts of several hours within a single day. The > problem is that the Guest has no way of telling that it is behind. > I am wondering if it is possible that one of your CPU clocks is drifting much more than the other. > This is not something that could happen on a physical machine. > > Then if the Wrapper's threads are jumping between the two CPUs, that might explain why the time is changing. This is just a guess as we have not been able to reproduce this here. > > Here are some docs on timekeeping with vmware. There are several settings in ESX 4 to improve things: > http://www.vmware.com/pdf/vmware_timekeeping.pdf > http://www.fjc.net/linux/linux-and-vmware-related-issues/linux-2-6-kernels-and-vmware-clock-drift-issues > > 3) Along the lines above, I am wondering what happens if you restrict the Wrapper to run on a specific CPU rather than allowing it to balance between the two. This can be done with the taskset command. > http://www.cyberciti.biz/faq/taskset-cpu-affinity-command/ > Would it be possible for you to try this out to force the Wrapper to a single CPU? > > Thanks in advance, > Leif > > On Tue, Nov 30, 2010 at 12:38 AM, Leif Mortenson <le...@ta...> wrote: >> Johannes, >> Thank you for the update on ntpd. Good to be able to remove that as a >> possibility. >> >> This time difference is also 4398 seconds. >>> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >>> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake >> >> We will keep working on this on this end as well. >> >> Cheers, >> Leif >> >> On Mon, Nov 29, 2010 at 11:08 PM, Johannes Nicolai <jni...@co...> wrote: >>> Hi Leif, >>> >>> I shutdown ntpd and still get a similar error: >>> >>> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper >>> state: STARTED STATUS | wrapper | 2010/11/29 06:03:05 | Loop: >>> handle JVM state: STARTED STATUS | wrapper | 2010/11/29 06:03:05 | >>> Loop: sleep STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: >>> nanosleep 100ms STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: >>> awake STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep >>> 100ms STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake >>> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger >>> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process JVM >>> output STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process >>> socket STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain >>> logger(2) STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle >>> wrapper state: STARTED STATUS | wrapper | 2010/11/29 06:03:05 | >>> Loop: handle JVM state: STARTED STATUS | wrapper | 2010/11/29 >>> 06:03:05 | Loop: sleep STATUS | wrapper | 2010/11/29 06:03:05 | >>> Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/29 06:03:05 | >>> Sleep: awake STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: >>> nanosleep 100ms STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: >>> awake STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain >>> logger STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process >>> JVM output STATUS | wrapper | 2010/11/29 06:03:05 | Loop: >>> process socket STATUS | wrapper | 2010/11/29 06:03:05 | Loop: >>> maintain logger(2) STATUS | wrapper | 2010/11/29 06:03:05 | >>> Loop: handle wrapper state: STARTED STATUS | wrapper | 2010/11/29 >>> 06:03:05 | Loop: handle JVM state: STARTED STATUS | wrapper | >>> 2010/11/29 06:03:05 | Loop: sleep STATUS | wrapper | 2010/11/29 >>> 06:03:05 | Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/29 >>> 07:16:23 | Sleep: awake STATUS | wrapper | 2010/11/29 07:16:23 | >>> Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/29 07:16:23 | >>> Sleep: awake STATUS | wrapper | 2010/11/29 07:16:23 | Loop: >>> maintain logger STATUS | wrapper | 2010/11/29 07:16:23 | Loop: >>> process JVM output STATUS | wrapper | 2010/11/29 07:16:23 | >>> Loop: process socket STATUS | wrapper | 2010/11/29 07:16:23 | >>> Loop: maintain logger(2) STATUS | wrapper | 2010/11/29 07:16:23 | >>> Loop: handle wrapper state: STARTED STATUS | wrapper | 2010/11/29 >>> 07:16:23 | Loop: handle JVM state: STARTED STATUS | wrapper | >>> 2010/11/29 07:16:23 | Loop: sleep >>> >>> I will ask ops for the time difference between host and vm. >>> >>> Best, Johannes >>> >>> -----Original Message----- >>> From: Leif Mortenson [mailto:le...@ta...] >>> Sent: Monday, November 29, 2010 5:39 AM >>> To: Johannes Nicolai >>> Cc: wra...@li... >>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in >>> wrapperSleep after some seconds >>> >>> Johannes, >>> Thank you this latest log. It is showing that the system time that the Wrapper sees is being changed. The change is not happening when we are in a nanosleep so I don't think they are related. >>> >>> The first event shows a jump forwards of 4398 seconds: >>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms >>> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake >>> >>> The second event ALSO shows a jump forwards of 4398 seconds: >>> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process socket >>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: maintain >>> logger(2) >>> >>> The fact that they are identical seems a little odd. I need to look >>> into this a bit, but could you double check the system time of the host of your VM server. Is it possible that vmware is trying to sync your host and guest, at the same time that your ntp is doing the same. >>> >>> The Wrapper seems to be working fine when the time jumps forward, but nanosleep is getting stuck later when something else happens. I am wondering if that "something" is the return of the system time to its previous value. We are running some tests along these lines here locally. >>> >>> The 4398 seconds number is 0x112e. So it does not appear to be a significant number in and of itself. >>> >>> Cheers, >>> Leif >>> >>> >>> On Fri, Nov 26, 2010 at 8:48 PM, Johannes Nicolai <jni...@co...> wrote: >>>> Hi Leif, >>>> >>>>>What is the timestamp in the lines after the "03:20:19" line? Does >>>>>the >>>> time go back immediately? >>>> This is the last line in wrapper.log >>>> >>>> After this one, the wrapper starts to hang. >>>> >>>> I added wrapper.loop_output=true to my config and now see the >>>> following last lines before the wrapper starts to hang: >>>> >>>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper >>>> | >>>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>>> 03:43:30 | Loop: process JVM output STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26 >>>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: handle wrapper >>>> state: STARTED >>>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state: >>>> STARTED >>>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper >>>> | >>>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>>> 03:43:30 | Loop: process JVM output STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26 >>>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:30 | Loop: handle wrapper >>>> state: STARTED >>>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state: >>>> STARTED >>>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper >>>> | >>>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>>> 2010/11/26 03:43:31 | Sleep: awake STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:31 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>>> 03:43:31 | Loop: process JVM output STATUS | wrapper | >>>> 2010/11/26 >>>> 03:43:31 | Loop: process socket STATUS | wrapper | 2010/11/26 >>>> 04:56:49 | Loop: maintain logger(2) STATUS | wrapper | >>>> 2010/11/26 >>>> 04:56:49 | Loop: handle wrapper >>>> state: STARTED >>>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: handle JVM state: >>>> STARTED >>>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: sleep STATUS | >>>> wrapper | 2010/11/26 04:56:49 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 04:56:49 | Sleep: awake >>>> >>>> Current time on my machine is 03:45:12 >>>> >>>> Best, Johannes >>>> >>>> -----Original Message----- >>>> From: Leif Mortenson [mailto:le...@ta...] >>>> Sent: Friday, November 26, 2010 12:27 PM >>>> To: Johannes Nicolai >>>> Cc: Leif Mortenson; <wra...@li...> >>>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in >>>> wrapperSleep after some seconds >>>> >>>> Johannesburg, >>>> That looks like a big clue. >>>> >>>> What is the timestamp in the lines after the "03:20:19" line? Does >>>> the time go back immediately? >>>> >>>> To get more output, can you try also setting the following? >>>> >>>> Wrapper.loop_output=true >>>> wrapper.timer_output=true >>>> >>>> I am on my mobile but I think those are correct. I am wondering if >>>> the time is being set high, then back again. The logging code >>>> queries the system time for each line. It immediately shows changes >>>> when there are time adjustments. >>>> >>>> Something like ntp changes seems unlikely as it is always right >>>> after the Wrapper starts though. >>>> >>>> - Leif >>>> >>>> >>>> On 2010/11/26, at 19:22, "Johannes Nicolai" <jni...@co...> wrote: >>>> >>>>> Hi Leif, >>>>> >>>>> Yes, once the process starts to hang (after ten seconds to one >>>> minute), both threads are always in the nanonsleep syscall. >>>>> These are the last lines in wrapper.log: >>>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep >>>>> 100ms STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake >>>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep >>>>> 100ms STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake >>>>> STATUS | wrapper | >>>>> 2010/11/26 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper >>>>> | >>>>> 2010/11/26 02:07:01 | Sleep: awake STATUS | wrapper | >>>>> 2010/11/26 >>>>> 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper | >>>>> 2010/11/26 >>>>> 02:07:01 | Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 >>>>> | >>>>> Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26 02:07:01 | >>>>> Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: >>>>> nanosleep 100ms STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: >>>>> awake >>>>> >>>>> The last output that differs from this pattern is DEBUG | wrapperp >>>>> | >>>>> 2010/11/26 02:06:39 | send a packet PING : ping STATUS | wrapper | >>>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper >>>>> | >>>>> 2010/11/26 02:06:39 | Sleep: awake STATUS | wrapper | >>>>> 2010/11/26 >>>>> 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper | >>>>> 2010/11/26 >>>>> 02:06:39 | Sleep: awake INFO | jvm 1 | 2010/11/26 02:06:39 >>>>> | WrapperManager Debug: >>>> Received a packet PING : ping >>>>> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: >>>>> Send a >>>> packet PING : ping >>>>> DEBUG | wrapperp | 2010/11/26 02:06:39 | read a packet PING : ping >>>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep >>>>> 100ms STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake >>>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep >>>>> 100ms STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake >>>>> STATUS | wrapper | >>>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms >>>>> >>>>> What I find surprising is the huge jump in time from STATUS | >>>>> wrapper >>>>> | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms to STATUS | >>>>> wrapper | 2010/11/26 03:20:19 | Sleep: awake >>>>> >>>>> If you look at the timestamps. The current date at this machine now >>>>> is >>>> 02:19:44, so the time for the last awake has not been reached by now. >>>>> >>>>> Best, Johannes >>>>> >>>>> -----Original Message----- >>>>> From: le...@ta... [mailto:le...@ta...] On >>>>> Behalf Of Leif Mortenson >>>>> Sent: Friday, November 26, 2010 11:03 AM >>>>> To: wra...@li... >>>>> Cc: Johannes Nicolai >>>>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in >>>>> wrapperSleep after some seconds >>>>> >>>>> Johannes, >>>>> When you get the hang, are you consistently seeing that both of the >>>> threads are getting stuck in the nanosleep calls? >>>>> >>>>> We have a low level debug mode that can be enabled to show calls to >>>> sleep. This was originally added to debug problems with usleep many >>>> years ago. It is not possible to go back to usleep because it has >>>>> many problems with multithreads and interrupts. Please try >>>>> setting the following and reproducing the problem. >>>>> >>>>> wrapper.sleep_output=TRUE >>>>> >>>>> It will only give us new information if any nanosleep calls are >>>> failing for any reason however. Otherwise it will simply say when >>>> the calls start and stop. >>>>> >>>>> We are continuing to look into this. >>>>> >>>>> Cheers, >>>>> Leif >>>>> >>>>> >>>>> On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller >>>> <chr...@ta...> wrote: >>>>>> Christopher & Johannes, >>>>>> >>>>>> nanosleep is kind of very low-level system function, for which I >>>>>> can't see any problems like this to be common. >>>>>> >>>>>> There are some hooks in your mail I'd like to get closer, you said >>>>>> you are running a 2CPU instance and on vmware. So i'm wondering >>>>>> whether sth. with this configuration is not working as expected. >>>>>> >>>>>> could you please run "cat /proc/<pid>/status" on the machine where >>>>>> <pid> is the and send us also the output from there? >>>>>> Furthermore, is NTP activated on the host? >>>>>> >>>>>> I did some tests on our virtual machines but were not able to >>>>>> reproduce the behavior you described yet... >>>>>> >>>>>> Cheers, >>>>>> Christian >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson >>>>>> <lei...@ta...> wrote: >>>>>>> Christopher & Johannes, >>>>>>> Thank you for the detailed report. This is not a problem which I >>>>>>> am >>>> >>>>>>> aware of to date. We set up a 2 CPU VM today with cent OS and >>>>>>> are running some tests locally. So far we have not been able to >>>>>>> reproduce the problem. >>>>>>> >>>>>>>> From what you said, it sounds like this problem only happens on >>>>>>> shutdown, and can be reproduced easily? >>>>>>> >>>>>>> Could you please try and reproduce this with the following >>>>>>> settings included in the wrapper.conf and then send me the >>>>>>> resulting wrapper.log file directly? It would be a bit large for the list. >>>>>>> wrapper.debug=true >>>>>>> wrapper.state_output=TRUE >>>>>>> >>>>>>> It sounds like the nanosleep implementation is getting stuck for >>>>>>> some reason. It is interesting that an interrupt caused by a >>>>>>> signal >>>> >>>>>>> is waking it up. We do not currently have a way to change to >>>>>>> usleep without recompiling. >>>>>>> >>>>>>> Cheers, >>>>>>> Leif >>>>>>> >>>>>>> >>>>>>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor >>>> <ct...@co...> wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I'm forwarding this on behalf of Johannes, he seems to be having >>>>>>>> problems with his list subscription. I apologize if this is a >>>>>>>> double post, I forgot to add the subject on the first mail. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --- snip --- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit >>>>>>>> CentOS >>>>>>>> 5 in a VM with 2 CPUs. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Whenever I stopped a Java program with the wrapper (different >>>>>>>> configurations), the wrapper stopped pinging the JVM after some >>>>>>>> seconds (which eventually leads to the JVM ending itself because >>>>>>>> it >>>> >>>>>>>> does not receive ping packages from the wrapper any more). >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The last system call that never returns according to strace is >>>>>>>> always >>>>>>>> nanosleep: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> gettimeofday({1290172014, 465559}, NULL) = 0 >>>>>>>> >>>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource >>>>>>>> temporarily >>>>>>>> unavailable) >>>>>>>> >>>>>>>> gettimeofday({1290172014, 465610}, NULL) = 0 >>>>>>>> >>>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource >>>>>>>> temporarily >>>>>>>> unavailable) >>>>>>>> >>>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0 >>>>>>>> >>>>>>>> nanosleep({0, 100000000}, NULL) = 0 >>>>>>>> >>>>>>>> gettimeofday({1290172014, 566505}, NULL) = 0 >>>>>>>> >>>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource >>>>>>>> temporarily >>>>>>>> unavailable) >>>>>>>> >>>>>>>> gettimeofday({1290172014, 566552}, NULL) = 0 >>>>>>>> >>>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource >>>>>>>> temporarily >>>>>>>> unavailable) >>>>>>>> >>>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0 >>>>>>>> >>>>>>>> nanosleep({0, 100000000}, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> If I call pstack on the wrapper, you can see that two threads >>>>>>>> currently hang in the nanosleep method: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thread 2 (Thread -1210287216 (LWP 13115)): >>>>>>>> >>>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>>> >>>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from >>>>>>>> /lib/libpthread.so.0 >>>>>>>> >>>>>>>> #2 0x0805b0d0 in wrapperSleep () >>>>>>>> >>>>>>>> #3 0x0805b420 in timerRunner () >>>>>>>> >>>>>>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0 >>>>>>>> >>>>>>>> #5 0x00c7512e in clone () from /lib/libc.so.6 >>>>>>>> >>>>>>>> Thread 1 (Thread -1208169792 (LWP 13114)): >>>>>>>> >>>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>>> >>>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from >>>>>>>> /lib/libpthread.so.0 >>>>>>>> >>>>>>>> #2 0x0805b0d0 in wrapperSleep () >>>>>>>> >>>>>>>> #3 0x08059f6c in wrapperEventLoop () >>>>>>>> >>>>>>>> #4 0x08056628 in wrapperRunConsole () >>>>>>>> >>>>>>>> #5 0x0805cce3 in main () >>>>>>>> >>>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> If I send a signal to the wrapper, it reacts again, but before, >>>>>>>> it hangs forever in the nanosleep method. >>>>>>>> >>>>>>>> Is this a known problem of ServiceWrapper running with multiple >>>> CPUs? >>>>>>>> >>>>>>>> Do I have to recompile with usleep support instead or is there >>>>>>>> an option to always use usleep? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Best, |
|
From: Leif M. <le...@ta...> - 2010-12-01 03:04:33
|
Johannes, A few more questions and testing ideas. 1) What is the version of vmware ESX that you were using? 2) We have been looking into how nanosleep works on CentOS. All Linux platforms use the CLOCK_MONOTONIC clock rather than the CLOCK_REALTIME, so they should be fine. Even if they were using CLOCK_REALTIME, the nanosleep system call is supposed to take system time changes into account and always use a relative time. http://www.kernel.org/doc/man-pages/online/pages/man2/clock_settime.2.html BUT. There is a note at the bottom of the above page which says that each CPU in an SMP system could be using a different clock source. This can result in them being slightly out of sync and have a slightly different frequency. One problem that I have noticed with vmware in the past is that when you are not using ntp, the guest system time can drift drastically if the guest is being starved of CPU because of other processes on the host. I have seem drifts of several hours within a single day. The problem is that the Guest has no way of telling that it is behind. I am wondering if it is possible that one of your CPU clocks is drifting much more than the other. This is not something that could happen on a physical machine. Then if the Wrapper's threads are jumping between the two CPUs, that might explain why the time is changing. This is just a guess as we have not been able to reproduce this here. Here are some docs on timekeeping with vmware. There are several settings in ESX 4 to improve things: http://www.vmware.com/pdf/vmware_timekeeping.pdf http://www.fjc.net/linux/linux-and-vmware-related-issues/linux-2-6-kernels-and-vmware-clock-drift-issues 3) Along the lines above, I am wondering what happens if you restrict the Wrapper to run on a specific CPU rather than allowing it to balance between the two. This can be done with the taskset command. http://www.cyberciti.biz/faq/taskset-cpu-affinity-command/ Would it be possible for you to try this out to force the Wrapper to a single CPU? Thanks in advance, Leif On Tue, Nov 30, 2010 at 12:38 AM, Leif Mortenson <le...@ta...> wrote: > Johannes, > Thank you for the update on ntpd. Good to be able to remove that as a > possibility. > > This time difference is also 4398 seconds. >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake > > We will keep working on this on this end as well. > > Cheers, > Leif > > On Mon, Nov 29, 2010 at 11:08 PM, Johannes Nicolai <jni...@co...> wrote: >> Hi Leif, >> >> I shutdown ntpd and still get a similar error: >> >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process JVM output >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process socket >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger(2) >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process JVM output >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process socket >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger(2) >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED >> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep >> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake >> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: maintain logger >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: process JVM output >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: process socket >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: maintain logger(2) >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: handle wrapper state: STARTED >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: handle JVM state: STARTED >> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: sleep >> >> I will ask ops for the time difference between host and vm. >> >> Best, Johannes >> >> -----Original Message----- >> From: Leif Mortenson [mailto:le...@ta...] >> Sent: Monday, November 29, 2010 5:39 AM >> To: Johannes Nicolai >> Cc: wra...@li... >> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in wrapperSleep after some seconds >> >> Johannes, >> Thank you this latest log. It is showing that the system time that the Wrapper sees is being changed. The change is not happening when we are in a nanosleep so I don't think they are related. >> >> The first event shows a jump forwards of 4398 seconds: >> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms >> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake >> >> The second event ALSO shows a jump forwards of 4398 seconds: >> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process socket >> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: maintain logger(2) >> >> The fact that they are identical seems a little odd. I need to look >> into this a bit, but could you double check the system time of the host of your VM server. Is it possible that vmware is trying to sync your host and guest, at the same time that your ntp is doing the same. >> >> The Wrapper seems to be working fine when the time jumps forward, but nanosleep is getting stuck later when something else happens. I am wondering if that "something" is the return of the system time to its previous value. We are running some tests along these lines here locally. >> >> The 4398 seconds number is 0x112e. So it does not appear to be a significant number in and of itself. >> >> Cheers, >> Leif >> >> >> On Fri, Nov 26, 2010 at 8:48 PM, Johannes Nicolai <jni...@co...> wrote: >>> Hi Leif, >>> >>>>What is the timestamp in the lines after the "03:20:19" line? Does >>>>the >>> time go back immediately? >>> This is the last line in wrapper.log >>> >>> After this one, the wrapper starts to hang. >>> >>> I added wrapper.loop_output=true to my config and now see the >>> following last lines before the wrapper starts to hang: >>> >>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | >>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: process JVM output STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: handle wrapper >>> state: STARTED >>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state: >>> STARTED >>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | >>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: process JVM output STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26 >>> 03:43:30 | Loop: handle wrapper >>> state: STARTED >>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state: >>> STARTED >>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | >>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | >>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper | >>> 2010/11/26 03:43:31 | Sleep: awake STATUS | wrapper | 2010/11/26 >>> 03:43:31 | Loop: maintain logger STATUS | wrapper | 2010/11/26 >>> 03:43:31 | Loop: process JVM output STATUS | wrapper | 2010/11/26 >>> 03:43:31 | Loop: process socket STATUS | wrapper | 2010/11/26 >>> 04:56:49 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26 >>> 04:56:49 | Loop: handle wrapper >>> state: STARTED >>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: handle JVM state: >>> STARTED >>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: sleep STATUS | >>> wrapper | 2010/11/26 04:56:49 | Sleep: nanosleep 100ms STATUS | >>> wrapper | 2010/11/26 04:56:49 | Sleep: awake >>> >>> Current time on my machine is 03:45:12 >>> >>> Best, Johannes >>> >>> -----Original Message----- >>> From: Leif Mortenson [mailto:le...@ta...] >>> Sent: Friday, November 26, 2010 12:27 PM >>> To: Johannes Nicolai >>> Cc: Leif Mortenson; <wra...@li...> >>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in >>> wrapperSleep after some seconds >>> >>> Johannesburg, >>> That looks like a big clue. >>> >>> What is the timestamp in the lines after the "03:20:19" line? Does >>> the time go back immediately? >>> >>> To get more output, can you try also setting the following? >>> >>> Wrapper.loop_output=true >>> wrapper.timer_output=true >>> >>> I am on my mobile but I think those are correct. I am wondering if >>> the time is being set high, then back again. The logging code queries >>> the system time for each line. It immediately shows changes when >>> there are time adjustments. >>> >>> Something like ntp changes seems unlikely as it is always right after >>> the Wrapper starts though. >>> >>> - Leif >>> >>> >>> On 2010/11/26, at 19:22, "Johannes Nicolai" <jni...@co...> wrote: >>> >>>> Hi Leif, >>>> >>>> Yes, once the process starts to hang (after ten seconds to one >>> minute), both threads are always in the nanonsleep syscall. >>>> These are the last lines in wrapper.log: >>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms >>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake STATUS | >>>> wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 02:07:01 | Sleep: awake STATUS | wrapper | >>>> 2010/11/26 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper | >>>> 2010/11/26 02:07:01 | Sleep: awake STATUS | wrapper | 2010/11/26 >>>> 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26 >>>> 02:07:01 | Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 | >>>> Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26 02:07:01 | >>>> Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: >>>> nanosleep 100ms STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: >>>> awake >>>> >>>> The last output that differs from this pattern is DEBUG | wrapperp | >>>> 2010/11/26 02:06:39 | send a packet PING : ping STATUS | wrapper | >>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper | >>>> 2010/11/26 02:06:39 | Sleep: awake STATUS | wrapper | 2010/11/26 >>>> 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26 >>>> 02:06:39 | Sleep: awake INFO | jvm 1 | 2010/11/26 02:06:39 | >>>> WrapperManager Debug: >>> Received a packet PING : ping >>>> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: Send >>>> a >>> packet PING : ping >>>> DEBUG | wrapperp | 2010/11/26 02:06:39 | read a packet PING : ping >>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms >>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake STATUS | >>>> wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms STATUS | >>>> wrapper | 2010/11/26 02:06:39 | Sleep: awake STATUS | wrapper | >>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms >>>> >>>> What I find surprising is the huge jump in time from STATUS | wrapper >>>> | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms to STATUS | >>>> wrapper | 2010/11/26 03:20:19 | Sleep: awake >>>> >>>> If you look at the timestamps. The current date at this machine now >>>> is >>> 02:19:44, so the time for the last awake has not been reached by now. >>>> >>>> Best, Johannes >>>> >>>> -----Original Message----- >>>> From: le...@ta... [mailto:le...@ta...] On >>>> Behalf Of Leif Mortenson >>>> Sent: Friday, November 26, 2010 11:03 AM >>>> To: wra...@li... >>>> Cc: Johannes Nicolai >>>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in >>>> wrapperSleep after some seconds >>>> >>>> Johannes, >>>> When you get the hang, are you consistently seeing that both of the >>> threads are getting stuck in the nanosleep calls? >>>> >>>> We have a low level debug mode that can be enabled to show calls to >>> sleep. This was originally added to debug problems with usleep many >>> years ago. It is not possible to go back to usleep because it has >>>> many problems with multithreads and interrupts. Please try setting >>>> the following and reproducing the problem. >>>> >>>> wrapper.sleep_output=TRUE >>>> >>>> It will only give us new information if any nanosleep calls are >>> failing for any reason however. Otherwise it will simply say when the >>> calls start and stop. >>>> >>>> We are continuing to look into this. >>>> >>>> Cheers, >>>> Leif >>>> >>>> >>>> On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller >>> <chr...@ta...> wrote: >>>>> Christopher & Johannes, >>>>> >>>>> nanosleep is kind of very low-level system function, for which I >>>>> can't see any problems like this to be common. >>>>> >>>>> There are some hooks in your mail I'd like to get closer, you said >>>>> you are running a 2CPU instance and on vmware. So i'm wondering >>>>> whether sth. with this configuration is not working as expected. >>>>> >>>>> could you please run "cat /proc/<pid>/status" on the machine where >>>>> <pid> is the and send us also the output from there? >>>>> Furthermore, is NTP activated on the host? >>>>> >>>>> I did some tests on our virtual machines but were not able to >>>>> reproduce the behavior you described yet... >>>>> >>>>> Cheers, >>>>> Christian >>>>> >>>>> >>>>> >>>>> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson >>>>> <lei...@ta...> wrote: >>>>>> Christopher & Johannes, >>>>>> Thank you for the detailed report. This is not a problem which I >>>>>> am >>> >>>>>> aware of to date. We set up a 2 CPU VM today with cent OS and are >>>>>> running some tests locally. So far we have not been able to >>>>>> reproduce the problem. >>>>>> >>>>>>> From what you said, it sounds like this problem only happens on >>>>>> shutdown, and can be reproduced easily? >>>>>> >>>>>> Could you please try and reproduce this with the following settings >>>>>> included in the wrapper.conf and then send me the resulting >>>>>> wrapper.log file directly? It would be a bit large for the list. >>>>>> wrapper.debug=true >>>>>> wrapper.state_output=TRUE >>>>>> >>>>>> It sounds like the nanosleep implementation is getting stuck for >>>>>> some reason. It is interesting that an interrupt caused by a >>>>>> signal >>> >>>>>> is waking it up. We do not currently have a way to change to usleep >>>>>> without recompiling. >>>>>> >>>>>> Cheers, >>>>>> Leif >>>>>> >>>>>> >>>>>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor >>> <ct...@co...> wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I'm forwarding this on behalf of Johannes, he seems to be having >>>>>>> problems with his list subscription. I apologize if this is a >>>>>>> double post, I forgot to add the subject on the first mail. >>>>>>> >>>>>>> >>>>>>> >>>>>>> --- snip --- >>>>>>> >>>>>>> >>>>>>> >>>>>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit >>>>>>> CentOS >>>>>>> 5 in a VM with 2 CPUs. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Whenever I stopped a Java program with the wrapper (different >>>>>>> configurations), the wrapper stopped pinging the JVM after some >>>>>>> seconds (which eventually leads to the JVM ending itself because >>>>>>> it >>> >>>>>>> does not receive ping packages from the wrapper any more). >>>>>>> >>>>>>> >>>>>>> >>>>>>> The last system call that never returns according to strace is >>>>>>> always >>>>>>> nanosleep: >>>>>>> >>>>>>> >>>>>>> >>>>>>> gettimeofday({1290172014, 465559}, NULL) = 0 >>>>>>> >>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource >>>>>>> temporarily >>>>>>> unavailable) >>>>>>> >>>>>>> gettimeofday({1290172014, 465610}, NULL) = 0 >>>>>>> >>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource >>>>>>> temporarily >>>>>>> unavailable) >>>>>>> >>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0 >>>>>>> >>>>>>> nanosleep({0, 100000000}, NULL) = 0 >>>>>>> >>>>>>> gettimeofday({1290172014, 566505}, NULL) = 0 >>>>>>> >>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource >>>>>>> temporarily >>>>>>> unavailable) >>>>>>> >>>>>>> gettimeofday({1290172014, 566552}, NULL) = 0 >>>>>>> >>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource >>>>>>> temporarily >>>>>>> unavailable) >>>>>>> >>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0 >>>>>>> >>>>>>> nanosleep({0, 100000000}, >>>>>>> >>>>>>> >>>>>>> >>>>>>> If I call pstack on the wrapper, you can see that two threads >>>>>>> currently hang in the nanosleep method: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thread 2 (Thread -1210287216 (LWP 13115)): >>>>>>> >>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>> >>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from >>>>>>> /lib/libpthread.so.0 >>>>>>> >>>>>>> #2 0x0805b0d0 in wrapperSleep () >>>>>>> >>>>>>> #3 0x0805b420 in timerRunner () >>>>>>> >>>>>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0 >>>>>>> >>>>>>> #5 0x00c7512e in clone () from /lib/libc.so.6 >>>>>>> >>>>>>> Thread 1 (Thread -1208169792 (LWP 13114)): >>>>>>> >>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>> >>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from >>>>>>> /lib/libpthread.so.0 >>>>>>> >>>>>>> #2 0x0805b0d0 in wrapperSleep () >>>>>>> >>>>>>> #3 0x08059f6c in wrapperEventLoop () >>>>>>> >>>>>>> #4 0x08056628 in wrapperRunConsole () >>>>>>> >>>>>>> #5 0x0805cce3 in main () >>>>>>> >>>>>>> #0 0x00994410 in __kernel_vsyscall () >>>>>>> >>>>>>> >>>>>>> >>>>>>> If I send a signal to the wrapper, it reacts again, but before, it >>>>>>> hangs forever in the nanosleep method. >>>>>>> >>>>>>> Is this a known problem of ServiceWrapper running with multiple >>> CPUs? >>>>>>> >>>>>>> Do I have to recompile with usleep support instead or is there an >>>>>>> option to always use usleep? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Best, |
|
From: Leif M. <le...@ta...> - 2010-11-29 15:44:48
|
Johannes,
Thank you for the update on ntpd. Good to be able to remove that as a
possibility.
This time difference is also 4398 seconds.
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake
We will keep working on this on this end as well.
Cheers,
Leif
On Mon, Nov 29, 2010 at 11:08 PM, Johannes Nicolai <jni...@co...> wrote:
> Hi Leif,
>
> I shutdown ntpd and still get a similar error:
>
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process JVM output
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process socket
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: awake
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process JVM output
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: process socket
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle wrapper state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: handle JVM state: STARTED
> STATUS | wrapper | 2010/11/29 06:03:05 | Loop: sleep
> STATUS | wrapper | 2010/11/29 06:03:05 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake
> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/29 07:16:23 | Sleep: awake
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: maintain logger
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: process JVM output
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: process socket
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: handle wrapper state: STARTED
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: handle JVM state: STARTED
> STATUS | wrapper | 2010/11/29 07:16:23 | Loop: sleep
>
> I will ask ops for the time difference between host and vm.
>
> Best, Johannes
>
> -----Original Message-----
> From: Leif Mortenson [mailto:le...@ta...]
> Sent: Monday, November 29, 2010 5:39 AM
> To: Johannes Nicolai
> Cc: wra...@li...
> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in wrapperSleep after some seconds
>
> Johannes,
> Thank you this latest log. It is showing that the system time that the Wrapper sees is being changed. The change is not happening when we are in a nanosleep so I don't think they are related.
>
> The first event shows a jump forwards of 4398 seconds:
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
>
> The second event ALSO shows a jump forwards of 4398 seconds:
> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process socket
> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: maintain logger(2)
>
> The fact that they are identical seems a little odd. I need to look
> into this a bit, but could you double check the system time of the host of your VM server. Is it possible that vmware is trying to sync your host and guest, at the same time that your ntp is doing the same.
>
> The Wrapper seems to be working fine when the time jumps forward, but nanosleep is getting stuck later when something else happens. I am wondering if that "something" is the return of the system time to its previous value. We are running some tests along these lines here locally.
>
> The 4398 seconds number is 0x112e. So it does not appear to be a significant number in and of itself.
>
> Cheers,
> Leif
>
>
> On Fri, Nov 26, 2010 at 8:48 PM, Johannes Nicolai <jni...@co...> wrote:
>> Hi Leif,
>>
>>>What is the timestamp in the lines after the "03:20:19" line? Does
>>>the
>> time go back immediately?
>> This is the last line in wrapper.log
>>
>> After this one, the wrapper starts to hang.
>>
>> I added wrapper.loop_output=true to my config and now see the
>> following last lines before the wrapper starts to hang:
>>
>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper |
>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper |
>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: process JVM output STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: handle wrapper
>> state: STARTED
>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state:
>> STARTED
>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper |
>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper |
>> 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: maintain logger STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: process JVM output STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: process socket STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26
>> 03:43:30 | Loop: handle wrapper
>> state: STARTED
>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state:
>> STARTED
>> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS |
>> wrapper | 2010/11/26 03:43:30 | Sleep: awake STATUS | wrapper |
>> 2010/11/26 03:43:30 | Sleep: nanosleep 100ms STATUS | wrapper |
>> 2010/11/26 03:43:31 | Sleep: awake STATUS | wrapper | 2010/11/26
>> 03:43:31 | Loop: maintain logger STATUS | wrapper | 2010/11/26
>> 03:43:31 | Loop: process JVM output STATUS | wrapper | 2010/11/26
>> 03:43:31 | Loop: process socket STATUS | wrapper | 2010/11/26
>> 04:56:49 | Loop: maintain logger(2) STATUS | wrapper | 2010/11/26
>> 04:56:49 | Loop: handle wrapper
>> state: STARTED
>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: handle JVM state:
>> STARTED
>> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: sleep STATUS |
>> wrapper | 2010/11/26 04:56:49 | Sleep: nanosleep 100ms STATUS |
>> wrapper | 2010/11/26 04:56:49 | Sleep: awake
>>
>> Current time on my machine is 03:45:12
>>
>> Best, Johannes
>>
>> -----Original Message-----
>> From: Leif Mortenson [mailto:le...@ta...]
>> Sent: Friday, November 26, 2010 12:27 PM
>> To: Johannes Nicolai
>> Cc: Leif Mortenson; <wra...@li...>
>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in
>> wrapperSleep after some seconds
>>
>> Johannesburg,
>> That looks like a big clue.
>>
>> What is the timestamp in the lines after the "03:20:19" line? Does
>> the time go back immediately?
>>
>> To get more output, can you try also setting the following?
>>
>> Wrapper.loop_output=true
>> wrapper.timer_output=true
>>
>> I am on my mobile but I think those are correct. I am wondering if
>> the time is being set high, then back again. The logging code queries
>> the system time for each line. It immediately shows changes when
>> there are time adjustments.
>>
>> Something like ntp changes seems unlikely as it is always right after
>> the Wrapper starts though.
>>
>> - Leif
>>
>>
>> On 2010/11/26, at 19:22, "Johannes Nicolai" <jni...@co...> wrote:
>>
>>> Hi Leif,
>>>
>>> Yes, once the process starts to hang (after ten seconds to one
>> minute), both threads are always in the nanonsleep syscall.
>>> These are the last lines in wrapper.log:
>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake STATUS |
>>> wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms STATUS |
>>> wrapper | 2010/11/26 02:07:01 | Sleep: awake STATUS | wrapper |
>>> 2010/11/26 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper |
>>> 2010/11/26 02:07:01 | Sleep: awake STATUS | wrapper | 2010/11/26
>>> 02:07:01 | Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26
>>> 02:07:01 | Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 |
>>> Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26 02:07:01 |
>>> Sleep: awake STATUS | wrapper | 2010/11/26 02:07:01 | Sleep:
>>> nanosleep 100ms STATUS | wrapper | 2010/11/26 03:20:19 | Sleep:
>>> awake
>>>
>>> The last output that differs from this pattern is DEBUG | wrapperp |
>>> 2010/11/26 02:06:39 | send a packet PING : ping STATUS | wrapper |
>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper |
>>> 2010/11/26 02:06:39 | Sleep: awake STATUS | wrapper | 2010/11/26
>>> 02:06:39 | Sleep: nanosleep 100ms STATUS | wrapper | 2010/11/26
>>> 02:06:39 | Sleep: awake INFO | jvm 1 | 2010/11/26 02:06:39 |
>>> WrapperManager Debug:
>> Received a packet PING : ping
>>> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: Send
>>> a
>> packet PING : ping
>>> DEBUG | wrapperp | 2010/11/26 02:06:39 | read a packet PING : ping
>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake STATUS |
>>> wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms STATUS |
>>> wrapper | 2010/11/26 02:06:39 | Sleep: awake STATUS | wrapper |
>>> 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>>>
>>> What I find surprising is the huge jump in time from STATUS | wrapper
>>> | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms to STATUS |
>>> wrapper | 2010/11/26 03:20:19 | Sleep: awake
>>>
>>> If you look at the timestamps. The current date at this machine now
>>> is
>> 02:19:44, so the time for the last awake has not been reached by now.
>>>
>>> Best, Johannes
>>>
>>> -----Original Message-----
>>> From: le...@ta... [mailto:le...@ta...] On
>>> Behalf Of Leif Mortenson
>>> Sent: Friday, November 26, 2010 11:03 AM
>>> To: wra...@li...
>>> Cc: Johannes Nicolai
>>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in
>>> wrapperSleep after some seconds
>>>
>>> Johannes,
>>> When you get the hang, are you consistently seeing that both of the
>> threads are getting stuck in the nanosleep calls?
>>>
>>> We have a low level debug mode that can be enabled to show calls to
>> sleep. This was originally added to debug problems with usleep many
>> years ago. It is not possible to go back to usleep because it has
>>> many problems with multithreads and interrupts. Please try setting
>>> the following and reproducing the problem.
>>>
>>> wrapper.sleep_output=TRUE
>>>
>>> It will only give us new information if any nanosleep calls are
>> failing for any reason however. Otherwise it will simply say when the
>> calls start and stop.
>>>
>>> We are continuing to look into this.
>>>
>>> Cheers,
>>> Leif
>>>
>>>
>>> On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller
>> <chr...@ta...> wrote:
>>>> Christopher & Johannes,
>>>>
>>>> nanosleep is kind of very low-level system function, for which I
>>>> can't see any problems like this to be common.
>>>>
>>>> There are some hooks in your mail I'd like to get closer, you said
>>>> you are running a 2CPU instance and on vmware. So i'm wondering
>>>> whether sth. with this configuration is not working as expected.
>>>>
>>>> could you please run "cat /proc/<pid>/status" on the machine where
>>>> <pid> is the and send us also the output from there?
>>>> Furthermore, is NTP activated on the host?
>>>>
>>>> I did some tests on our virtual machines but were not able to
>>>> reproduce the behavior you described yet...
>>>>
>>>> Cheers,
>>>> Christian
>>>>
>>>>
>>>>
>>>> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson
>>>> <lei...@ta...> wrote:
>>>>> Christopher & Johannes,
>>>>> Thank you for the detailed report. This is not a problem which I
>>>>> am
>>
>>>>> aware of to date. We set up a 2 CPU VM today with cent OS and are
>>>>> running some tests locally. So far we have not been able to
>>>>> reproduce the problem.
>>>>>
>>>>>> From what you said, it sounds like this problem only happens on
>>>>> shutdown, and can be reproduced easily?
>>>>>
>>>>> Could you please try and reproduce this with the following settings
>>>>> included in the wrapper.conf and then send me the resulting
>>>>> wrapper.log file directly? It would be a bit large for the list.
>>>>> wrapper.debug=true
>>>>> wrapper.state_output=TRUE
>>>>>
>>>>> It sounds like the nanosleep implementation is getting stuck for
>>>>> some reason. It is interesting that an interrupt caused by a
>>>>> signal
>>
>>>>> is waking it up. We do not currently have a way to change to usleep
>>>>> without recompiling.
>>>>>
>>>>> Cheers,
>>>>> Leif
>>>>>
>>>>>
>>>>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor
>> <ct...@co...> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I'm forwarding this on behalf of Johannes, he seems to be having
>>>>>> problems with his list subscription. I apologize if this is a
>>>>>> double post, I forgot to add the subject on the first mail.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --- snip ---
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit
>>>>>> CentOS
>>>>>> 5 in a VM with 2 CPUs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Whenever I stopped a Java program with the wrapper (different
>>>>>> configurations), the wrapper stopped pinging the JVM after some
>>>>>> seconds (which eventually leads to the JVM ending itself because
>>>>>> it
>>
>>>>>> does not receive ping packages from the wrapper any more).
>>>>>>
>>>>>>
>>>>>>
>>>>>> The last system call that never returns according to strace is
>>>>>> always
>>>>>> nanosleep:
>>>>>>
>>>>>>
>>>>>>
>>>>>> gettimeofday({1290172014, 465559}, NULL) = 0
>>>>>>
>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>>>> temporarily
>>>>>> unavailable)
>>>>>>
>>>>>> gettimeofday({1290172014, 465610}, NULL) = 0
>>>>>>
>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>>>> temporarily
>>>>>> unavailable)
>>>>>>
>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>>>
>>>>>> nanosleep({0, 100000000}, NULL) = 0
>>>>>>
>>>>>> gettimeofday({1290172014, 566505}, NULL) = 0
>>>>>>
>>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>>>> temporarily
>>>>>> unavailable)
>>>>>>
>>>>>> gettimeofday({1290172014, 566552}, NULL) = 0
>>>>>>
>>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>>>> temporarily
>>>>>> unavailable)
>>>>>>
>>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>>>
>>>>>> nanosleep({0, 100000000},
>>>>>>
>>>>>>
>>>>>>
>>>>>> If I call pstack on the wrapper, you can see that two threads
>>>>>> currently hang in the nanosleep method:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thread 2 (Thread -1210287216 (LWP 13115)):
>>>>>>
>>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>>
>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from
>>>>>> /lib/libpthread.so.0
>>>>>>
>>>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>>>
>>>>>> #3 0x0805b420 in timerRunner ()
>>>>>>
>>>>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>>>>>>
>>>>>> #5 0x00c7512e in clone () from /lib/libc.so.6
>>>>>>
>>>>>> Thread 1 (Thread -1208169792 (LWP 13114)):
>>>>>>
>>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>>
>>>>>> #1 0x00d22506 in __nanosleep_nocancel () from
>>>>>> /lib/libpthread.so.0
>>>>>>
>>>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>>>
>>>>>> #3 0x08059f6c in wrapperEventLoop ()
>>>>>>
>>>>>> #4 0x08056628 in wrapperRunConsole ()
>>>>>>
>>>>>> #5 0x0805cce3 in main ()
>>>>>>
>>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>>
>>>>>>
>>>>>>
>>>>>> If I send a signal to the wrapper, it reacts again, but before, it
>>>>>> hangs forever in the nanosleep method.
>>>>>>
>>>>>> Is this a known problem of ServiceWrapper running with multiple
>> CPUs?
>>>>>>
>>>>>> Do I have to recompile with usleep support instead or is there an
>>>>>> option to always use usleep?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Best,
|
|
From: Leif M. <le...@ta...> - 2010-11-29 04:39:03
|
Johannes,
Thank you this latest log. It is showing that the system time that
the Wrapper sees is being changed. The change is not happening when
we are in a nanosleep so I don't think they are related.
The first event shows a jump forwards of 4398 seconds:
STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
The second event ALSO shows a jump forwards of 4398 seconds:
STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process socket
STATUS | wrapper | 2010/11/26 04:56:49 | Loop: maintain logger(2)
The fact that they are identical seems a little odd. I need to look
into this a bit, but could you double check the system time of the
host of your VM server. Is it possible that vmware is trying to sync
your host and guest, at the same time that your ntp is doing the same.
The Wrapper seems to be working fine when the time jumps forward, but
nanosleep is getting stuck later when something else happens. I am
wondering if that "something" is the return of the system time to its
previous value. We are running some tests along these lines here
locally.
The 4398 seconds number is 0x112e. So it does not appear to be a
significant number in and of itself.
Cheers,
Leif
On Fri, Nov 26, 2010 at 8:48 PM, Johannes Nicolai <jni...@co...> wrote:
> Hi Leif,
>
>>What is the timestamp in the lines after the "03:20:19" line? Does the
> time go back immediately?
> This is the last line in wrapper.log
>
> After this one, the wrapper starts to hang.
>
> I added wrapper.loop_output=true to my config and now see the following
> last lines before the wrapper starts to hang:
>
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: maintain logger
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: process JVM output
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: process socket
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle wrapper
> state: STARTED
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state:
> STARTED
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: maintain logger
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: process JVM output
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: process socket
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle wrapper
> state: STARTED
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: handle JVM state:
> STARTED
> STATUS | wrapper | 2010/11/26 03:43:30 | Loop: sleep
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:30 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:43:31 | Sleep: awake
> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: maintain logger
> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process JVM output
> STATUS | wrapper | 2010/11/26 03:43:31 | Loop: process socket
> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: maintain logger(2)
> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: handle wrapper
> state: STARTED
> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: handle JVM state:
> STARTED
> STATUS | wrapper | 2010/11/26 04:56:49 | Loop: sleep
> STATUS | wrapper | 2010/11/26 04:56:49 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 04:56:49 | Sleep: awake
>
> Current time on my machine is 03:45:12
>
> Best, Johannes
>
> -----Original Message-----
> From: Leif Mortenson [mailto:le...@ta...]
> Sent: Friday, November 26, 2010 12:27 PM
> To: Johannes Nicolai
> Cc: Leif Mortenson; <wra...@li...>
> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in
> wrapperSleep after some seconds
>
> Johannesburg,
> That looks like a big clue.
>
> What is the timestamp in the lines after the "03:20:19" line? Does the
> time go back immediately?
>
> To get more output, can you try also setting the following?
>
> Wrapper.loop_output=true
> wrapper.timer_output=true
>
> I am on my mobile but I think those are correct. I am wondering if the
> time is being set high, then back again. The logging code queries the
> system time for each line. It immediately shows changes when there are
> time adjustments.
>
> Something like ntp changes seems unlikely as it is always right after
> the Wrapper starts though.
>
> - Leif
>
>
> On 2010/11/26, at 19:22, "Johannes Nicolai" <jni...@co...> wrote:
>
>> Hi Leif,
>>
>> Yes, once the process starts to hang (after ten seconds to one
> minute), both threads are always in the nanonsleep syscall.
>> These are the last lines in wrapper.log:
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
>>
>> The last output that differs from this pattern is DEBUG | wrapperp |
>> 2010/11/26 02:06:39 | send a packet PING : ping
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
>> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug:
> Received a packet PING : ping
>> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: Send a
> packet PING : ping
>> DEBUG | wrapperp | 2010/11/26 02:06:39 | read a packet PING : ping
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
>> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>>
>> What I find surprising is the huge jump in time from
>> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
>> to
>> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
>>
>> If you look at the timestamps. The current date at this machine now is
> 02:19:44, so the time for the last awake has not been reached by now.
>>
>> Best, Johannes
>>
>> -----Original Message-----
>> From: le...@ta... [mailto:le...@ta...] On
>> Behalf Of Leif Mortenson
>> Sent: Friday, November 26, 2010 11:03 AM
>> To: wra...@li...
>> Cc: Johannes Nicolai
>> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in
>> wrapperSleep after some seconds
>>
>> Johannes,
>> When you get the hang, are you consistently seeing that both of the
> threads are getting stuck in the nanosleep calls?
>>
>> We have a low level debug mode that can be enabled to show calls to
> sleep. This was originally added to debug problems with usleep many
> years ago. It is not possible to go back to usleep because it has
>> many problems with multithreads and interrupts. Please try setting
>> the following and reproducing the problem.
>>
>> wrapper.sleep_output=TRUE
>>
>> It will only give us new information if any nanosleep calls are
> failing for any reason however. Otherwise it will simply say when the
> calls start and stop.
>>
>> We are continuing to look into this.
>>
>> Cheers,
>> Leif
>>
>>
>> On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller
> <chr...@ta...> wrote:
>>> Christopher & Johannes,
>>>
>>> nanosleep is kind of very low-level system function, for which I
>>> can't see any problems like this to be common.
>>>
>>> There are some hooks in your mail I'd like to get closer, you said
>>> you are running a 2CPU instance and on vmware. So i'm wondering
>>> whether sth. with this configuration is not working as expected.
>>>
>>> could you please run "cat /proc/<pid>/status" on the machine where
>>> <pid> is the and send us also the output from there?
>>> Furthermore, is NTP activated on the host?
>>>
>>> I did some tests on our virtual machines but were not able to
>>> reproduce the behavior you described yet...
>>>
>>> Cheers,
>>> Christian
>>>
>>>
>>>
>>> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson
>>> <lei...@ta...> wrote:
>>>> Christopher & Johannes,
>>>> Thank you for the detailed report. This is not a problem which I am
>
>>>> aware of to date. We set up a 2 CPU VM today with cent OS and are
>>>> running some tests locally. So far we have not been able to
>>>> reproduce the problem.
>>>>
>>>>> From what you said, it sounds like this problem only happens on
>>>> shutdown, and can be reproduced easily?
>>>>
>>>> Could you please try and reproduce this with the following settings
>>>> included in the wrapper.conf and then send me the resulting
>>>> wrapper.log file directly? It would be a bit large for the list.
>>>> wrapper.debug=true
>>>> wrapper.state_output=TRUE
>>>>
>>>> It sounds like the nanosleep implementation is getting stuck for
>>>> some reason. It is interesting that an interrupt caused by a signal
>
>>>> is waking it up. We do not currently have a way to change to usleep
>>>> without recompiling.
>>>>
>>>> Cheers,
>>>> Leif
>>>>
>>>>
>>>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor
> <ct...@co...> wrote:
>>>>> Hi all,
>>>>>
>>>>>
>>>>>
>>>>> I'm forwarding this on behalf of Johannes, he seems to be having
>>>>> problems with his list subscription. I apologize if this is a
>>>>> double post, I forgot to add the subject on the first mail.
>>>>>
>>>>>
>>>>>
>>>>> --- snip ---
>>>>>
>>>>>
>>>>>
>>>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS
>>>>> 5 in a VM with 2 CPUs.
>>>>>
>>>>>
>>>>>
>>>>> Whenever I stopped a Java program with the wrapper (different
>>>>> configurations), the wrapper stopped pinging the JVM after some
>>>>> seconds (which eventually leads to the JVM ending itself because it
>
>>>>> does not receive ping packages from the wrapper any more).
>>>>>
>>>>>
>>>>>
>>>>> The last system call that never returns according to strace is
>>>>> always
>>>>> nanosleep:
>>>>>
>>>>>
>>>>>
>>>>> gettimeofday({1290172014, 465559}, NULL) = 0
>>>>>
>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>>> temporarily
>>>>> unavailable)
>>>>>
>>>>> gettimeofday({1290172014, 465610}, NULL) = 0
>>>>>
>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>>> temporarily
>>>>> unavailable)
>>>>>
>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>>
>>>>> nanosleep({0, 100000000}, NULL) = 0
>>>>>
>>>>> gettimeofday({1290172014, 566505}, NULL) = 0
>>>>>
>>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>>> temporarily
>>>>> unavailable)
>>>>>
>>>>> gettimeofday({1290172014, 566552}, NULL) = 0
>>>>>
>>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>>> temporarily
>>>>> unavailable)
>>>>>
>>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>>
>>>>> nanosleep({0, 100000000},
>>>>>
>>>>>
>>>>>
>>>>> If I call pstack on the wrapper, you can see that two threads
>>>>> currently hang in the nanosleep method:
>>>>>
>>>>>
>>>>>
>>>>> Thread 2 (Thread -1210287216 (LWP 13115)):
>>>>>
>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>
>>>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>>>
>>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>>
>>>>> #3 0x0805b420 in timerRunner ()
>>>>>
>>>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>>>>>
>>>>> #5 0x00c7512e in clone () from /lib/libc.so.6
>>>>>
>>>>> Thread 1 (Thread -1208169792 (LWP 13114)):
>>>>>
>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>
>>>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>>>
>>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>>
>>>>> #3 0x08059f6c in wrapperEventLoop ()
>>>>>
>>>>> #4 0x08056628 in wrapperRunConsole ()
>>>>>
>>>>> #5 0x0805cce3 in main ()
>>>>>
>>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>>
>>>>>
>>>>>
>>>>> If I send a signal to the wrapper, it reacts again, but before, it
>>>>> hangs forever in the nanosleep method.
>>>>>
>>>>> Is this a known problem of ServiceWrapper running with multiple
> CPUs?
>>>>>
>>>>> Do I have to recompile with usleep support instead or is there an
>>>>> option to always use usleep?
>>>>>
>>>>>
>>>>>
>>>>> Best,
|
|
From: Leif M. <le...@ta...> - 2010-11-26 11:52:18
|
Johannesburg,
That looks like a big clue.
What is the timestamp in the lines after the "03:20:19" line? Does the time go back immediately?
To get more output, can you try also setting the following?
Wrapper.loop_output=true
wrapper.timer_output=true
I am on my mobile but I think those are correct. I am wondering if the time is being set high, then back again. The logging code queries the system time for each line. It immediately shows changes when there are time adjustments.
Something like ntp changes seems unlikely as it is always right after the Wrapper starts though.
- Leif
On 2010/11/26, at 19:22, "Johannes Nicolai" <jni...@co...> wrote:
> Hi Leif,
>
> Yes, once the process starts to hang (after ten seconds to one minute), both threads are always in the nanonsleep syscall.
> These are the last lines in wrapper.log:
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
>
> The last output that differs from this pattern is
> DEBUG | wrapperp | 2010/11/26 02:06:39 | send a packet PING : ping
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: Received a packet PING : ping
> INFO | jvm 1 | 2010/11/26 02:06:39 | WrapperManager Debug: Send a packet PING : ping
> DEBUG | wrapperp | 2010/11/26 02:06:39 | read a packet PING : ping
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: awake
> STATUS | wrapper | 2010/11/26 02:06:39 | Sleep: nanosleep 100ms
>
> What I find surprising is the huge jump in time from
> STATUS | wrapper | 2010/11/26 02:07:01 | Sleep: nanosleep 100ms
> to
> STATUS | wrapper | 2010/11/26 03:20:19 | Sleep: awake
>
> If you look at the timestamps. The current date at this machine now is 02:19:44, so the time for the last awake has not been reached by now.
>
> Best, Johannes
>
> -----Original Message-----
> From: le...@ta... [mailto:le...@ta...] On Behalf Of Leif Mortenson
> Sent: Friday, November 26, 2010 11:03 AM
> To: wra...@li...
> Cc: Johannes Nicolai
> Subject: Re: [Wrapper-user] ServiceWrapper starts to hang in wrapperSleep after some seconds
>
> Johannes,
> When you get the hang, are you consistently seeing that both of the threads are getting stuck in the nanosleep calls?
>
> We have a low level debug mode that can be enabled to show calls to sleep. This was originally added to debug problems with usleep many years ago. It is not possible to go back to usleep because it has
> many problems with multithreads and interrupts. Please try setting
> the following and reproducing the problem.
>
> wrapper.sleep_output=TRUE
>
> It will only give us new information if any nanosleep calls are failing for any reason however. Otherwise it will simply say when the calls start and stop.
>
> We are continuing to look into this.
>
> Cheers,
> Leif
>
>
> On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller <chr...@ta...> wrote:
>> Christopher & Johannes,
>>
>> nanosleep is kind of very low-level system function, for which I can't
>> see any problems like this to be common.
>>
>> There are some hooks in your mail I'd like to get closer, you said you
>> are running a 2CPU instance and on vmware. So i'm wondering whether
>> sth. with this configuration is not working as expected.
>>
>> could you please run "cat /proc/<pid>/status" on the machine where
>> <pid> is the and send us also the output from there?
>> Furthermore, is NTP activated on the host?
>>
>> I did some tests on our virtual machines but were not able to
>> reproduce the behavior you described yet...
>>
>> Cheers,
>> Christian
>>
>>
>>
>> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson
>> <lei...@ta...> wrote:
>>> Christopher & Johannes,
>>> Thank you for the detailed report. This is not a problem which I am
>>> aware of to date. We set up a 2 CPU VM today with cent OS and are
>>> running some tests locally. So far we have not been able to
>>> reproduce the problem.
>>>
>>>> From what you said, it sounds like this problem only happens on
>>> shutdown, and can be reproduced easily?
>>>
>>> Could you please try and reproduce this with the following settings
>>> included in the wrapper.conf and then send me the resulting
>>> wrapper.log file directly? It would be a bit large for the list.
>>> wrapper.debug=true
>>> wrapper.state_output=TRUE
>>>
>>> It sounds like the nanosleep implementation is getting stuck for some
>>> reason. It is interesting that an interrupt caused by a signal is
>>> waking it up. We do not currently have a way to change to usleep
>>> without recompiling.
>>>
>>> Cheers,
>>> Leif
>>>
>>>
>>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor <ct...@co...> wrote:
>>>> Hi all,
>>>>
>>>>
>>>>
>>>> I'm forwarding this on behalf of Johannes, he seems to be having
>>>> problems with his list subscription. I apologize if this is a double
>>>> post, I forgot to add the subject on the first mail.
>>>>
>>>>
>>>>
>>>> --- snip ---
>>>>
>>>>
>>>>
>>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS
>>>> 5 in a VM with 2 CPUs.
>>>>
>>>>
>>>>
>>>> Whenever I stopped a Java program with the wrapper (different
>>>> configurations), the wrapper stopped pinging the JVM after some
>>>> seconds (which eventually leads to the JVM ending itself because it
>>>> does not receive ping packages from the wrapper any more).
>>>>
>>>>
>>>>
>>>> The last system call that never returns according to strace is
>>>> always
>>>> nanosleep:
>>>>
>>>>
>>>>
>>>> gettimeofday({1290172014, 465559}, NULL) = 0
>>>>
>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>> temporarily
>>>> unavailable)
>>>>
>>>> gettimeofday({1290172014, 465610}, NULL) = 0
>>>>
>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>> temporarily
>>>> unavailable)
>>>>
>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>
>>>> nanosleep({0, 100000000}, NULL) = 0
>>>>
>>>> gettimeofday({1290172014, 566505}, NULL) = 0
>>>>
>>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
>>>> temporarily
>>>> unavailable)
>>>>
>>>> gettimeofday({1290172014, 566552}, NULL) = 0
>>>>
>>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
>>>> temporarily
>>>> unavailable)
>>>>
>>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>>
>>>> nanosleep({0, 100000000},
>>>>
>>>>
>>>>
>>>> If I call pstack on the wrapper, you can see that two threads
>>>> currently hang in the nanosleep method:
>>>>
>>>>
>>>>
>>>> Thread 2 (Thread -1210287216 (LWP 13115)):
>>>>
>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>
>>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>>
>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>
>>>> #3 0x0805b420 in timerRunner ()
>>>>
>>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>>>>
>>>> #5 0x00c7512e in clone () from /lib/libc.so.6
>>>>
>>>> Thread 1 (Thread -1208169792 (LWP 13114)):
>>>>
>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>
>>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>>
>>>> #2 0x0805b0d0 in wrapperSleep ()
>>>>
>>>> #3 0x08059f6c in wrapperEventLoop ()
>>>>
>>>> #4 0x08056628 in wrapperRunConsole ()
>>>>
>>>> #5 0x0805cce3 in main ()
>>>>
>>>> #0 0x00994410 in __kernel_vsyscall ()
>>>>
>>>>
>>>>
>>>> If I send a signal to the wrapper, it reacts again, but before, it
>>>> hangs forever in the nanosleep method.
>>>>
>>>> Is this a known problem of ServiceWrapper running with multiple CPUs?
>>>>
>>>> Do I have to recompile with usleep support instead or is there an
>>>> option to always use usleep?
>>>>
>>>>
>>>>
>>>> Best,
|
|
From: Leif M. <lei...@ta...> - 2010-11-26 10:02:52
|
Johannes,
When you get the hang, are you consistently seeing that both of the
threads are getting stuck in the nanosleep calls?
We have a low level debug mode that can be enabled to show calls to
sleep. This was originally added to debug problems with usleep many
years ago. It is not possible to go back to usleep because it has
many problems with multithreads and interrupts. Please try setting
the following and reproducing the problem.
wrapper.sleep_output=TRUE
It will only give us new information if any nanosleep calls are
failing for any reason however. Otherwise it will simply say when the
calls start and stop.
We are continuing to look into this.
Cheers,
Leif
On Fri, Nov 26, 2010 at 11:58 AM, Christian Mueller
<chr...@ta...> wrote:
> Christopher & Johannes,
>
> nanosleep is kind of very low-level system function, for which I can't
> see any problems like this to be common.
>
> There are some hooks in your mail I'd like to get closer, you said you
> are running a 2CPU instance and on vmware. So i'm wondering whether
> sth. with this configuration is not working as expected.
>
> could you please run "cat /proc/<pid>/status" on the machine where
> <pid> is the and send us also the output from there?
> Furthermore, is NTP activated on the host?
>
> I did some tests on our virtual machines but were not able to
> reproduce the behavior you described yet...
>
> Cheers,
> Christian
>
>
>
> On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson
> <lei...@ta...> wrote:
>> Christopher & Johannes,
>> Thank you for the detailed report. This is not a problem which I am
>> aware of to date. We set up a 2 CPU VM today with cent OS and are
>> running some tests locally. So far we have not been able to reproduce
>> the problem.
>>
>> >From what you said, it sounds like this problem only happens on
>> shutdown, and can be reproduced easily?
>>
>> Could you please try and reproduce this with the following settings
>> included in the wrapper.conf and then send me the resulting
>> wrapper.log file directly? It would be a bit large for the list.
>> wrapper.debug=true
>> wrapper.state_output=TRUE
>>
>> It sounds like the nanosleep implementation is getting stuck for some
>> reason. It is interesting that an interrupt caused by a signal is
>> waking it up. We do not currently have a way to change to usleep
>> without recompiling.
>>
>> Cheers,
>> Leif
>>
>>
>> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor <ct...@co...> wrote:
>>> Hi all,
>>>
>>>
>>>
>>> I’m forwarding this on behalf of Johannes, he seems to be having problems
>>> with his list subscription. I apologize if this is a double post, I forgot
>>> to add the subject on the first mail.
>>>
>>>
>>>
>>> --- snip ---
>>>
>>>
>>>
>>> I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS 5 in a
>>> VM with 2 CPUs.
>>>
>>>
>>>
>>> Whenever I stopped a Java program with the wrapper (different
>>> configurations), the wrapper stopped pinging the JVM after some seconds
>>> (which eventually leads to the JVM ending itself because it does not receive
>>> ping packages from the wrapper any more).
>>>
>>>
>>>
>>> The last system call that never returns according to strace is always
>>> nanosleep:
>>>
>>>
>>>
>>> gettimeofday({1290172014, 465559}, NULL) = 0
>>>
>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
>>> unavailable)
>>>
>>> gettimeofday({1290172014, 465610}, NULL) = 0
>>>
>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
>>> unavailable)
>>>
>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>
>>> nanosleep({0, 100000000}, NULL) = 0
>>>
>>> gettimeofday({1290172014, 566505}, NULL) = 0
>>>
>>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
>>> unavailable)
>>>
>>> gettimeofday({1290172014, 566552}, NULL) = 0
>>>
>>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
>>> unavailable)
>>>
>>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>>
>>> nanosleep({0, 100000000},
>>>
>>>
>>>
>>> If I call pstack on the wrapper, you can see that two threads currently hang
>>> in the nanosleep method:
>>>
>>>
>>>
>>> Thread 2 (Thread -1210287216 (LWP 13115)):
>>>
>>> #0 0x00994410 in __kernel_vsyscall ()
>>>
>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>
>>> #2 0x0805b0d0 in wrapperSleep ()
>>>
>>> #3 0x0805b420 in timerRunner ()
>>>
>>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>>>
>>> #5 0x00c7512e in clone () from /lib/libc.so.6
>>>
>>> Thread 1 (Thread -1208169792 (LWP 13114)):
>>>
>>> #0 0x00994410 in __kernel_vsyscall ()
>>>
>>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>>
>>> #2 0x0805b0d0 in wrapperSleep ()
>>>
>>> #3 0x08059f6c in wrapperEventLoop ()
>>>
>>> #4 0x08056628 in wrapperRunConsole ()
>>>
>>> #5 0x0805cce3 in main ()
>>>
>>> #0 0x00994410 in __kernel_vsyscall ()
>>>
>>>
>>>
>>> If I send a signal to the wrapper, it reacts again, but before, it hangs
>>> forever in the nanosleep method.
>>>
>>> Is this a known problem of ServiceWrapper running with multiple CPUs?
>>>
>>> Do I have to recompile with usleep support instead or is there an option to
>>> always use usleep?
>>>
>>>
>>>
>>> Best,
|
|
From: Christian M. <chr...@ta...> - 2010-11-26 03:25:04
|
Christopher & Johannes,
nanosleep is kind of very low-level system function, for which I can't
see any problems like this to be common.
There are some hooks in your mail I'd like to get closer, you said you
are running a 2CPU instance and on vmware. So i'm wondering whether
sth. with this configuration is not working as expected.
could you please run "cat /proc/<pid>/status" on the machine where
<pid> is the and send us also the output from there?
Furthermore, is NTP activated on the host?
I did some tests on our virtual machines but were not able to
reproduce the behavior you described yet...
Cheers,
Christian
On Wed, Nov 24, 2010 at 5:26 PM, Leif Mortenson
<lei...@ta...> wrote:
> Christopher & Johannes,
> Thank you for the detailed report. This is not a problem which I am
> aware of to date. We set up a 2 CPU VM today with cent OS and are
> running some tests locally. So far we have not been able to reproduce
> the problem.
>
> >From what you said, it sounds like this problem only happens on
> shutdown, and can be reproduced easily?
>
> Could you please try and reproduce this with the following settings
> included in the wrapper.conf and then send me the resulting
> wrapper.log file directly? It would be a bit large for the list.
> wrapper.debug=true
> wrapper.state_output=TRUE
>
> It sounds like the nanosleep implementation is getting stuck for some
> reason. It is interesting that an interrupt caused by a signal is
> waking it up. We do not currently have a way to change to usleep
> without recompiling.
>
> Cheers,
> Leif
>
>
> On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor <ct...@co...> wrote:
>> Hi all,
>>
>>
>>
>> I’m forwarding this on behalf of Johannes, he seems to be having problems
>> with his list subscription. I apologize if this is a double post, I forgot
>> to add the subject on the first mail.
>>
>>
>>
>> --- snip ---
>>
>>
>>
>> I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS 5 in a
>> VM with 2 CPUs.
>>
>>
>>
>> Whenever I stopped a Java program with the wrapper (different
>> configurations), the wrapper stopped pinging the JVM after some seconds
>> (which eventually leads to the JVM ending itself because it does not receive
>> ping packages from the wrapper any more).
>>
>>
>>
>> The last system call that never returns according to strace is always
>> nanosleep:
>>
>>
>>
>> gettimeofday({1290172014, 465559}, NULL) = 0
>>
>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
>> unavailable)
>>
>> gettimeofday({1290172014, 465610}, NULL) = 0
>>
>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
>> unavailable)
>>
>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>
>> nanosleep({0, 100000000}, NULL) = 0
>>
>> gettimeofday({1290172014, 566505}, NULL) = 0
>>
>> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
>> unavailable)
>>
>> gettimeofday({1290172014, 566552}, NULL) = 0
>>
>> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
>> unavailable)
>>
>> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>>
>> nanosleep({0, 100000000},
>>
>>
>>
>> If I call pstack on the wrapper, you can see that two threads currently hang
>> in the nanosleep method:
>>
>>
>>
>> Thread 2 (Thread -1210287216 (LWP 13115)):
>>
>> #0 0x00994410 in __kernel_vsyscall ()
>>
>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>
>> #2 0x0805b0d0 in wrapperSleep ()
>>
>> #3 0x0805b420 in timerRunner ()
>>
>> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>>
>> #5 0x00c7512e in clone () from /lib/libc.so.6
>>
>> Thread 1 (Thread -1208169792 (LWP 13114)):
>>
>> #0 0x00994410 in __kernel_vsyscall ()
>>
>> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>>
>> #2 0x0805b0d0 in wrapperSleep ()
>>
>> #3 0x08059f6c in wrapperEventLoop ()
>>
>> #4 0x08056628 in wrapperRunConsole ()
>>
>> #5 0x0805cce3 in main ()
>>
>> #0 0x00994410 in __kernel_vsyscall ()
>>
>>
>>
>> If I send a signal to the wrapper, it reacts again, but before, it hangs
>> forever in the nanosleep method.
>>
>> Is this a known problem of ServiceWrapper running with multiple CPUs?
>>
>> Do I have to recompile with usleep support instead or is there an option to
>> always use usleep?
>>
>>
>>
>> Best,
>>
>> --Christopher (on behalf of Johannes)
>
> ------------------------------------------------------------------------------
> Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
> Tap into the largest installed PC base & get more eyes on your game by
> optimizing for Intel(R) Graphics Technology. Get started today with the
> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
> http://p.sf.net/sfu/intelisp-dev2dev
> _______________________________________________
> Wrapper-user mailing list
> Wra...@li...
> https://lists.sourceforge.net/lists/listinfo/wrapper-user
>
|
|
From: Michael B. <mic...@we...> - 2010-11-25 14:37:05
|
Hi Leif! Thanks for the hint with the -c switch, I noticed the sample scripts that come with the wrapper distribution (StartTestWrapper-NT) use the -t switch by default, and that was the reason for it running in session 0. It now works as expected. Thanks you very much, Michael Am 25.11.2010 um 11:57 schrieb Leif Mortenson: Michael, If you launch the Wrapper in console mode then it should be in the current desktop. bin/wrapper.exe -c ../conf/wrapper.conf Please let me know how this works for you. Cheers, Leif 2010/11/25 Michael Böckling <mic...@we...<mailto:mic...@we...>>: Hi Leif, thanks for the reply. I m aware of this situation, thats why I use a workaround. I have an interactive user auto-logon on bootup and start the app in that user's session (as a regular process, not as a service). I simply would like to use the configuration model and the features of the wrapper (e.g. jvm auto-relaunch on OutOfMemoryException), I don't want to start my app as a service, I gave that up. Is there a way to do that? Even the normal start/stop scripts seem to launch my app in session 0, not just the install-as-service stuff. Regards, Michael Am 25.11.2010 um 04:02 schrieb Leif Mortenson: Michael, Unfortunately, Microsoft changed the way Windows handles Interactive services to improve security. It is no longer possible for a process running as a service to communicate with any desktop other than the session 0 desktop. We are working on a way to work around this but it is going to involve multiple processes and will NOT allow a normal Java application running as a service which has a GUI to display its GUI directly. This is still a little ways off however. It will involve launching secondary Wrapper and optionally Java processes in the individual desktop's user space that will interact with each desktop. These child processes will then communicate back to the central Wrapper and or JVM that is always running as a service. This is still in progress, so I would appreciate any feedback from the community on what they need in this area. Cheers, Leif 2010/11/25 Michael Böckling <mic...@we...<mailto:mic...@we...>>: Hi! I have a special case where I would like to run a java "service" in an interactive session (session 1) on a windows 7 box. Is this possible? Basically: how to use the wrapper without running the JVM in session 0. Any help is welcome. Regards, Michael Michael Böckling Westernacher Products & Services AG ------------------------------------------------------------------------------ Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! Tap into the largest installed PC base & get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev _______________________________________________ Wrapper-user mailing list Wra...@li... https://lists.sourceforge.net/lists/listinfo/wrapper-user Michael Böckling Westernacher Products & Services AG Telefon: +49 (0)711 72 25 95 33 E-Mail: mic...@we...<mailto:mic...@we...> Innovating Business & IT Westernacher Products & Services AG Königstraße 26 · 70173 Stuttgart Telefon: +49 711 72 25 95 0 Telefax: +49 711 72 25 95 99 Internet: www.westernacher.com<http://www.westernacher.com/> Amtsgericht Stuttgart HRB 733577 Vorstandsvorsitzender: Walter H. Büttner Vorstand: Dr. Michael Schäfer Vorsitzender des Aufsichtsrates: Dr.-Ing. Bernd Bergner Aufsichtsrat: Dr.-Ing. Martin Weiss M.S., Tino Jezewski Vertraulichkeitsklausel: Diese E-Mail enthält vertrauliche oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie die Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder deren Inhalt ist nicht gestattet. |
|
From: Leif M. <lei...@ta...> - 2010-11-25 10:57:52
|
Michael, If you launch the Wrapper in console mode then it should be in the current desktop. bin/wrapper.exe -c ../conf/wrapper.conf Please let me know how this works for you. Cheers, Leif 2010/11/25 Michael Böckling <mic...@we...>: > Hi Leif, > thanks for the reply. I m aware of this situation, thats why I use a > workaround. I have an interactive user auto-logon on bootup and start the > app in that user's session (as a regular process, not as a service). > I simply would like to use the configuration model and the features of the > wrapper (e.g. jvm auto-relaunch on OutOfMemoryException), I don't want to > start my app as a service, I gave that up. Is there a way to do that? Even > the normal start/stop scripts seem to launch my app in session 0, not just > the install-as-service stuff. > Regards, > Michael > > Am 25.11.2010 um 04:02 schrieb Leif Mortenson: > > Michael, > Unfortunately, Microsoft changed the way Windows handles Interactive > services to improve security. It is no longer possible for a process > running as a service to communicate with any desktop other than the > session 0 desktop. > > We are working on a way to work around this but it is going to involve > multiple processes and will NOT allow a normal Java application > running as a service which has a GUI to display its GUI directly. > This is still a little ways off however. It will involve launching > secondary Wrapper and optionally Java processes in the individual > desktop's user space that will interact with each desktop. These > child processes will then communicate back to the central Wrapper and > or JVM that is always running as a service. > > This is still in progress, so I would appreciate any feedback from the > community on what they need in this area. > > Cheers, > Leif > > 2010/11/25 Michael Böckling <mic...@we...>: > > Hi! > > I have a special case where I would like to run a java "service" in an > > interactive session (session 1) on a windows 7 box. Is this possible? > > Basically: how to use the wrapper without running the JVM in session 0. > > Any help is welcome. > > Regards, > > Michael > > Michael Böckling > > Westernacher Products & Services AG |
|
From: Michael B. <mic...@we...> - 2010-11-25 09:34:17
|
Hi Leif, thanks for the reply. I m aware of this situation, thats why I use a workaround. I have an interactive user auto-logon on bootup and start the app in that user's session (as a regular process, not as a service). I simply would like to use the configuration model and the features of the wrapper (e.g. jvm auto-relaunch on OutOfMemoryException), I don't want to start my app as a service, I gave that up. Is there a way to do that? Even the normal start/stop scripts seem to launch my app in session 0, not just the install-as-service stuff. Regards, Michael Am 25.11.2010 um 04:02 schrieb Leif Mortenson: Michael, Unfortunately, Microsoft changed the way Windows handles Interactive services to improve security. It is no longer possible for a process running as a service to communicate with any desktop other than the session 0 desktop. We are working on a way to work around this but it is going to involve multiple processes and will NOT allow a normal Java application running as a service which has a GUI to display its GUI directly. This is still a little ways off however. It will involve launching secondary Wrapper and optionally Java processes in the individual desktop's user space that will interact with each desktop. These child processes will then communicate back to the central Wrapper and or JVM that is always running as a service. This is still in progress, so I would appreciate any feedback from the community on what they need in this area. Cheers, Leif 2010/11/25 Michael Böckling <mic...@we...<mailto:mic...@we...>>: Hi! I have a special case where I would like to run a java "service" in an interactive session (session 1) on a windows 7 box. Is this possible? Basically: how to use the wrapper without running the JVM in session 0. Any help is welcome. Regards, Michael Michael Böckling Westernacher Products & Services AG ------------------------------------------------------------------------------ Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! Tap into the largest installed PC base & get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev _______________________________________________ Wrapper-user mailing list Wra...@li... https://lists.sourceforge.net/lists/listinfo/wrapper-user Michael Böckling Westernacher Products & Services AG Telefon: +49 (0)711 72 25 95 33 E-Mail: mic...@we...<mailto:mic...@we...> Innovating Business & IT Westernacher Products & Services AG Königstraße 26 · 70173 Stuttgart Telefon: +49 711 72 25 95 0 Telefax: +49 711 72 25 95 99 Internet: www.westernacher.com<http://www.westernacher.com/> Amtsgericht Stuttgart HRB 733577 Vorstandsvorsitzender: Walter H. Büttner Vorstand: Dr. Michael Schäfer Vorsitzender des Aufsichtsrates: Dr.-Ing. Bernd Bergner Aufsichtsrat: Dr.-Ing. Martin Weiss M.S., Tino Jezewski Vertraulichkeitsklausel: Diese E-Mail enthält vertrauliche oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie die Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder deren Inhalt ist nicht gestattet. |
|
From: Leif M. <lei...@ta...> - 2010-11-25 03:02:23
|
Michael, Unfortunately, Microsoft changed the way Windows handles Interactive services to improve security. It is no longer possible for a process running as a service to communicate with any desktop other than the session 0 desktop. We are working on a way to work around this but it is going to involve multiple processes and will NOT allow a normal Java application running as a service which has a GUI to display its GUI directly. This is still a little ways off however. It will involve launching secondary Wrapper and optionally Java processes in the individual desktop's user space that will interact with each desktop. These child processes will then communicate back to the central Wrapper and or JVM that is always running as a service. This is still in progress, so I would appreciate any feedback from the community on what they need in this area. Cheers, Leif 2010/11/25 Michael Böckling <mic...@we...>: > Hi! > I have a special case where I would like to run a java "service" in an > interactive session (session 1) on a windows 7 box. Is this possible? > Basically: how to use the wrapper without running the JVM in session 0. > Any help is welcome. > Regards, > Michael > > Michael Böckling > Westernacher Products & Services AG |
|
From: Michael B. <mic...@we...> - 2010-11-24 16:58:49
|
Hi! I have a special case where I would like to run a java "service" in an interactive session (session 1) on a windows 7 box. Is this possible? Basically: how to use the wrapper without running the JVM in session 0. Any help is welcome. Regards, Michael Michael Böckling Westernacher Products & Services AG Telefon: +49 (0)711 72 25 95 33 E-Mail: mic...@we...<mailto:mic...@we...> Innovating Business & IT Westernacher Products & Services AG Königstraße 26 · 70173 Stuttgart Telefon: +49 711 72 25 95 0 Telefax: +49 711 72 25 95 99 Internet: www.westernacher.com<http://www.westernacher.com/> Amtsgericht Stuttgart HRB 733577 Vorstandsvorsitzender: Walter H. Büttner Vorstand: Dr. Michael Schäfer Vorsitzender des Aufsichtsrates: Dr.-Ing. Bernd Bergner Aufsichtsrat: Dr.-Ing. Martin Weiss M.S., Tino Jezewski Vertraulichkeitsklausel: Diese E-Mail enthält vertrauliche oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie die Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder deren Inhalt ist nicht gestattet. |
|
From: Leif M. <lei...@ta...> - 2010-11-24 08:27:04
|
Christopher & Johannes,
Thank you for the detailed report. This is not a problem which I am
aware of to date. We set up a 2 CPU VM today with cent OS and are
running some tests locally. So far we have not been able to reproduce
the problem.
>From what you said, it sounds like this problem only happens on
shutdown, and can be reproduced easily?
Could you please try and reproduce this with the following settings
included in the wrapper.conf and then send me the resulting
wrapper.log file directly? It would be a bit large for the list.
wrapper.debug=true
wrapper.state_output=TRUE
It sounds like the nanosleep implementation is getting stuck for some
reason. It is interesting that an interrupt caused by a signal is
waking it up. We do not currently have a way to change to usleep
without recompiling.
Cheers,
Leif
On Tue, Nov 23, 2010 at 7:31 PM, Christopher Taylor <ct...@co...> wrote:
> Hi all,
>
>
>
> I’m forwarding this on behalf of Johannes, he seems to be having problems
> with his list subscription. I apologize if this is a double post, I forgot
> to add the subject on the first mail.
>
>
>
> --- snip ---
>
>
>
> I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS 5 in a
> VM with 2 CPUs.
>
>
>
> Whenever I stopped a Java program with the wrapper (different
> configurations), the wrapper stopped pinging the JVM after some seconds
> (which eventually leads to the JVM ending itself because it does not receive
> ping packages from the wrapper any more).
>
>
>
> The last system call that never returns according to strace is always
> nanosleep:
>
>
>
> gettimeofday({1290172014, 465559}, NULL) = 0
>
> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
> unavailable)
>
> gettimeofday({1290172014, 465610}, NULL) = 0
>
> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
> unavailable)
>
> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>
> nanosleep({0, 100000000}, NULL) = 0
>
> gettimeofday({1290172014, 566505}, NULL) = 0
>
> read(5, 0x8895070, 1024) = -1 EAGAIN (Resource temporarily
> unavailable)
>
> gettimeofday({1290172014, 566552}, NULL) = 0
>
> recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource temporarily
> unavailable)
>
> waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
>
> nanosleep({0, 100000000},
>
>
>
> If I call pstack on the wrapper, you can see that two threads currently hang
> in the nanosleep method:
>
>
>
> Thread 2 (Thread -1210287216 (LWP 13115)):
>
> #0 0x00994410 in __kernel_vsyscall ()
>
> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>
> #2 0x0805b0d0 in wrapperSleep ()
>
> #3 0x0805b420 in timerRunner ()
>
> #4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
>
> #5 0x00c7512e in clone () from /lib/libc.so.6
>
> Thread 1 (Thread -1208169792 (LWP 13114)):
>
> #0 0x00994410 in __kernel_vsyscall ()
>
> #1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
>
> #2 0x0805b0d0 in wrapperSleep ()
>
> #3 0x08059f6c in wrapperEventLoop ()
>
> #4 0x08056628 in wrapperRunConsole ()
>
> #5 0x0805cce3 in main ()
>
> #0 0x00994410 in __kernel_vsyscall ()
>
>
>
> If I send a signal to the wrapper, it reacts again, but before, it hangs
> forever in the nanosleep method.
>
> Is this a known problem of ServiceWrapper running with multiple CPUs?
>
> Do I have to recompile with usleep support instead or is there an option to
> always use usleep?
>
>
>
> Best,
>
> --Christopher (on behalf of Johannes)
|
|
From: Christopher T. <ct...@co...> - 2010-11-23 10:31:40
|
Hi all,
I'm forwarding this on behalf of Johannes, he seems to be having
problems with his list subscription. I apologize if this is a double
post, I forgot to add the subject on the first mail.
--- snip ---
I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS 5 in
a VM with 2 CPUs.
Whenever I stopped a Java program with the wrapper (different
configurations), the wrapper stopped pinging the JVM after some seconds
(which eventually leads to the JVM ending itself because it does not
receive ping packages from the wrapper any more).
The last system call that never returns according to strace is always
nanosleep:
gettimeofday({1290172014, 465559}, NULL) = 0
read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1290172014, 465610}, NULL) = 0
recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
temporarily unavailable)
waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
nanosleep({0, 100000000}, NULL) = 0
gettimeofday({1290172014, 566505}, NULL) = 0
read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1290172014, 566552}, NULL) = 0
recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
temporarily unavailable)
waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
nanosleep({0, 100000000},
If I call pstack on the wrapper, you can see that two threads currently
hang in the nanosleep method:
Thread 2 (Thread -1210287216 (LWP 13115)):
#0 0x00994410 in __kernel_vsyscall ()
#1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
#2 0x0805b0d0 in wrapperSleep ()
#3 0x0805b420 in timerRunner ()
#4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
#5 0x00c7512e in clone () from /lib/libc.so.6
Thread 1 (Thread -1208169792 (LWP 13114)):
#0 0x00994410 in __kernel_vsyscall ()
#1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
#2 0x0805b0d0 in wrapperSleep ()
#3 0x08059f6c in wrapperEventLoop ()
#4 0x08056628 in wrapperRunConsole ()
#5 0x0805cce3 in main ()
#0 0x00994410 in __kernel_vsyscall ()
If I send a signal to the wrapper, it reacts again, but before, it hangs
forever in the nanosleep method.
Is this a known problem of ServiceWrapper running with multiple CPUs?
Do I have to recompile with usleep support instead or is there an option
to always use usleep?
Best,
--Christopher (on behalf of Johannes)
|
|
From: Christopher T. <ct...@co...> - 2010-11-22 09:29:39
|
Hi all,
I'm forwarding this on behalf of Johannes, he seems to be having
problems with his list subscription.
--- snip ---
I am using ServiceWrapper (stable version 3.5.6) under 32bit CentOS 5 in
a VM with 2 CPUs.
Whenever I stopped a Java program with the wrapper (different
configurations), the wrapper stopped pinging the JVM after some seconds
(which eventually leads to the JVM ending itself because it does not
receive ping packages from the wrapper any more).
The last system call that never returns according to strace is always
nanosleep:
gettimeofday({1290172014, 465559}, NULL) = 0
read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1290172014, 465610}, NULL) = 0
recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
temporarily unavailable)
waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
nanosleep({0, 100000000}, NULL) = 0
gettimeofday({1290172014, 566505}, NULL) = 0
read(5, 0x8895070, 1024) = -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1290172014, 566552}, NULL) = 0
recv(7, 0xbffa98cb, 1, 0) = -1 EAGAIN (Resource
temporarily unavailable)
waitpid(13065, 0xbffa98a8, WNOHANG|WSTOPPED) = 0
nanosleep({0, 100000000},
If I call pstack on the wrapper, you can see that two threads currently
hang in the nanosleep method:
Thread 2 (Thread -1210287216 (LWP 13115)):
#0 0x00994410 in __kernel_vsyscall ()
#1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
#2 0x0805b0d0 in wrapperSleep ()
#3 0x0805b420 in timerRunner ()
#4 0x00d1b2db in start_thread () from /lib/libpthread.so.0
#5 0x00c7512e in clone () from /lib/libc.so.6
Thread 1 (Thread -1208169792 (LWP 13114)):
#0 0x00994410 in __kernel_vsyscall ()
#1 0x00d22506 in __nanosleep_nocancel () from /lib/libpthread.so.0
#2 0x0805b0d0 in wrapperSleep ()
#3 0x08059f6c in wrapperEventLoop ()
#4 0x08056628 in wrapperRunConsole ()
#5 0x0805cce3 in main ()
#0 0x00994410 in __kernel_vsyscall ()
If I send a signal to the wrapper, it reacts again, but before, it hangs
forever in the nanosleep method.
Is this a known problem of ServiceWrapper running with multiple CPUs?
Do I have to recompile with usleep support instead or is there an option
to always use usleep?
Best,
--Christopher (on behalf of Johannes)
|
165 messages has been excluded from this view by a project administrator.
