Menu
▾
▴
wrapper-user
|
From: no s. <mrs...@gm...> - 2007-09-21 14:09:49
|
I'm using JavaWrapper 3.2.3 and the JRE 1.6.0_01. I have an application that is using JIntegra to talk to COM objects on the local machine that is controlling video encoding hardware. If the system is not loaded things seem to run great and I never see crashes. If I load the software so that it's hovering around 100% cpu I eventually (after 7 hours & not always exactly at this time) get crashes. In the wrapper logs I see this: The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) JVM exited unexpectedly. Dr Watson shows this: Application exception occurred: App: c:\ipe\mvcp\javawrapper\bin\wrapper.exe (pid=2952) When: 5/1/2007 @ 17:28:49.263 Exception number: c0000005 (access violation) *----> System Information <----* Computer Name: PROXYCT...... User Name: ipv..... Terminal Session Id: 1 Number of Processors: 4 Processor Type: x86 Family 15 Model 4 Stepping 3 Windows Version: 5.2 Current Build: 3790 Service Pack: 1 Current Type: Multiprocessor Free Registered Organization: TBS Registered Owner: TBS *----> Task List <----* 0 System Process 4 System 388 smss.exe 616 csrss.exe 704 winlogon.exe 764 services.exe 776 lsass.exe 988 svchost.exe 1060 svchost.exe 1128 svchost.exe 1196 svchost.exe 1252 svchost.exe 1364 ccSetMgr.exe 1428 ccEvtMgr.exe 864 spoolsv.exe 996 msdtc.exe 1644 vcagent.exe 1664 DefWatch.exe 1692 svchost.exe 1756 svchost.exe 1792 SavRoam.exe 1956 sntlkeyssrvr.exe 1972 spnsrvnt.exe 2000 snmp.exe 2032 snmptrap.exe 464 SVBootService.exe 480 Rtvscan.exe 516 smhstart.exe 580 cpqnimgt.exe 2152 hpsmhd.exe 2184 CpqRcmc.exe 2196 cqmgserv.exe 2240 cqmgstor.exe 2300 sysdown.exe 2420 cqmghost.exe 2464 wmiprvse.exe 2512 svchost.exe 2916 rotatelogs.exe 2924 rotatelogs.exe 2964 hpsmhd.exe 3068 rotatelogs.exe 3076 rotatelogs.exe 2740 wmiprvse.exe 2340 logon.scr 2960 csrss.exe 624 winlogon.exe 1708 rdpclip.exe 3164 Explorer.EXE 2748 cpqteam.exe 2592 ccApp.exe 3128 oobechk.exe 1832 mshta.exe 5128 cmd.exe 5136 bash.exe 5976 csrss.exe 5384 winlogon.exe 6052 rdpclip.exe 4192 Explorer.EXE 3144 cpqteam.exe 5820 ccApp.exe 5824 oobechk.exe 5468 mshta.exe 2624 mmc.exe 856 mmc.exe 5484 bash.exe 5376 cmd.exe 2952 wrapper.exe 576 drwtsn32.exe *----> Module List <----* 0000000000400000 - 0000000000435000: c:\ipe\mvcp\javawrapper\bin\wrapper.exe 0000000071bb0000 - 0000000071bb9000: C:\WINDOWS\system32\WSOCK32.dll 0000000071bf0000 - 0000000071bf8000: C:\WINDOWS\system32\WS2HELP.dll 0000000071c00000 - 0000000071c17000: C:\WINDOWS\system32\WS2_32.dll 0000000077380000 - 0000000077412000: C:\WINDOWS\system32\USER32.dll 0000000077ba0000 - 0000000077bfa000: C:\WINDOWS\system32\msvcrt.dll 0000000077c00000 - 0000000077c49000: C:\WINDOWS\system32\GDI32.dll 0000000077c50000 - 0000000077cef000: C:\WINDOWS\system32\RPCRT4.dll 0000000077da0000 - 0000000077df2000: C:\WINDOWS\system32\SHLWAPI.dll 0000000077e40000 - 0000000077f42000: C:\WINDOWS\system32\kernel32.dll 0000000077f50000 - 0000000077fec000: C:\WINDOWS\system32\ADVAPI32.dll 000000007c800000 - 000000007c8c0000: C:\WINDOWS\system32\ntdll.dll *----> State Dump for Thread Id 0x15d0 <----* eax=01c78c37 ebx=ffff0000 ecx=bb40e508 edx=b34fdbe6 esi=00000000 edi=bb40e64e eip=77e41fb5 esp=0012ff94 ebp=0012ff94 iopl=0 nv up ei pl zr na po nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010246 *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll - function: kernel32!GetSystemTimeAsFileTime 77e41f99 90 nop kernel32!GetSystemTimeAsFileTime: 77e41f9a 8bff mov edi,edi 77e41f9c 55 push ebp 77e41f9d 8bec mov ebp,esp 77e41f9f a11800fe7f mov eax,[SharedUserData+0x18 (7ffe0018)] 77e41fa4 8b151400fe7f mov edx,[SharedUserData+0x14 (7ffe0014)] 77e41faa 3b051c00fe7f cmp eax,[SharedUserData+0x1c (7ffe001c)] 77e41fb0 75ed jnz kernel32!GetSystemTimeAsFileTime+0x5 (77e41f9f) 77e41fb2 8b4d08 mov ecx,[ebp+0x8] FAULT ->77e41fb5 8911 mov [ecx],edx ds:0023:bb40e508=???????? 77e41fb7 894104 mov [ecx+0x4],eax 77e41fba 5d pop ebp 77e41fbb c20400 ret 0x4 77e41fbe 90 nop 77e41fbf 90 nop 77e41fc0 90 nop 77e41fc1 90 nop 77e41fc2 90 nop kernel32!VirtualProtect: 77e41fc3 8bff mov edi,edi *----> Stack Back Trace <----* *** ERROR: Module load completed but symbols could not be loaded for c:\ipe\mvcp\javawrapper\bin\wrapper.exe ChildEBP RetAddr Args to Child WARNING: Stack unwind information not available. Following frames may be wrong. 0012ff94 0041b080 bb40e508 00000000 00000000 kernel32!GetSystemTimeAsFileTime+0x1b 0012ffbd e500413c 0077e523 00000000 00000000 wrapper+0x1b080 590012ff 00000000 00000000 00000000 00000000 0xe500413c *----> Raw Stack Dump <----* 000000000012ff94 bd ff 12 00 80 b0 41 00 - 08 e5 40 bb 00 00 00 00 ......A...@..... 000000000012ffa4 00 00 00 00 00 f0 fd 7f - 00 00 00 00 74 1e 82 7c ............t..| 000000000012ffb4 00 00 00 00 00 00 00 00 - f0 ff 12 00 59 3c 41 00 ............Y<A. 000000000012ffc4 e5 23 e5 77 00 00 00 00 - 00 00 00 00 00 f0 fd 7f .#.w............ 000000000012ffd4 05 00 00 c0 c8 ff 12 00 - c0 fb 12 00 ff ff ff ff ................ 000000000012ffe4 d0 b7 e6 77 f0 23 e5 77 - 00 00 00 00 00 00 00 00 ...w.#.w........ 000000000012fff4 00 00 00 00 54 3c 41 00 - 00 00 00 00 41 63 74 78 ....T<A.....Actx 0000000000130004 20 00 00 00 01 00 00 00 - 38 4f 00 00 24 01 00 00 .......8O..$... 0000000000130014 00 00 00 00 20 00 00 00 - 00 00 00 00 14 00 00 00 .... ........... 0000000000130024 01 00 00 00 0a 00 00 00 - 34 00 00 00 b4 01 00 00 ........4....... 0000000000130034 01 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 ................ 0000000000130044 00 00 00 00 00 00 00 00 - 02 00 00 00 00 00 00 00 ................ 0000000000130054 00 00 00 00 00 00 00 00 - 14 03 00 00 9c 01 00 00 ................ 0000000000130064 00 00 00 00 5b 49 59 2d - b0 04 00 00 32 00 00 00 ....[IY-....2... 0000000000130074 e4 04 00 00 d2 02 00 00 - 00 00 00 00 ba 71 32 f3 .............q2. 0000000000130084 b8 07 00 00 4a 00 00 00 - 04 08 00 00 f6 02 00 00 ....J........... 0000000000130094 00 00 00 00 cd ea ce 32 - fc 0a 00 00 42 00 00 00 .......2....B... 00000000001300a4 40 0b 00 00 08 03 00 00 - 00 00 00 00 79 e8 3c e2 @...........y.<. 00000000001300b4 48 0e 00 00 32 00 00 00 - 7c 0e 00 00 d2 02 00 00 H...2...|....... 00000000001300c4 00 00 00 00 e4 02 02 83 - 50 11 00 00 46 00 00 00 ........P...F... |
|
From: no s. <mrs...@gm...> - 2007-10-01 17:46:25
|
Here are the system details: OS Name Microsoft(R) Windows(R) Server 2003, Standard Edition Version 5.2.3790 Service Pack 2 Build 3790 Other OS Description Not Available OS Manufacturer Microsoft Corporation System Name hostname System Manufacturer HP System Model ProLiant DL380 G4 System Type X86-based PC Processor x86 Family 15 Model 4 Stepping 3 GenuineIntel ~3000 Mhz Processor x86 Family 15 Model 4 Stepping 10 GenuineIntel ~3000 Mhz Processor x86 Family 15 Model 4 Stepping 3 GenuineIntel ~3000 Mhz Processor x86 Family 15 Model 4 Stepping 10 GenuineIntel ~3000 Mhz BIOS Version/Date HP P51, 10/10/2005 SMBIOS Version 2.3 Windows Directory C:\WINDOWS System Directory C:\WINDOWS\system32 Boot Device \Device\HarddiskVolume1 Locale United States Hardware Abstraction Layer Version = "5.2.3790.3959(srv03_sp2_rtm.070216-1710)" User Name Not Available Time Zone Eastern Daylight Time Total Physical Memory 2,047.47 MB Available Physical Memory 1.26 GB Total Virtual Memory 3.36 GB Available Virtual Memory 2.78 GB Page File Space 1.50 GB Page File C:\pagefile.sys |
|
From: Leif M. <le...@ta...> - 2007-10-01 18:09:35
|
Mark, Let me confirm, the message from the wrapper.log is showing that the JVM process is crashing. Does the Wrapper restart the JVM correctly? The DrWatson output is confusing. Is it saying that the Wrapper process crashed as well? The JVM crash itself could be caused by anything. In all cases I have encountered, the crashes would also happen without the Wrapper in the equation. When native libraries are being used, especially custom ones, they tend to be related to the cause. Java developers are not used to the memory management issues that have to be handled in the JNI world of C. Then on top of that, there are lots of strange releases that are required when using Java objects that C developers are not used to dealing with. Is this the product you are using? http://www.intrinsyc.com/products/jintegra.asp Try asking their support about the crash. It may be something they are aware of. Cheers, Leif no spam wrote: > I'm using JavaWrapper 3.2.3 and the JRE 1.6.0_01. I have an > application that is using JIntegra to talk to COM objects on the local > machine that is controlling video encoding hardware. If the system is > not loaded things seem to run great and I never see crashes. If > I load the software so that it's hovering around 100% cpu I eventually (after 7 hours & not always exactly at this time) get crashes. > > > In the wrapper logs I see this: > > The JVM process terminated due to an uncaught exception: > EXCEPTION_ACCESS_VIOLATION (0xc0000005) > JVM exited unexpectedly. > > > Dr Watson shows this: > > Application exception occurred: > App: c:\ipe\mvcp\javawrapper\bin\wrapper.exe (pid=2952) > When: 5/1/2007 @ 17:28:49.263 > Exception number: c0000005 (access violation) <snip> |
|
From: no s. <mrs...@gm...> - 2007-10-02 03:02:22
|
Hi Leif, The wrapper process seems to have problems restarting the jvm process. In fact after this crash there will be several other crashes noticeable in the event viewer and also in the wrapper log. Right the Dr Watson output is very confusing. I've been running a test since last Thursday without the wrapper and everything seems to be running great. I believe that my last test ran better when I upped the ping time in the wrapper config. This process runs in parallel with a second JVM process that hovers around 90-95% cpu, could this be an issue? Yes I'm using JIntegra to talk to another vendor's objects (IPV). I should know a bit more in a few days if things keep running. This is really some interesting software here at CNN. This code controls all the low resolution encoders for the newsroom. It has to be 100% stable which is why I have been load testing for the past few weeks. Thanks for your prompt reply. I really want to be able to run this via the wrapper as it's so convenient. Mark |
|
From: Leif M. <le...@ta...> - 2007-10-02 03:56:58
|
Mark, What does the wrapper.log look like just before the crash? I am wondering if the Wrapper is thinking that the JVM is frozen and attempting to restart it. If the native code is not written correctly it could be leading to a crash as the JVM tries to shutdown. I ask because you mentioned that making the ping timeout longer helped. Could you enable wrapper.debug=true and try this again. Then post back or send the resulting log files (from a couple minutes before the crash) to the list or to me directly if it has private data. Cheers, Leif no spam wrote: > Hi Leif, > > The wrapper process seems to have problems restarting the jvm > process. In fact after this crash there will be several other crashes > noticeable in the event viewer and also in the wrapper log. > > Right the Dr Watson output is very confusing. I've been running a > test since last Thursday without the wrapper and everything seems to > be running great. I believe that my last test ran better when I upped > the ping time in the wrapper config. This process runs in parallel > with a second JVM process that hovers around 90-95% cpu, could this be > an issue? > > Yes I'm using JIntegra to talk to another vendor's objects (IPV). I > should know a bit more in a few days if things keep running. This is > really some interesting software here at CNN. This code controls all > the low resolution encoders for the newsroom. It has to be 100% > stable which is why I have been load testing for the past few weeks. > > Thanks for your prompt reply. I really want to be able to run this > via the wrapper as it's so convenient. > > Mark |
|
From: no s. <mrs...@gm...> - 2007-10-02 17:20:38
|
Here is what the log looks like. I will enable debug and try and get more information: Can you give me details on how the ping works? ERROR | wrapper | 2007/09/25 03:25:31 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:25:31 | JVM exited unexpectedly. STATUS | wrapper | 2007/09/25 03:25:35 | Launching a JVM... INFO | jvm 2 | 2007/09/25 03:25:36 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 2 | 2007/09/25 03:25:36 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 2 | 2007/09/25 03:25:36 | INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | *************************************************** INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | Thank you for choosing J-Integra. INFO | jvm 2 | 2007/09/25 03:25:38 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 2 | 2007/09/25 03:25:38 | documentation, support, and purchasing information. INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | *************************************************** INFO | jvm 2 | 2007/09/25 03:25:38 | INFO | jvm 2 | 2007/09/25 03:25:38 | ERROR | wrapper | 2007/09/25 03:29:36 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:29:36 | JVM exited unexpectedly. STATUS | wrapper | 2007/09/25 03:29:41 | Launching a JVM... INFO | jvm 3 | 2007/09/25 03:29:41 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 3 | 2007/09/25 03:29:41 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 3 | 2007/09/25 03:29:41 | INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | *************************************************** INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | Thank you for choosing J-Integra. INFO | jvm 3 | 2007/09/25 03:29:48 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 3 | 2007/09/25 03:29:48 | documentation, support, and purchasing information. INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | *************************************************** INFO | jvm 3 | 2007/09/25 03:29:48 | INFO | jvm 3 | 2007/09/25 03:29:48 | ERROR | wrapper | 2007/09/25 03:34:00 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:34:00 | JVM exited unexpectedly. STATUS | wrapper | 2007/09/25 03:34:04 | Launching a JVM... INFO | jvm 4 | 2007/09/25 03:34:04 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 4 | 2007/09/25 03:34:04 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 4 | 2007/09/25 03:34:04 | INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | *************************************************** INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | Thank you for choosing J-Integra. INFO | jvm 4 | 2007/09/25 03:34:33 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 4 | 2007/09/25 03:34:33 | documentation, support, and purchasing information. INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | *************************************************** INFO | jvm 4 | 2007/09/25 03:34:33 | INFO | jvm 4 | 2007/09/25 03:34:33 | ERROR | wrapper | 2007/09/25 03:38:54 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:38:54 | JVM exited unexpectedly. STATUS | wrapper | 2007/09/25 03:38:58 | Launching a JVM... INFO | jvm 5 | 2007/09/25 03:38:58 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 5 | 2007/09/25 03:38:58 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 5 | 2007/09/25 03:38:58 | INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | *************************************************** INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | Thank you for choosing J-Integra. INFO | jvm 5 | 2007/09/25 03:39:01 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 5 | 2007/09/25 03:39:01 | documentation, support, and purchasing information. INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | *************************************************** INFO | jvm 5 | 2007/09/25 03:39:01 | INFO | jvm 5 | 2007/09/25 03:39:01 | ERROR | wrapper | 2007/09/25 03:42:40 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:42:40 | JVM exited unexpectedly. STATUS | wrapper | 2007/09/25 03:42:44 | Launching a JVM... INFO | jvm 6 | 2007/09/25 03:42:45 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 6 | 2007/09/25 03:42:45 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 6 | 2007/09/25 03:42:45 | INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | *************************************************** INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | Thank you for choosing J-Integra. INFO | jvm 6 | 2007/09/25 03:43:48 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 6 | 2007/09/25 03:43:48 | documentation, support, and purchasing information. INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | *************************************************** INFO | jvm 6 | 2007/09/25 03:43:48 | INFO | jvm 6 | 2007/09/25 03:43:48 | ERROR | wrapper | 2007/09/25 03:47:24 | The JVM process terminated due to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) ERROR | wrapper | 2007/09/25 03:47:24 | JVM exited unexpectedly. FATAL | wrapper | 2007/09/25 03:47:24 | There were 5 failed launches in a row, each lasting less than 300 seconds. Giving up. FATAL | wrapper | 2007/09/25 03:47:24 | There may be a configuration problem: please check the logs. STATUS | wrapper | 2007/09/25 03:47:24 | <-- Wrapper Stopped STATUS | wrapper | 2007/09/25 21:30:48 | --> Wrapper Started as Service STATUS | wrapper | 2007/09/25 21:30:49 | Launching a JVM... INFO | jvm 1 | 2007/09/25 21:30:50 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org INFO | jvm 1 | 2007/09/25 21:30:50 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 1 | 2007/09/25 21:30:50 | INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | J-Integra 2.5 Copyright (C) 1998-2005 Intrinsyc Software International, Inc. http://j-integra.intrinsyc.com/ INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | *************************************************** INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | Thank you for choosing J-Integra. INFO | jvm 1 | 2007/09/25 21:31:06 | Please visit http://j-integra.intrinsyc.com/ for INFO | jvm 1 | 2007/09/25 21:31:06 | documentation, support, and purchasing information. INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | *************************************************** INFO | jvm 1 | 2007/09/25 21:31:06 | INFO | jvm 1 | 2007/09/25 21:31:06 | On 10/1/07, Leif Mortenson <le...@ta...> wrote: > > Mark, > What does the wrapper.log look like just before the crash? I am > wondering if the Wrapper is thinking that the JVM is frozen and > attempting to restart it. If the native code is not written correctly > it could be leading to a crash as the JVM tries to shutdown. > I ask because you mentioned that making the ping timeout longer > helped. > > Could you enable wrapper.debug=true and try this again. Then > post back or send the resulting log files (from a couple minutes before > the crash) to the list or to me directly if it has private data. > > Cheers, > Leif > > no spam wrote: > > Hi Leif, > > > > The wrapper process seems to have problems restarting the jvm > > process. In fact after this crash there will be several other crashes > > noticeable in the event viewer and also in the wrapper log. > > > > Right the Dr Watson output is very confusing. I've been running a > > test since last Thursday without the wrapper and everything seems to > > be running great. I believe that my last test ran better when I upped > > the ping time in the wrapper config. This process runs in parallel > > with a second JVM process that hovers around 90-95% cpu, could this be > > an issue? > > > > Yes I'm using JIntegra to talk to another vendor's objects (IPV). I > > should know a bit more in a few days if things keep running. This is > > really some interesting software here at CNN. This code controls all > > the low resolution encoders for the newsroom. It has to be 100% > > stable which is why I have been load testing for the past few weeks. > > > > Thanks for your prompt reply. I really want to be able to run this > > via the wrapper as it's so convenient. > > > > Mark > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Wrapper-user mailing list > Wra...@li... > https://lists.sourceforge.net/lists/listinfo/wrapper-user > |
|
From: Leif M. <le...@ta...> - 2007-10-02 23:56:43
|
Mark, So it looks like once it restarts once, it crashes consistently between 4 and 4.5 minutes after the new JVM has started again. If you look at the task manager, does it look like the memory usage of the Java process is growing? One possibility is a memory leak. In native C code, it is common to allocate memory and assume that the returned pointer is valid. If the system runs out of memory and the allocation fails however, NULL will be returned and you will get an ACCESS_VIOLATION if that NULL pointer is then used. The following two properties will cause the Wrapper to output the memory usage of the Wrapper and Java processes once per second. Try adding this in addition to the debug output and then reproduce your problem. It will show if there are any memory leaks before the JVM crashes. wrapper.memory_output=true wrapper.memory_output.interval=1 wraper.debug=true If your second JVM or another process is the one consuming all the memory then this may not give you the information you need as this process could be running out of memory even though its own memory usage has not grown that much. Looking at the task manager would shed light on this. You said that this application is encoding video files. Is it possible that the system is running fine over time because is it encoding say files A, B, C, D. But then when it hits file E it is encountering a problem where that leads to the crash. If the system works like a queue then file E would be retried after a restart so the next crash would be more immediate. Cheers, Leif no spam wrote: > Here is what the log looks like. I will enable debug and try and get > more information: > > Can you give me details on how the ping works? > > > ERROR | wrapper | 2007/09/25 03:25:31 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:25:31 | JVM exited unexpectedly. > STATUS | wrapper | 2007/09/25 03:25:35 | Launching a JVM... > INFO | jvm 2 | 2007/09/25 03:25:36 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 2 | 2007/09/25 03:25:36 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 2 | 2007/09/25 03:25:36 | > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ <http://j-integra.intrinsyc.com/> > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | > *************************************************** > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | Thank you for choosing J-Integra. > INFO | jvm 2 | 2007/09/25 03:25:38 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 2 | 2007/09/25 03:25:38 | documentation, support, and > purchasing information. > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | > *************************************************** > INFO | jvm 2 | 2007/09/25 03:25:38 | > INFO | jvm 2 | 2007/09/25 03:25:38 | > ERROR | wrapper | 2007/09/25 03:29:36 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:29:36 | JVM exited unexpectedly. > STATUS | wrapper | 2007/09/25 03:29:41 | Launching a JVM... > INFO | jvm 3 | 2007/09/25 03:29:41 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 3 | 2007/09/25 03:29:41 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 3 | 2007/09/25 03:29:41 | > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | > *************************************************** > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | Thank you for choosing J-Integra. > INFO | jvm 3 | 2007/09/25 03:29:48 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 3 | 2007/09/25 03:29:48 | documentation, support, and > purchasing information. > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | > *************************************************** > INFO | jvm 3 | 2007/09/25 03:29:48 | > INFO | jvm 3 | 2007/09/25 03:29:48 | > ERROR | wrapper | 2007/09/25 03:34:00 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:34:00 | JVM exited unexpectedly. > STATUS | wrapper | 2007/09/25 03:34:04 | Launching a JVM... > INFO | jvm 4 | 2007/09/25 03:34:04 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 4 | 2007/09/25 03:34:04 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 4 | 2007/09/25 03:34:04 | > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ <http://j-integra.intrinsyc.com/> > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | > *************************************************** > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | Thank you for choosing J-Integra. > INFO | jvm 4 | 2007/09/25 03:34:33 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 4 | 2007/09/25 03:34:33 | documentation, support, and > purchasing information. > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | > *************************************************** > INFO | jvm 4 | 2007/09/25 03:34:33 | > INFO | jvm 4 | 2007/09/25 03:34:33 | > ERROR | wrapper | 2007/09/25 03:38:54 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:38:54 | JVM exited unexpectedly. > STATUS | wrapper | 2007/09/25 03:38:58 | Launching a JVM... > INFO | jvm 5 | 2007/09/25 03:38:58 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 5 | 2007/09/25 03:38:58 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 5 | 2007/09/25 03:38:58 | > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | > *************************************************** > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | Thank you for choosing J-Integra. > INFO | jvm 5 | 2007/09/25 03:39:01 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 5 | 2007/09/25 03:39:01 | documentation, support, and > purchasing information. > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | > *************************************************** > INFO | jvm 5 | 2007/09/25 03:39:01 | > INFO | jvm 5 | 2007/09/25 03:39:01 | > ERROR | wrapper | 2007/09/25 03:42:40 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:42:40 | JVM exited unexpectedly. > STATUS | wrapper | 2007/09/25 03:42:44 | Launching a JVM... > INFO | jvm 6 | 2007/09/25 03:42:45 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 6 | 2007/09/25 03:42:45 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 6 | 2007/09/25 03:42:45 | > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ <http://j-integra.intrinsyc.com/> > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | > *************************************************** > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | Thank you for choosing J-Integra. > INFO | jvm 6 | 2007/09/25 03:43:48 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 6 | 2007/09/25 03:43:48 | documentation, support, and > purchasing information. > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | > *************************************************** > INFO | jvm 6 | 2007/09/25 03:43:48 | > INFO | jvm 6 | 2007/09/25 03:43:48 | > ERROR | wrapper | 2007/09/25 03:47:24 | The JVM process terminated due > to an uncaught exception: EXCEPTION_ACCESS_VIOLATION (0xc0000005) > ERROR | wrapper | 2007/09/25 03:47:24 | JVM exited unexpectedly. > FATAL | wrapper | 2007/09/25 03:47:24 | There were 5 failed launches > in a row, each lasting less than 300 seconds. Giving up. > FATAL | wrapper | 2007/09/25 03:47:24 | There may be a configuration > problem: please check the logs. > STATUS | wrapper | 2007/09/25 03:47:24 | <-- Wrapper Stopped > STATUS | wrapper | 2007/09/25 21:30:48 | --> Wrapper Started as Service > STATUS | wrapper | 2007/09/25 21:30:49 | Launching a JVM... > INFO | jvm 1 | 2007/09/25 21:30:50 | Wrapper (Version 3.2.3) > http://wrapper.tanukisoftware.org > INFO | jvm 1 | 2007/09/25 21:30:50 | Copyright 1999-2006 Tanuki > Software, Inc. All Rights Reserved. > INFO | jvm 1 | 2007/09/25 21:30:50 | > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | J-Integra 2.5 Copyright (C) > 1998-2005 Intrinsyc Software International, Inc. > http://j-integra.intrinsyc.com/ > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | > *************************************************** > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | Thank you for choosing J-Integra. > INFO | jvm 1 | 2007/09/25 21:31:06 | Please visit > http://j-integra.intrinsyc.com/ for > INFO | jvm 1 | 2007/09/25 21:31:06 | documentation, support, and > purchasing information. > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | > *************************************************** > INFO | jvm 1 | 2007/09/25 21:31:06 | > INFO | jvm 1 | 2007/09/25 21:31:06 | > > > > On 10/1/07, *Leif Mortenson* < le...@ta... > <mailto:le...@ta...>> wrote: > > Mark, > What does the wrapper.log look like just before the crash? I am > wondering if the Wrapper is thinking that the JVM is frozen and > attempting to restart it. If the native code is not written correctly > it could be leading to a crash as the JVM tries to shutdown. > I ask because you mentioned that making the ping timeout longer > helped. > > Could you enable wrapper.debug=true and try this again. Then > post back or send the resulting log files (from a couple minutes > before > the crash) to the list or to me directly if it has private data. > > Cheers, > Leif > > no spam wrote: > > Hi Leif, > > > > The wrapper process seems to have problems restarting the jvm > > process. In fact after this crash there will be several other > crashes > > noticeable in the event viewer and also in the wrapper log. > > > > Right the Dr Watson output is very confusing. I've been running a > > test since last Thursday without the wrapper and everything > seems to > > be running great. I believe that my last test ran better when I > upped > > the ping time in the wrapper config. This process runs in parallel > > with a second JVM process that hovers around 90-95% cpu, could > this be > > an issue? > > > > Yes I'm using JIntegra to talk to another vendor's objects (IPV). I > > should know a bit more in a few days if things keep > running. This is > > really some interesting software here at CNN. This code > controls all > > the low resolution encoders for the newsroom. It has to be 100% > > stable which is why I have been load testing for the past few weeks. > > > > Thanks for your prompt reply. I really want to be able to run this > > via the wrapper as it's so convenient. > > > > Mark > |
|
From: no s. <mrs...@gm...> - 2007-10-03 14:31:01
|
I have network, cpu and memory graphs for my performance testing. The graphs show that memory is stable and not running out. I'm running a long term test now on another server to see what happens with debug turned on. Mark |
