Menu
▾
▴
wrapper-user
|
From: Paul C. <cas...@au...> - 2004-03-31 23:44:19
|
Leif, Thanks for your info - I think I might have to buy a book and learn inside out about Java GC/memory usage. Our app takes around 950MB to start due to loading all the GIS stuff into RAM upon startup. The application author's theory was that reading the GIS stuff from RAM is heaps faster than from disk and it has to be read into RAM at some stage (when used), so it might as well stay there! Our app takes around 15 minutes to start on a fairly decent pair of boxes - the bottleneck is a gigabit Ethernet crossover cable running between the servers. Our client has ordered a pair of fibre gigabit cards for each box and a managed fibre switch, but they haven't turned up yet. The app takes 6 minutes to start when app and DB are on the same box, but that's less scalable (HD / RAM capacity constraints). Due to the startup time, we don't want to do anything to extend startup - that's why we're happy to allocate a fair chunk to the initial memory. It's interesting though - I've noticed that since reducing the initial memory value, the peak memory usage (according to Windows anyway) tends to remain pretty darn close to the initial memory value. This indicates that what you've outlined below seems to be working pretty well (ie when more RAM is needed, try a GC first). So what I've found is that by reducing the initial memory value in the config file, our JVM hasn't hung for the past 3 weeks. This is a good sign. The problem that's now occurring more frequently is the java.lang.OutOfMemoryError, which indicates that at some stage the process needs more RAM than is available: either the GC fails or something else is going on. And now that I think of it, we can't tell what the process maxes out at because by the time that I realise there's a problem the app has been restarted and Windows has lost the peak memory of the process that crashed (of course). I've turned on some logging as suggested to dump out memory usage, so we'll see what this shows. Incidently, the most I've been able to allocate to Java (Xmx) is about 1625MB on Windows 2000 Advanced Server. On NT (previous platform) it was only about 1100MB - although that was with JDK 1.3.0, so that may have had an impact. If I attempt starting with more than 1625MB, the app fails to start. I think this is just a Java/Windows relationship thing. The log files show the java.lang.OutOfMemoryError's for a while before the JVM eventually crashes. Is there anyway to set up the Wrapper so that I can be notified (preferably by email) of this error so that I can remote takeover the box and have a look at things before the Wrapper restarts it? Regards, Paul Casanova |---------+----------------------------------------> | | Leif Mortenson | | | <le...@ta...> | | | Sent by: | | | wra...@li...| | | ceforge.net | | | | | | | | | 31/03/2004 06:13 PM | | | Please respond to | | | wrapper-user | |---------+----------------------------------------> >--------------------------------------------------------------------------------------------------------------| | | | To: wra...@li... | | cc: | | Subject: Re: [Wrapper-user] JVM hang - Xms / Xmx | >--------------------------------------------------------------------------------------------------------------| Paul, Here is Java memory as I understand it. :-] As I understand it, the only reason to set the init memory value to anything other than the default is if you know that your application will always need a certain amount of memory on startup. This will cause the JVM to allocate all of that memory in one large chunk rather than bit by bit as the application loads and initializes all of its data. This can improve performance a bit. The JVM starts out with it allocated memory size set to the init memory size. If at any point, the JVM needs more memory than it has allocated then it first attempts to do a GC. If that fails then it will check to see if the allocated size is less than the max size. If it is then another block of memory is allocated, thus increasing the allocated memory. If the max size is hit then you will get an out of memory exception. This amount of real memory on the system does not cause OutOfMemory exceptions unless the max size set when launching the JVM is larger than the total Virtual memory on the machine. If you start using swap memory however then performance will be severely degraded (not an issue with you) Each time a GC is performed, old memory will be freed up from the allocated memory. As the JVM runs, its free memory will slowly decrease until the above is repeated. From what I have seen, once a block of memory is allocated from the system it is never returned. So the allocated memory will stick at the largest size since the JVM was started. (The exception appears to be 1.4.2 I have noticed that it seems to give memory back to the system?) Now, remember what I said about how the the JVM uses free memory until it is exhausted. Well if that free memory is very large then the time between GCs will be long. But the size of each GC will be quite large. This appears to cause a larger hiccup in JVM performance. For this reason, I usually try to keep the init memory small so that the free memory block is never any larger than is absolutely needed. This also makes sure that I don't use any more system memory than is really needed. So what happens if you don't set the initmemory at all. (Ie, leave it at 3MB) Your application should still work but it may not start up quite as fast. I am wondering how much memory the application takes on startup. Then what happens to the memory over time. It you have set the init memory to 1400 but it is still growing to 1600, this means that the application is really requiring that much memory. This could be because of a leak or because you actually need more than 1600MB. What happens if you set it to 2000MB Are you familiar with the memory methods in the Runtime class? Try setting up a thread in your program which logs the Allocated Memory, Free Memory, and then via a little math, the in use memory to the console once per minute or so. This will give you a much more accurate view of what is happening memory wise than you will get from watching the memory usage of Java in the Task Manager. You may also want to call System.gc just before taking the memory measurements to make sure you are looking at the memory usage without any garbage in memory. I always use the Instrumentation package that I wrote for the Apache Avalon Excalibur project to monitor memory and other data over time. It lets you view data like the images I attached. Much easier to understand than log output. That project is quite stable and well tested but at the moment it is lacking in documentation. Something I need to get around to doing. http://avalon.apache.org/excalibur/index.html More below. Paul Casanova wrote: >Thanks Leif. > >We have 6.3GB of RAM in a Win2K Advanced Server box - usually around 3GB >free, so there's no chance of running out of physical RAM. > >We were using: ># Initial Java Heap Size (in MB) >wrapper.java.initmemory=1600 > ># Maximum Java Heap Size (in MB) >wrapper.java.maxmemory=1600 > > >Now using: ># Initial Java Heap Size (in MB) >wrapper.java.initmemory=1400 > ># Maximum Java Heap Size (in MB) >wrapper.java.maxmemory=1600 > >When first trying the change, I tried initmemory=1200, then 1300, now 1400 >just to make sure it was functioning ok. In the Windows Task Manager the >Java process always peaked closed to each of these figures - never exceeded >1600MB as it did previously (used to get up to 1780MB: 1600MB + JVM >overhead I assume). > >I've seen this hotspot error once before since switching to JSW (months >back) - before hand it probably would have just been lost (svrany - MS >product) - yay for JSW I say. > >Once again I don't seem to be able to find anything concrete about this >error in Sun's Java forums, other than suggestions that native font dll's >may have become corrupt - but then one would expect the error to occur >regularly. It happened twice in one day last week, and then today, with >varying durations between the java.lang.OutOfMemoryError's and the JVM >crash. They've both been there each time though. > > I am wondering if 1600MB is too small or if your application is leaking memory? It would not be fun profiling an app that large. :-/ >We had always previously set the min and max heap values to the same. I >couldn't find much Sun info on how Xms and Xmx alter the behaviour of >garbage collection (or any other behaviour for that matter) - only in >http://java.sun.com/docs/hotspot/gc/. I just gave it a whirl and it seems >to have made a difference to the JVM hanging. > > What changes did you make? >I also read in one of the forums that issues with native calls from the JVM >were rectified in 1.4.1_02 +. Have you also heard this? > > I had not heard of any problems with earlier versions. Cheers, Leif #### IM.png has been removed from this note on April 01, 2004 by Paul Casanova |
|
From: Leif M. <le...@ta...> - 2004-04-01 08:13:14
|
Paul,
Paul Casanova wrote:
>Thanks for your info - I think I might have to buy a book and learn inside
>out about Java GC/memory usage.
>
>Our app takes around 950MB to start due to loading all the GIS stuff into
>RAM upon startup. The application author's theory was that reading the GIS
>stuff from RAM is heaps faster than from disk and it has to be read into
>RAM at some stage (when used), so it might as well stay there!
>
>
Not sure if this is an option. But would is it possible to break the
data up into
3 groups?
1) Data that is currently in use (stored in memory and on the local disk)
2) Data that is likely to be used (transfered and then stored to the
local disk)
3) Other data (stored only on the remote server)
Net, is it possible to timestamp the data so you know when you need to
obtain the
latest data from the remote server again? If data has not changed then
you could
continue using data stored on the disk.
This would increase the complexity of your app a bit (If it is even
possible) But should
reduce the memory footprint by quite a bit and also improve startup
times, at least for
the second invocation of the application.
>Our app takes around 15 minutes to start on a fairly decent pair of boxes -
>the bottleneck is a gigabit Ethernet crossover cable running between the
>servers. Our client has ordered a pair of fibre gigabit cards for each box
>and a managed fibre switch, but they haven't turned up yet. The app takes
>6 minutes to start when app and DB are on the same box, but that's less
>scalable (HD / RAM capacity constraints).
>
>Due to the startup time, we don't want to do anything to extend startup -
>that's why we're happy to allocate a fair chunk to the initial memory.
>
>It's interesting though - I've noticed that since reducing the initial
>memory value, the peak memory usage (according to Windows anyway) tends to
>remain pretty darn close to the initial memory value. This indicates that
>what you've outlined below seems to be working pretty well (ie when more
>RAM is needed, try a GC first).
>
>
Good that sounds like you are seeing what I expected. Just as an
experiment. Try
lowering the initial memory down to the default (3MB) and see how much of a
difference there is in your application's startup time. Your app is
much larger than
anything I have used. But I have not seen all that large a difference
when the app
is in the 512MB range. I am interested to hear how significant the
difference is.
This experiment will also give you a better understanding of how much
memory the
application really needs and when that memory is needed. Do this at the
same time
as you have that memory logging background thread running. You should
then be
able to monitor how memory is allocated as your app is starting up and
running.
>So what I've found is that by reducing the initial memory value in the
>config file, our JVM hasn't hung for the past 3 weeks. This is a good
>sign.
>
>The problem that's now occurring more frequently is the
>java.lang.OutOfMemoryError, which indicates that at some stage the process
>needs more RAM than is available: either the GC fails or something else is
>going on. And now that I think of it, we can't tell what the process maxes
>out at because by the time that I realise there's a problem the app has
>been restarted and Windows has lost the peak memory of the process that
>crashed (of course).
>
>
I think it is fairly safe to assume that GC is working correctly. Which
means that
something in the application is actually eating up all available
memory. What
exactly that is yet to be determined.
1) The obvious is an object leak someplace. These are easily located by
adding the
following to your Wrapper.conf file:
wrapper.java.additional.1=-Xrunhprof:depth=8
wrapper.shutdown.timeout=0
This will increase the runtime memory usage some and also affects the
performance of
the application. But it should still be usable. When you go to
shutdown the JVM. I
will appear to be locked up for quite a while as it writes out the
profile data file. I
have seen this take a couple minutes. But I would not be surprised if
it took 10
minutes for your application. (That is the reason for the shutdown
timeout setting.
We don't want the Wrapper killing the JVM as it tries to save all the
data we worked
so hard to collect)
When the JVM exits you will see the following in the console or log file.
---
Dumping Java heap ... allocation sites ... done.
---
Look in the directory where Wrapper.exe is located and you will fine a
file: java.hprof.txt
This file could be quite large, so I hope you have a good editor.
Search for "SITES BEGIN". You will see a section toward the end of the
file that looks
like this:
---
SITES BEGIN (ordered by live bytes) Fri Oct 11 12:56:48 2002
percent live alloc'ed stack class
rank self accum bytes objs bytes objs trace name
1 6.80% 6.80% 544264 4779 544264 4779 0 [C
2 2.78% 9.59% 222624 964 223280 966 6361 [C
3 2.14% 11.73% 171248 1262 173504 1299 1 [C
4 2.12% 13.85% 169368 106 169368 106 17124 [B
5 2.08% 15.93% 166736 183 166736 183 0 [B
6 1.71% 17.64% 136624 397 208088 796 17100 [C
7 1.55% 19.18% 123664 435 123664 435 16788 [C
---
This section is ordered by how much memory is being used by objects which
were created at specific places in the code. If you are lucky, the
first row will
be a very large number.
The second to the last column is the Trace number. This says where the
objects
were created. In the above example this is "4779".
To find out where these [C (Character Arrays) where created, do another
search
for "TRACE 4779:"
I get the following:
---
TRACE 4779:
java.lang.StringBuffer.toString(StringBuffer.java:1233)
org.apache.catalina.startup.Catalina.createContextCommon(Catalina.java:614)
org.apache.catalina.startup.Catalina.createStartMapperDefaultContext(Catalina.java:521)
org.apache.catalina.startup.Catalina.createStartMapper(Catalina.java:391)
org.apache.catalina.startup.Catalina.start(Catalina.java:722)
org.apache.catalina.startup.Catalina.execute(Catalina.java:681)
org.apache.catalina.startup.Catalina.process(Catalina.java:179)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:Native
method)
---
You may find that you need to increase the stack depth, but that takes
more memory
and I usually find a depth of 8 to be sufficient.
2) I have seen cases where synchronization errors in HotSpot compiled
applications
have caused memory corruption. In most cases, such corruption will
cause the JVM
to crash, but in others it has simply resulted in a memory leak that
doesn't show up
in the profiling output above.
I wonder about this second option because you are also experiencing JVM
crashes.
These can be a real bi_ch to track down, so lets hope this is not the cause.
One way to check this is to try disabling the JIT/HotSpot compiler.
Doing so will
often cause more Java like errors to start showing up as nothing has
been optimized
into fast machine code.
>I've turned on some logging as suggested to dump out memory usage, so we'll
>see what this shows.
>
>Incidently, the most I've been able to allocate to Java (Xmx) is about
>1625MB on Windows 2000 Advanced Server. On NT (previous platform) it was
>only about 1100MB - although that was with JDK 1.3.0, so that may have had
>an impact. If I attempt starting with more than 1625MB, the app fails to
>start. I think this is just a Java/Windows relationship thing.
>
>
Looking at the following page, it looks like Java started supporting
2GB+ heap sizes as
of 1.3.1. You might want to give that a try. There are also several
command line
options described on this page which could interest you.
http://java.sun.com/docs/hotspot/ism.html
Looking at those docs means that I need to change the way I handle the
memory settings
in the Wrapper. Some of them require that the -Xms and -Xmx values
which are always
set by the Wrapper not be set.
>The log files show the java.lang.OutOfMemoryError's for a while before the
>JVM eventually crashes. Is there anyway to set up the Wrapper so that I
>can be notified (preferably by email) of this error so that I can remote
>takeover the box and have a look at things before the Wrapper restarts it?
>
>
Not really at the moment. What is triggering the JVM to be restarted now?
Have you registered an output trigger on the OutOfMemoryError console
output?
If the Wrapper is killing the JVM because it stops responding to pings
then you can
simply set the Ping timeout to 0. This will not enable you to get an
email, but it would
stop the JVM from restarting it.
Cheers,
Leif
|
|
From: Leif M. <le...@ta...> - 2004-04-01 10:22:35
|
Leif Mortenson wrote: > Looking at those docs means that I need to change the way I handle the > memory settings > in the Wrapper. Some of them require that the -Xms and -Xmx values > which are always > set by the Wrapper not be set. Ok, this is done. 3.1.0 will not pass memory parameters to the JVM if the init and max properties are not specified in the wrapper.conf. This means that you can also now specify them manually using the wrapper.java.additional.<n> properties as well. Cheers, Leif |
|
From: Paul C. <cas...@au...> - 2004-04-07 10:50:18
|
Hi Leif,
I tried adding -Xrunhprof:depth=8 to the config file, but the JVM always
suffers a runtime error r6017 ("unexpected multithread lock error") and
restarts. Any ideas?
Tried starting with initial heap size of 100MB and it took around the same
time to start as normal, but when I added the -Xrun... it started doing
full GC's when starting up the app, so it took longer. Mind you, before I
added -Xrun... it was doing around 10 GC's (not full) during startup with
100MB initial!!
Does -Xrunhprof usually have such overhead / impact?
Regards,
Paul Casanova
IBM RSSC-Ballarat
UB01
Locked Bag 1999
Ballarat
Victoria 3350
Telephone:
61-3-53273458
Fax:
61-3-53273599
|---------+---------------------------------------->
| | Leif Mortenson |
| | <le...@ta...> |
| | Sent by: |
| | wra...@li...|
| | ceforge.net |
| | |
| | |
| | 01/04/2004 06:13 PM |
| | Please respond to |
| | wrapper-user |
|---------+---------------------------------------->
>--------------------------------------------------------------------------------------------------------------|
| |
| To: wra...@li... |
| cc: |
| Subject: Re: [Wrapper-user] JVM hang - Xms / Xmx |
>--------------------------------------------------------------------------------------------------------------|
Paul,
Paul Casanova wrote:
>Thanks for your info - I think I might have to buy a book and learn inside
>out about Java GC/memory usage.
>
>Our app takes around 950MB to start due to loading all the GIS stuff into
>RAM upon startup. The application author's theory was that reading the
GIS
>stuff from RAM is heaps faster than from disk and it has to be read into
>RAM at some stage (when used), so it might as well stay there!
>
>
Not sure if this is an option. But would is it possible to break the
data up into
3 groups?
1) Data that is currently in use (stored in memory and on the local disk)
2) Data that is likely to be used (transfered and then stored to the
local disk)
3) Other data (stored only on the remote server)
Net, is it possible to timestamp the data so you know when you need to
obtain the
latest data from the remote server again? If data has not changed then
you could
continue using data stored on the disk.
This would increase the complexity of your app a bit (If it is even
possible) But should
reduce the memory footprint by quite a bit and also improve startup
times, at least for
the second invocation of the application.
>Our app takes around 15 minutes to start on a fairly decent pair of boxes
-
>the bottleneck is a gigabit Ethernet crossover cable running between the
>servers. Our client has ordered a pair of fibre gigabit cards for each
box
>and a managed fibre switch, but they haven't turned up yet. The app takes
>6 minutes to start when app and DB are on the same box, but that's less
>scalable (HD / RAM capacity constraints).
>
>Due to the startup time, we don't want to do anything to extend startup -
>that's why we're happy to allocate a fair chunk to the initial memory.
>
>It's interesting though - I've noticed that since reducing the initial
>memory value, the peak memory usage (according to Windows anyway) tends to
>remain pretty darn close to the initial memory value. This indicates that
>what you've outlined below seems to be working pretty well (ie when more
>RAM is needed, try a GC first).
>
>
Good that sounds like you are seeing what I expected. Just as an
experiment. Try
lowering the initial memory down to the default (3MB) and see how much of a
difference there is in your application's startup time. Your app is
much larger than
anything I have used. But I have not seen all that large a difference
when the app
is in the 512MB range. I am interested to hear how significant the
difference is.
This experiment will also give you a better understanding of how much
memory the
application really needs and when that memory is needed. Do this at the
same time
as you have that memory logging background thread running. You should
then be
able to monitor how memory is allocated as your app is starting up and
running.
>So what I've found is that by reducing the initial memory value in the
>config file, our JVM hasn't hung for the past 3 weeks. This is a good
>sign.
>
>The problem that's now occurring more frequently is the
>java.lang.OutOfMemoryError, which indicates that at some stage the process
>needs more RAM than is available: either the GC fails or something else is
>going on. And now that I think of it, we can't tell what the process
maxes
>out at because by the time that I realise there's a problem the app has
>been restarted and Windows has lost the peak memory of the process that
>crashed (of course).
>
>
I think it is fairly safe to assume that GC is working correctly. Which
means that
something in the application is actually eating up all available
memory. What
exactly that is yet to be determined.
1) The obvious is an object leak someplace. These are easily located by
adding the
following to your Wrapper.conf file:
wrapper.java.additional.1=-Xrunhprof:depth=8
wrapper.shutdown.timeout=0
This will increase the runtime memory usage some and also affects the
performance of
the application. But it should still be usable. When you go to
shutdown the JVM. I
will appear to be locked up for quite a while as it writes out the
profile data file. I
have seen this take a couple minutes. But I would not be surprised if
it took 10
minutes for your application. (That is the reason for the shutdown
timeout setting.
We don't want the Wrapper killing the JVM as it tries to save all the
data we worked
so hard to collect)
When the JVM exits you will see the following in the console or log file.
---
Dumping Java heap ... allocation sites ... done.
---
Look in the directory where Wrapper.exe is located and you will fine a
file: java.hprof.txt
This file could be quite large, so I hope you have a good editor.
Search for "SITES BEGIN". You will see a section toward the end of the
file that looks
like this:
---
SITES BEGIN (ordered by live bytes) Fri Oct 11 12:56:48 2002
percent live alloc'ed stack class
rank self accum bytes objs bytes objs trace name
1 6.80% 6.80% 544264 4779 544264 4779 0 [C
2 2.78% 9.59% 222624 964 223280 966 6361 [C
3 2.14% 11.73% 171248 1262 173504 1299 1 [C
4 2.12% 13.85% 169368 106 169368 106 17124 [B
5 2.08% 15.93% 166736 183 166736 183 0 [B
6 1.71% 17.64% 136624 397 208088 796 17100 [C
7 1.55% 19.18% 123664 435 123664 435 16788 [C
---
This section is ordered by how much memory is being used by objects which
were created at specific places in the code. If you are lucky, the
first row will
be a very large number.
The second to the last column is the Trace number. This says where the
objects
were created. In the above example this is "4779".
To find out where these [C (Character Arrays) where created, do another
search
for "TRACE 4779:"
I get the following:
---
TRACE 4779:
java.lang.StringBuffer.toString(StringBuffer.java:1233)
org.apache.catalina.startup.Catalina.createContextCommon(Catalina.java:614)
org.apache.catalina.startup.Catalina.createStartMapperDefaultContext(Catalina.java:521)
org.apache.catalina.startup.Catalina.createStartMapper(Catalina.java:391)
org.apache.catalina.startup.Catalina.start(Catalina.java:722)
org.apache.catalina.startup.Catalina.execute(Catalina.java:681)
org.apache.catalina.startup.Catalina.process(Catalina.java:179)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:Native
method)
---
You may find that you need to increase the stack depth, but that takes
more memory
and I usually find a depth of 8 to be sufficient.
2) I have seen cases where synchronization errors in HotSpot compiled
applications
have caused memory corruption. In most cases, such corruption will
cause the JVM
to crash, but in others it has simply resulted in a memory leak that
doesn't show up
in the profiling output above.
I wonder about this second option because you are also experiencing JVM
crashes.
These can be a real bi_ch to track down, so lets hope this is not the
cause.
One way to check this is to try disabling the JIT/HotSpot compiler.
Doing so will
often cause more Java like errors to start showing up as nothing has
been optimized
into fast machine code.
>I've turned on some logging as suggested to dump out memory usage, so
we'll
>see what this shows.
>
>Incidently, the most I've been able to allocate to Java (Xmx) is about
>1625MB on Windows 2000 Advanced Server. On NT (previous platform) it was
>only about 1100MB - although that was with JDK 1.3.0, so that may have had
>an impact. If I attempt starting with more than 1625MB, the app fails to
>start. I think this is just a Java/Windows relationship thing.
>
>
Looking at the following page, it looks like Java started supporting
2GB+ heap sizes as
of 1.3.1. You might want to give that a try. There are also several
command line
options described on this page which could interest you.
http://java.sun.com/docs/hotspot/ism.html
Looking at those docs means that I need to change the way I handle the
memory settings
in the Wrapper. Some of them require that the -Xms and -Xmx values
which are always
set by the Wrapper not be set.
>The log files show the java.lang.OutOfMemoryError's for a while before the
>JVM eventually crashes. Is there anyway to set up the Wrapper so that I
>can be notified (preferably by email) of this error so that I can remote
>takeover the box and have a look at things before the Wrapper restarts it?
>
>
Not really at the moment. What is triggering the JVM to be restarted now?
Have you registered an output trigger on the OutOfMemoryError console
output?
If the Wrapper is killing the JVM because it stops responding to pings
then you can
simply set the Ping timeout to 0. This will not enable you to get an
email, but it would
stop the JVM from restarting it.
Cheers,
Leif
-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Wrapper-user mailing list
Wra...@li...
https://lists.sourceforge.net/lists/listinfo/wrapper-user
|
|
From: Leif M. <le...@ta...> - 2004-04-07 14:11:42
|
Paul,
Paul Casanova wrote:
>I tried adding -Xrunhprof:depth=8 to the config file, but the JVM always
>suffers a runtime error r6017 ("unexpected multithread lock error") and
>restarts. Any ideas?
>
>
Doing a Google search on that error message. It looks like a C runtime
error.
In other words, Java is having a bad day.
Searching the Sun Bug database, the following issue came up:
http://developer.java.sun.com/developer/bugParade/bugs/4724252.html
That bug says it has been fixed in Java 1.4.1. But the original post
says that it not
been a problem in 1.3.1_04. That just means that their app had not been
able to
reproduce it however.
>Tried starting with initial heap size of 100MB and it took around the same
>time to start as normal, but when I added the -Xrun... it started doing
>full GC's when starting up the app, so it took longer. Mind you, before I
>added -Xrun... it was doing around 10 GC's (not full) during startup with
>100MB initial!!
>
>Does -Xrunhprof usually have such overhead / impact?
>
>
Profiling takes a bunch of memory and also has a very noticeable effect of
performance. Depending on the application of course, but it is usually
still
usable if you normally have enough CPU / Memory to spare.
Most likely your app is using a bunch more memory than normal and is thus
having problems hitting some upper memory limit. Have you tried increasing
the max memory setting? If that is not possible, is it possible to
test with a
smaller data set?
Cheers,
Leif
|
|
From: Paul C. <cas...@au...> - 2004-04-07 23:09:11
|
Yep - looks like pushing for a 1.4.x upgrade is the way to go. Our app is a government GIS roads mapping tool for an Australian state (that's as much as I can say). On startup it loads all the data relating to each layer of the map (eg freeways / local raods / railway lines etc etc) into memory so that they are all there ready for 300+ users to turn on/off whichever layer(s) they want - when they want. I have a feeling that some of the problems with our app center around the printing method used. When the app was developed (around 5 years ago) Java printing was slow and problematic (according to code comments) and so the app generates a postscript or HPGL file using an OS installed print driver, then Java calls the Windows print command to send the file to the user's network printer. Since setting the min/max RAM to different values I've been able to get some more pieces to the puzzle when the app crashes. A couple of days ago someone ran a plot (print) and the system appeared to get into an infinite loop. The Windows CPU time for the Java process kept ticking over (second for second) - although CPU for the process was only 6-7% (which indicates that it's a complicated loop - possibly between the client and server - there was no network activity between the app server and the database server which have a dedicated network card). However, RAM for the Java process steadily increased towards the maximum from about 1300MB when I first saw it to 1730MB before the client requested a restart. I wanted to let it go till it crashed, but the client comes first. The app was initially written to transfer files from the server to the user's PC via FTP (yuck!), which I've scrapped and now just uses Tomcat (which was in use for another component of the app anyway). I'm thinking that the whole approach to printing needs to be overhauled, and I've read somewhere that 1.4.x has a new print api. Any ideas / pointers / suggestions (from anyone)? Has any reader had experience with network printing from Java? Any feedback would be greatly appreciated (as always). Regards, Paul Casanova |---------+----------------------------------------> | | Leif Mortenson | | | <le...@ta...> | | | Sent by: | | | wra...@li...| | | ceforge.net | | | | | | | | | 08/04/2004 12:09 AM | | | Please respond to | | | wrapper-user | |---------+----------------------------------------> >--------------------------------------------------------------------------------------------------------------| | | | To: wra...@li... | | cc: | | Subject: Re: [Wrapper-user] JVM hang - Xms / Xmx | >--------------------------------------------------------------------------------------------------------------| Paul, Paul Casanova wrote: >I tried adding -Xrunhprof:depth=8 to the config file, but the JVM always >suffers a runtime error r6017 ("unexpected multithread lock error") and >restarts. Any ideas? > > Doing a Google search on that error message. It looks like a C runtime error. In other words, Java is having a bad day. Searching the Sun Bug database, the following issue came up: http://developer.java.sun.com/developer/bugParade/bugs/4724252.html That bug says it has been fixed in Java 1.4.1. But the original post says that it not been a problem in 1.3.1_04. That just means that their app had not been able to reproduce it however. >Tried starting with initial heap size of 100MB and it took around the same >time to start as normal, but when I added the -Xrun... it started doing >full GC's when starting up the app, so it took longer. Mind you, before I >added -Xrun... it was doing around 10 GC's (not full) during startup with >100MB initial!! > >Does -Xrunhprof usually have such overhead / impact? > > Profiling takes a bunch of memory and also has a very noticeable effect of performance. Depending on the application of course, but it is usually still usable if you normally have enough CPU / Memory to spare. Most likely your app is using a bunch more memory than normal and is thus having problems hitting some upper memory limit. Have you tried increasing the max memory setting? If that is not possible, is it possible to test with a smaller data set? Cheers, Leif ------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ Wrapper-user mailing list Wra...@li... https://lists.sourceforge.net/lists/listinfo/wrapper-user |
|
From: Leif M. <le...@ta...> - 2004-04-08 02:31:09
|
Paul, Paul Casanova wrote: >Yep - looks like pushing for a 1.4.x upgrade is the way to go. > > The bug states that the problem was added in 1.4.0 and then fixed in 1.4.1 so I am not sure it is the exact same problem. The bug does state that the JVM starts leaking memory until it finally crashes. That does sound like what you saw below. >Our app is a government GIS roads mapping tool for an Australian state >(that's as much as I can say). On startup it loads all the data relating >to each layer of the map (eg freeways / local raods / railway lines etc >etc) into memory so that they are all there ready for 300+ users to turn >on/off whichever layer(s) they want - when they want. > >I have a feeling that some of the problems with our app center around the >printing method used. When the app was developed (around 5 years ago) Java >printing was slow and problematic (according to code comments) and so the >app generates a postscript or HPGL file using an OS installed print driver, >then Java calls the Windows print command to send the file to the user's >network printer. Since setting the min/max RAM to different values I've >been able to get some more pieces to the puzzle when the app crashes. A >couple of days ago someone ran a plot (print) and the system appeared to >get into an infinite loop. The Windows CPU time for the Java process kept >ticking over (second for second) - although CPU for the process was only >6-7% (which indicates that it's a complicated loop - possibly between the >client and server - there was no network activity between the app server >and the database server which have a dedicated network card). However, RAM >for the Java process steadily increased towards the maximum from about >1300MB when I first saw it to 1730MB before the client requested a restart. >I wanted to let it go till it crashed, but the client comes first. > >The app was initially written to transfer files from the server to the >user's PC via FTP (yuck!), which I've scrapped and now just uses Tomcat >(which was in use for another component of the app anyway). > >I'm thinking that the whole approach to printing needs to be overhauled, >and I've read somewhere that 1.4.x has a new print api. Any ideas / >pointers / suggestions (from anyone)? > >Has any reader had experience with network printing from Java? Any >feedback would be greatly appreciated (as always). > > I have not had any experience with Java printing. But a quick search returned this: http://www.javaworld.com/javaworld/jw-10-2000/jw-1020-print.html It is part 1 in a multi part series. Links to additional parts are at the bottom of the page. Here is the JSR on the Unified Print API: http://www.jcp.org/en/jsr/detail?id=6 Cheers, Leif |
