|
From: Paul M. <pm...@co...> - 2007-06-23 13:47:23
|
While running the 1.3.3-1 release, I've seen a number of "OutOfMemeoryError" exceptions happen. This was also very prevalent in previous releases. To avoid this, we have deployed multiple disks in our boxes with a raid0 array dedicated solely to writing rrd data. But still the problem occurs. =20 In a past thread, David wrote: =20 This I suspect to be the problem. I've recently seen this at another site. We have a poor implementation of of a call used in Hibernate to load up all a node's data in the collector because we currently have no transaction boundary in the collector code (I know, lot of mumbo jumbo). So when a node has a *lot* of interfaces like these, then there is the potential for an extraordinary amount of memory to be used when, if implemented correctly, we wouldn't have this problem. We're going to have to address it before we release 1.3.3. =20 -David =20 =20 =20 Does anyone know if this has been addressed? =20 =20 In the output.log I see the following: =20 SPermGen: 42478K->42478K(42496K)], 3.1469600 secs] 21930.975: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.4699820 secs] 21934.446: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.1089670 secs] 21937.555: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9852640 secs] 21940.544: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9777510 secs] 21943.530: [Full GC [PSYoungGen: 181952K->181939K(260224K)] [PSOldGen: 699072K->699072K(699072K)] 881024K->881011K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9809020 secs] 21946.524: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699072K->699072K(699072K)] 881024K->881024K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9602660 secs] 21949.484: [Full GC [PSYoungGen: 181952K->181941K(260224K)] [PSOldGen: 699072K->699071K(699072K)] 881024K->881013K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9825690 secs] Exception in thread "Timer-0" java.lang.OutOfMemoryError: Java heap space 21952.483: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.1370260 secs] 21955.620: [Full GC [PSYoungGen: 181952K->181941K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881013K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.6898070 secs] 21959.311: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9579090 secs] 21962.269: [Full GC [PSYoungGen: 181952K->181946K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881018K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9631330 secs] 21965.233: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9715310 secs] 21968.205: [Full GC [PSYoungGen: 181952K->181948K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881020K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9725410 secs] 21971.178: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9689150 secs] 21974.147: [Full GC [PSYoungGen: 181952K->181949K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881021K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9716320 secs] 21977.120: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9771430 secs] 21980.097: [Full GC [PSYoungGen: 181952K->181949K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881021K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9788080 secs] 21983.076: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.0162600 secs] 21986.093: [Full GC [PSYoungGen: 181952K->181951K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 3.0092040 secs] 21989.103: [Full GC [PSYoungGen: 181952K->181952K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9578070 secs] 21992.060: [Full GC [PSYoungGen: 181952K->181951K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9662550 secs] 21995.027: [Full GC [PSYoungGen: 181951K->181951K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9741520 secs] 21998.002: [Full GC [PSYoungGen: 181951K->181951K(260224K)] [PSOldGen: 699071K->699071K(699072K)] 881023K->881023K(959296K) [PSPermGen: 42478K->42478K(42496K)], 2.9815980 secs] # # An unexpected error has been detected by HotSpot Virtual Machine: # # SIGSEGV (0xb) at pc=3D0x0000002a9591549e, pid=3D16039, = tid=3D1113024864 # # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_10-b03 mixed mode) # Problematic frame: # V [libjvm.so+0x3af49e] # # An error report file with more information is saved as hs_err_pid16039.log # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # =20 =20 collectd.log also shows an out of memory error: =20 java.lang.OutOfMemoryError: Java heap space =20 =20 Paul |