From: Osullivan L. <L.O...@sw...> - 2013-08-15 16:32:49
|
Hi Till, Thanks for this :) As I mentioned before, we don't have a GUI on the server but I believe the JVM visualisers can be run remotely. I'll have to get the Solr Port unblocked.... Garbage collection does appear to be working - I get a gc.log like: 6191.595: [GC 3029222K->2186774K(5227648K), 0.0455620 secs] 6203.290: [GC 3030230K->2189576K(5227648K), 0.0504960 secs] 6211.724: [GC 3033288K->2190686K(5227840K), 0.0486040 secs] 6219.039: [GC 3034398K->2193114K(5227904K), 0.0384730 secs] 6225.964: [GC 3037082K->2195097K(5225984K), 0.0487640 secs] 6234.120: [GC 3039065K->2195519K(5228160K), 0.0359140 secs] 6241.113: [GC 3039551K->2197534K(5227840K), 0.0366620 secs] 6250.560: [GC 3041566K->2199140K(5228416K), 0.0470680 secs] 6256.603: [GC 3043876K->2200811K(5228288K), 0.0479080 secs] 6261.250: [GC 3045547K->2199797K(5228352K), 0.0383680 secs] 6268.816: [GC 3044789K->2206847K(5228480K), 0.0457030 secs] 6275.300: [GC 3051839K->2206550K(5226048K), 0.0481280 secs] 6278.510: [GC 3049110K->2208566K(5227264K), 0.0483090 secs] 6285.131: [GC 3051126K->2211148K(5227072K), 0.0379300 secs] 6289.086: [GC 3053836K->2208923K(5221248K), 0.0613130 secs] 6292.976: [GC 3051611K->2213059K(5227520K), 0.0359410 secs] 6301.405: [GC 3055747K->2213314K(5227136K), 0.0485240 secs] 6309.637: [GC 3056002K->2216701K(5227520K), 0.0356450 secs] 6317.714: [GC 3059901K->2218004K(5227584K), 0.0502940 secs] 6322.573: [GC 3061204K->2219390K(5228096K), 0.0759970 secs] 6327.569: [GC 3063358K->2221726K(5227840K), 0.0373840 secs] 6331.212: [GC 3065694K->2223362K(5228032K), 0.0552920 secs] 6336.438: [GC 3067458K->2224028K(5228032K), 0.0478070 secs] 6342.723: [GC 3068124K->2224962K(5228288K), 0.0450840 secs] 6348.897: [GC 3069442K->2227022K(5228160K), 0.0368760 secs] Cheers, Luke On 08/15/2013 05:11 PM, Till Kinstler wrote: > Am 15.08.2013 17:44, schrieb Osullivan L.: >> Hi Till, >> >> Thanks for your reply :) >> >> Our Java details are as follows: >> >> java version "1.6.0_27" >> OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1) >> OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) > Ok, so you don't have the G1 collector (or only in an "experimental > state"). And we found the Oracle Java distributions somewhat more stable > and "well behaving" than the OpenJDK versions. You can get the latest > Oracle Java here: > http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html > But it is not free (as in freedom) software and is not part of the > Ubuntu package management, so you have to watch out for security issues > and update it manually. > >> Basically, our memory usage climbs higher and higher so that the server >> becomes unstable and starts killing processes by itself. > That rather sounds like garbage collection is not happening at all and > then, when it finally starts, kills the machine? > In principle garbage collection in the "new generation" area of the heap > sets in very frequently (every few minutes) in typical heavily used Solr > instances. Garbage collection in the "old area" should happen less > frequently, but still from time to time... > >> Our index is at most 600,000 and >> I don't think that should require more than 5GB of Ram or am I being naive? > No, I think that should definitely work. Maybe your heap is too big? Set > Xms and Xmx to lower values to make garbage collection work more > frequently but less "forceful"? > >> I have no experience with Java whatsoever so am not sure how best to >> experiment on a live server. Do you think I should try: >> >> JAVA_OPTIONS="-server -d64 -Xms5120m -Xmx5120m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+AggressiveOpts -XX:NewRatio=8 -Xloggc:/var/log/vufind2/gc.log" > Well, give it a try and keep watching it. As I said, maybe with even > less heap space (3 GB?). You can monitor heap usage and garbage > collection visually with a tool like jconsole (part of the Java > development kit) or Visual JVM. If you see nice "sawtooth" patterns > without Solr stopping or going OOM all is fine... After some rounds of > "optimizing" you can usually see when things start to go wrong (for > example in very "hectic" sawtooth patterns)... > > Till > > ------------------------------------------------------------------------------ > Get 100% visibility into Java/.NET code with AppDynamics Lite! > It's a free troubleshooting tool designed for production. > Get down to code-level detail for bottlenecks, with <2% overhead. > Download for free and get started troubleshooting in minutes. > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > _______________________________________________ > Vufind-tech mailing list > Vuf...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-tech |