Re: [Prevayler-discussion] GC for Large Heaps

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

That's brilliant Klaus, thanks! This'll choose the G1 collector in Java 7 by
default, right?

I have no idea how big my heap will become. But I'd like to prepare for sky
being the limit. Which is also why I desperately want to be able to
hot-swap/upgrade machines to add RAM seamlessly while my app is still
running. This was planned for Prevayler 3, right? Should I wait for this or
should I do clustering with an app server to do replication? Do these work
ok for this purpose (i.e. just being able to literally switch off one of the
computers any time, chuck some RAM in, and carry on, repeat with the other
computer, or just swap for a new computers, one at a time, with the right
configuration)? Which Java app server is best for this? Or should I wait for
Prevayler 3?

On 13 October 2011 15:45, Klaus Wuestefeld <ma...@kl...> wrote:

> The Java VM since Java5 has simplified GC options that they call
> "ergonomic". You can choose to minimize any two out of the following
> three things, according to your needs:
>
>  1) Pause time for exclusive GC.
>  2) % CPU for GC em parallel with your application.
>  3) Use of memory.
>
> Java5:
> http://download.oracle.com/javase/1.5.0/docs/guide/vm/gc-ergonomics.html
> Java7:
> http://download.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html
>
> My choice of VM options for prevalent system servers is:
>  -server
>  -XX:MaxGCPauseMillis=500
>  -XX:GCTimeRatio=3
>
> MaxGCPauseMillis=500 gives the VM a target of not pausing for more
> than 500 millis for GC. You can use less. It will simply increase the
> pressure on CPU usage and RAM.
>
> GCTimeRatio=3 will give the parallel garbage collector 25% of CPU (75%
> for the app) before it starts allocating more heap. The default is 99
> which gives only 1% of CPU to the collector and 99% to the app. With
> prevalent systems the problem is normally not CPU but RAM, so it is
> good to give the collector ample CPU.
>
> Given the above, the VM will apply a pretty cool adaptive heap
> generation allocation strategy:
>
> "1. If the GC pause time is greater than the pause time goal then
> reduce the generations sizes to better attain the goal.
> 2. If the pause time goal is being met then consider the application's
> throughput goal. If the application's throughput goal is not being
> met, then increase the sizes of the generations to better attain the
> goal.
> 3. If both the pause time goal and the throughput goal are being met,
> then the size of the generations are decreased to reduce footprint."
>
> ;)
>
> Klaus
> PS: Naveen, did you answer Ralph's question on how big your heap will be?
>
>
> On Thu, Oct 13, 2011 at 9:05 AM, Diego Fincatto
> <die...@gm...> wrote:
> > Maybe terracotta can help!
> > http://www.terracotta.org/products/bigmemory
> >
> >
> > On Thu, Oct 13, 2011 at 8:53 AM, Naveen Chawla <nav...@gm...>
> wrote:
> >> I agree, there's no way forcing garbage collection is "pauseless" as
> such,
> >> I'm just unsure if it could be a cheap workaround for "slow down now,
> save
> >> later". It turns out ordinary Java has garbage collector options G1 and
> CMS,
> >> either of which may altogether avoid long pauses, I don't know.
> >>
> >> So I'm thinking G1 or CMS gc, with heap size fixed to the maximum
> >> comfortable RAM (unless G1 or CMS still triggers long pauses with large
> >> memories). Anyone recommend different?
> >>
> >> On 13 October 2011 10:54, Ralph Johnson <jo...@cs...> wrote:
> >>>
> >>> If you want to collect more frequently, just make the size of your
> >>> memory partitions smaller.   The only reason to ask for a collection
> >>> is because you know that this is a good time to do the work, i.e. you
> >>> aren't doing anything else.
> >>>
> >>> If you could get a pauseless g.c. by manual intervention then they
> >>> wouldn't be so hard to make.
> >>>
> >>> -Ralph
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> All the data continuously generated in your IT infrastructure contains
> a
> >>> definitive record of customers, application performance, security
> >>> threats, fraudulent activity and more. Splunk takes this data and makes
> >>> sense of it. Business sense. IT sense. Common sense.
> >>> http://p.sf.net/sfu/splunk-d2d-oct
> >>> _______________________________________________
> >>> To unsubscribe go to the end of this page:
> >>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >>> _______________________________________________
> >>> "Databases in Memoriam" -- http://www.prevayler.org
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> All the data continuously generated in your IT infrastructure contains a
> >> definitive record of customers, application performance, security
> >> threats, fraudulent activity and more. Splunk takes this data and makes
> >> sense of it. Business sense. IT sense. Common sense.
> >> http://p.sf.net/sfu/splunk-d2d-oct
> >> _______________________________________________
> >> To unsubscribe go to the end of this page:
> >> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >> _______________________________________________
> >> "Databases in Memoriam" -- http://www.prevayler.org
> >>
> >>
> >
> >
> ------------------------------------------------------------------------------
> > All the data continuously generated in your IT infrastructure contains a
> > definitive record of customers, application performance, security
> > threats, fraudulent activity and more. Splunk takes this data and makes
> > sense of it. Business sense. IT sense. Common sense.
> > http://p.sf.net/sfu/splunk-d2d-oct
> > _______________________________________________
> > To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> > _______________________________________________
> > "Databases in Memoriam" -- http://www.prevayler.org
> >
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>