Re: GC for Large Heaps

The simplest solution for my app seems like Java app server replication
across 2 computers using the app server's built-in clustering, if it works.
This is because it seems simpler to replicate the whole web app for
failover/switch-off availability, as I would presume this does, rather than
just the prevalent system. Have you ever used this? But if
Prevayler-mirroring does indeed intend to replicate the whole web app, which
I sincerely hope it does (by for example, demanding the web app's path
is passed to Prevayler's "makeReadyForMirroring" method), then I'd love to
be able to go to the console of any new or upgraded computer, simply
download a utility app from Prevayler to it, and type:

prevayler_produce_mirror myApp

which automatically listens on its all connections for a
"Prevayler-mirroring-ready" system named "myApp" on other
computers, downloads and produces a master-capable-mirror on this computer
and launches it, and you just carry on, rather than having to code and
build this utility application every time. That's because this would
be a standard procedure anyway, the only variable being what you substitute
for "myApp". The benefit of this over the aforementioned
app-server-replication might be that I might be able to use
a different/better app server on each new mirror?

Sorry if these are stupid ideas. But they seem to make sense to me at the
moment.

I just can't see how true failover replication can be achieved with e.g. 2
computers without replicating the entire web app. It's like 1 + 1 =/= 3 to
me. Am I wrong?


On 13 October 2011 23:39, Klaus Wuestefeld <[email protected]> wrote:

> Prevayler mirroring is demoed with demo2 inside the Prevayler distro.
> Run it with mirroring and let me know.
>
> It is only a proof of concept, though. Some simple things would still
> have to be implemented: if the master crashes and restarts, for
> example, the clients will not attempt to reconnect.
>
> Klaus
>
>
> On Thu, Oct 13, 2011 at 2:50 PM, Naveen Chawla <[email protected]>
> wrote:
> > That's brilliant Klaus, thanks! This'll choose the G1 collector in Java 7
> by
> > default, right?
> >
> > I have no idea how big my heap will become. But I'd like to prepare for
> sky
> > being the limit. Which is also why I desperately want to be able to
> > hot-swap/upgrade machines to add RAM seamlessly while my app is still
> > running. This was planned for Prevayler 3, right? Should I wait for this
> or
> > should I do clustering with an app server to do replication? Do these
> work
> > ok for this purpose (i.e. just being able to literally switch off one of
> the
> > computers any time, chuck some RAM in, and carry on, repeat with the
> other
> > computer, or just swap for a new computers, one at a time, with the right
> > configuration)? Which Java app server is best for this? Or should I wait
> for
> > Prevayler 3?
> >
> > On 13 October 2011 15:45, Klaus Wuestefeld <[email protected]> wrote:
> >>
> >> The Java VM since Java5 has simplified GC options that they call
> >> "ergonomic". You can choose to minimize any two out of the following
> >> three things, according to your needs:
> >>
> >>  1) Pause time for exclusive GC.
> >>  2) % CPU for GC em parallel with your application.
> >>  3) Use of memory.
> >>
> >> Java5:
> >>
> http://download.oracle.com/javase/1.5.0/docs/guide/vm/gc-ergonomics.html
> >> Java7:
> >>
> http://download.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html
> >>
> >> My choice of VM options for prevalent system servers is:
> >>  -server
> >>  -XX:MaxGCPauseMillis=500
> >>  -XX:GCTimeRatio=3
> >>
> >> MaxGCPauseMillis=500 gives the VM a target of not pausing for more
> >> than 500 millis for GC. You can use less. It will simply increase the
> >> pressure on CPU usage and RAM.
> >>
> >> GCTimeRatio=3 will give the parallel garbage collector 25% of CPU (75%
> >> for the app) before it starts allocating more heap. The default is 99
> >> which gives only 1% of CPU to the collector and 99% to the app. With
> >> prevalent systems the problem is normally not CPU but RAM, so it is
> >> good to give the collector ample CPU.
> >>
> >> Given the above, the VM will apply a pretty cool adaptive heap
> >> generation allocation strategy:
> >>
> >> "1. If the GC pause time is greater than the pause time goal then
> >> reduce the generations sizes to better attain the goal.
> >> 2. If the pause time goal is being met then consider the application's
> >> throughput goal. If the application's throughput goal is not being
> >> met, then increase the sizes of the generations to better attain the
> >> goal.
> >> 3. If both the pause time goal and the throughput goal are being met,
> >> then the size of the generations are decreased to reduce footprint."
> >>
> >> ;)
> >>
> >> Klaus
> >> PS: Naveen, did you answer Ralph's question on how big your heap will
> be?
> >>
> >>
> >> On Thu, Oct 13, 2011 at 9:05 AM, Diego Fincatto
> >> <[email protected]> wrote:
> >> > Maybe terracotta can help!
> >> > http://www.terracotta.org/products/bigmemory
> >> >
> >> >
> >> > On Thu, Oct 13, 2011 at 8:53 AM, Naveen Chawla <[email protected]
> >
> >> > wrote:
> >> >> I agree, there's no way forcing garbage collection is "pauseless" as
> >> >> such,
> >> >> I'm just unsure if it could be a cheap workaround for "slow down now,
> >> >> save
> >> >> later". It turns out ordinary Java has garbage collector options G1
> and
> >> >> CMS,
> >> >> either of which may altogether avoid long pauses, I don't know.
> >> >>
> >> >> So I'm thinking G1 or CMS gc, with heap size fixed to the maximum
> >> >> comfortable RAM (unless G1 or CMS still triggers long pauses with
> large
> >> >> memories). Anyone recommend different?
> >> >>
> >> >> On 13 October 2011 10:54, Ralph Johnson <[email protected]> wrote:
> >> >>>
> >> >>> If you want to collect more frequently, just make the size of your
> >> >>> memory partitions smaller.   The only reason to ask for a collection
> >> >>> is because you know that this is a good time to do the work, i.e.
> you
> >> >>> aren't doing anything else.
> >> >>>
> >> >>> If you could get a pauseless g.c. by manual intervention then they
> >> >>> wouldn't be so hard to make.
> >> >>>
> >> >>> -Ralph
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> ------------------------------------------------------------------------------
> >> >>> All the data continuously generated in your IT infrastructure
> contains
> >> >>> a
> >> >>> definitive record of customers, application performance, security
> >> >>> threats, fraudulent activity and more. Splunk takes this data and
> >> >>> makes
> >> >>> sense of it. Business sense. IT sense. Common sense.
> >> >>> http://p.sf.net/sfu/splunk-d2d-oct
> >> >>> _______________________________________________
> >> >>> To unsubscribe go to the end of this page:
> >> >>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >> >>> _______________________________________________
> >> >>> "Databases in Memoriam" -- http://www.prevayler.org
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> All the data continuously generated in your IT infrastructure
> contains
> >> >> a
> >> >> definitive record of customers, application performance, security
> >> >> threats, fraudulent activity and more. Splunk takes this data and
> makes
> >> >> sense of it. Business sense. IT sense. Common sense.
> >> >> http://p.sf.net/sfu/splunk-d2d-oct
> >> >> _______________________________________________
> >> >> To unsubscribe go to the end of this page:
> >> >> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >> >> _______________________________________________
> >> >> "Databases in Memoriam" -- http://www.prevayler.org
> >> >>
> >> >>
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > All the data continuously generated in your IT infrastructure contains
> a
> >> > definitive record of customers, application performance, security
> >> > threats, fraudulent activity and more. Splunk takes this data and
> makes
> >> > sense of it. Business sense. IT sense. Common sense.
> >> > http://p.sf.net/sfu/splunk-d2d-oct
> >> > _______________________________________________
> >> > To unsubscribe go to the end of this page:
> >> > http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >> > _______________________________________________
> >> > "Databases in Memoriam" -- http://www.prevayler.org
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> All the data continuously generated in your IT infrastructure contains a
> >> definitive record of customers, application performance, security
> >> threats, fraudulent activity and more. Splunk takes this data and makes
> >> sense of it. Business sense. IT sense. Common sense.
> >> http://p.sf.net/sfu/splunk-d2d-oct
> >> _______________________________________________
> >> To unsubscribe go to the end of this page:
> >> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >> _______________________________________________
> >> "Databases in Memoriam" -- http://www.prevayler.org
> >
> >
> >
> ------------------------------------------------------------------------------
> > All the data continuously generated in your IT infrastructure contains a
> > definitive record of customers, application performance, security
> > threats, fraudulent activity and more. Splunk takes this data and makes
> > sense of it. Business sense. IT sense. Common sense.
> > http://p.sf.net/sfu/splunk-d2d-oct
> > _______________________________________________
> > To unsubscribe go to the end of this page:
> > http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> > _______________________________________________
> > "Databases in Memoriam" -- http://www.prevayler.org
> >
> >
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org