Re: Web App Mirroring

I like the idea of Prevayler simply spawning a mirror from a remote
server, including the code.

For that Prevayler would have to know about and control the classpath.
That is also good for making sure transactions are replayed and
snapshots are read using the correct code. ;)

I am currently working on a startup that will require mirroring if our
site usage ramps up as we hope. I have never user app server
clustering, so I cannot say. I suggest you get more experience playing
with prevayler before we continue discussing. Some of the things you
say dont make any sense.

You might want to get in touch with the SolidStateHttp project folks.
https://github.com/bihaiko/SolidStateHttp They intend to implement the
ideas previously discussed in this list as HttPrevayler: prevalence at
the http level, so you can use any web framework and simply forget
about Transactions, etc...

Klaus


On Fri, Oct 14, 2011 at 9:44 AM, Naveen Chawla <[email protected]> wrote:
> The simplest solution for my app seems like Java app server replication
> across 2 computers using the app server's built-in clustering, if it works.
> This is because it seems simpler to replicate the whole web app for
> failover/switch-off availability, as I would presume this does, rather than
> just the prevalent system. Have you ever used this? But if
> Prevayler-mirroring does indeed intend to replicate the whole web app, which
> I sincerely hope it does (by for example, demanding the web app's path
> is passed to Prevayler's "makeReadyForMirroring" method), then I'd love to
> be able to go to the console of any new or upgraded computer, simply
> download a utility app from Prevayler to it, and type:
>
> prevayler_produce_mirror myApp
>
> which automatically listens on its all connections for a
> "Prevayler-mirroring-ready" system named "myApp" on other
> computers, downloads and produces a master-capable-mirror on this computer
> and launches it, and you just carry on, rather than having to code and
> build this utility application every time. That's because this would
> be a standard procedure anyway, the only variable being what you substitute
> for "myApp". The benefit of this over the aforementioned
> app-server-replication might be that I might be able to use
> a different/better app server on each new mirror?
> Sorry if these are stupid ideas. But they seem to make sense to me at the
> moment.
>
> I just can't see how true failover replication can be achieved with e.g. 2
> computers without replicating the entire web app. It's like 1 + 1 =/= 3 to
> me. Am I wrong?
>
> On 13 October 2011 23:39, Klaus Wuestefeld <[email protected]> wrote:
>>
>> Prevayler mirroring is demoed with demo2 inside the Prevayler distro.
>> Run it with mirroring and let me know.
>>
>> It is only a proof of concept, though. Some simple things would still
>> have to be implemented: if the master crashes and restarts, for
>> example, the clients will not attempt to reconnect.
>>
>> Klaus
>>
>>
>> On Thu, Oct 13, 2011 at 2:50 PM, Naveen Chawla <[email protected]>
>> wrote:
>> > That's brilliant Klaus, thanks! This'll choose the G1 collector in Java
>> > 7 by
>> > default, right?
>> >
>> > I have no idea how big my heap will become. But I'd like to prepare for
>> > sky
>> > being the limit. Which is also why I desperately want to be able to
>> > hot-swap/upgrade machines to add RAM seamlessly while my app is still
>> > running. This was planned for Prevayler 3, right? Should I wait for this
>> > or
>> > should I do clustering with an app server to do replication? Do these
>> > work
>> > ok for this purpose (i.e. just being able to literally switch off one of
>> > the
>> > computers any time, chuck some RAM in, and carry on, repeat with the
>> > other
>> > computer, or just swap for a new computers, one at a time, with the
>> > right
>> > configuration)? Which Java app server is best for this? Or should I wait
>> > for
>> > Prevayler 3?
>> >
>> > On 13 October 2011 15:45, Klaus Wuestefeld <[email protected]> wrote:
>> >>
>> >> The Java VM since Java5 has simplified GC options that they call
>> >> "ergonomic". You can choose to minimize any two out of the following
>> >> three things, according to your needs:
>> >>
>> >>  1) Pause time for exclusive GC.
>> >>  2) % CPU for GC em parallel with your application.
>> >>  3) Use of memory.
>> >>
>> >> Java5:
>> >>
>> >> http://download.oracle.com/javase/1.5.0/docs/guide/vm/gc-ergonomics.html
>> >> Java7:
>> >>
>> >> http://download.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html
>> >>
>> >> My choice of VM options for prevalent system servers is:
>> >>  -server
>> >>  -XX:MaxGCPauseMillis=500
>> >>  -XX:GCTimeRatio=3
>> >>
>> >> MaxGCPauseMillis=500 gives the VM a target of not pausing for more
>> >> than 500 millis for GC. You can use less. It will simply increase the
>> >> pressure on CPU usage and RAM.
>> >>
>> >> GCTimeRatio=3 will give the parallel garbage collector 25% of CPU (75%
>> >> for the app) before it starts allocating more heap. The default is 99
>> >> which gives only 1% of CPU to the collector and 99% to the app. With
>> >> prevalent systems the problem is normally not CPU but RAM, so it is
>> >> good to give the collector ample CPU.
>> >>
>> >> Given the above, the VM will apply a pretty cool adaptive heap
>> >> generation allocation strategy:
>> >>
>> >> "1. If the GC pause time is greater than the pause time goal then
>> >> reduce the generations sizes to better attain the goal.
>> >> 2. If the pause time goal is being met then consider the application's
>> >> throughput goal. If the application's throughput goal is not being
>> >> met, then increase the sizes of the generations to better attain the
>> >> goal.
>> >> 3. If both the pause time goal and the throughput goal are being met,
>> >> then the size of the generations are decreased to reduce footprint."
>> >>
>> >> ;)
>> >>
>> >> Klaus
>> >> PS: Naveen, did you answer Ralph's question on how big your heap will
>> >> be?
>> >>
>> >>
>> >> On Thu, Oct 13, 2011 at 9:05 AM, Diego Fincatto
>> >> <[email protected]> wrote:
>> >> > Maybe terracotta can help!
>> >> > http://www.terracotta.org/products/bigmemory
>> >> >
>> >> >
>> >> > On Thu, Oct 13, 2011 at 8:53 AM, Naveen Chawla
>> >> > <[email protected]>
>> >> > wrote:
>> >> >> I agree, there's no way forcing garbage collection is "pauseless" as
>> >> >> such,
>> >> >> I'm just unsure if it could be a cheap workaround for "slow down
>> >> >> now,
>> >> >> save
>> >> >> later". It turns out ordinary Java has garbage collector options G1
>> >> >> and
>> >> >> CMS,
>> >> >> either of which may altogether avoid long pauses, I don't know.
>> >> >>
>> >> >> So I'm thinking G1 or CMS gc, with heap size fixed to the maximum
>> >> >> comfortable RAM (unless G1 or CMS still triggers long pauses with
>> >> >> large
>> >> >> memories). Anyone recommend different?
>> >> >>
>> >> >> On 13 October 2011 10:54, Ralph Johnson <[email protected]> wrote:
>> >> >>>
>> >> >>> If you want to collect more frequently, just make the size of your
>> >> >>> memory partitions smaller.   The only reason to ask for a
>> >> >>> collection
>> >> >>> is because you know that this is a good time to do the work, i.e.
>> >> >>> you
>> >> >>> aren't doing anything else.
>> >> >>>
>> >> >>> If you could get a pauseless g.c. by manual intervention then they
>> >> >>> wouldn't be so hard to make.
>> >> >>>
>> >> >>> -Ralph
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> ------------------------------------------------------------------------------
>> >> >>> All the data continuously generated in your IT infrastructure
>> >> >>> contains
>> >> >>> a
>> >> >>> definitive record of customers, application performance, security
>> >> >>> threats, fraudulent activity and more. Splunk takes this data and
>> >> >>> makes
>> >> >>> sense of it. Business sense. IT sense. Common sense.
>> >> >>> http://p.sf.net/sfu/splunk-d2d-oct
>> >> >>> _______________________________________________
>> >> >>> To unsubscribe go to the end of this page:
>> >> >>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> >> >>> _______________________________________________
>> >> >>> "Databases in Memoriam" -- http://www.prevayler.org
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> All the data continuously generated in your IT infrastructure
>> >> >> contains
>> >> >> a
>> >> >> definitive record of customers, application performance, security
>> >> >> threats, fraudulent activity and more. Splunk takes this data and
>> >> >> makes
>> >> >> sense of it. Business sense. IT sense. Common sense.
>> >> >> http://p.sf.net/sfu/splunk-d2d-oct
>> >> >> _______________________________________________
>> >> >> To unsubscribe go to the end of this page:
>> >> >> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> >> >> _______________________________________________
>> >> >> "Databases in Memoriam" -- http://www.prevayler.org
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------------------------------------------------
>> >> > All the data continuously generated in your IT infrastructure
>> >> > contains a
>> >> > definitive record of customers, application performance, security
>> >> > threats, fraudulent activity and more. Splunk takes this data and
>> >> > makes
>> >> > sense of it. Business sense. IT sense. Common sense.
>> >> > http://p.sf.net/sfu/splunk-d2d-oct
>> >> > _______________________________________________
>> >> > To unsubscribe go to the end of this page:
>> >> > http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> >> > _______________________________________________
>> >> > "Databases in Memoriam" -- http://www.prevayler.org
>> >> >
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> All the data continuously generated in your IT infrastructure contains
>> >> a
>> >> definitive record of customers, application performance, security
>> >> threats, fraudulent activity and more. Splunk takes this data and makes
>> >> sense of it. Business sense. IT sense. Common sense.
>> >> http://p.sf.net/sfu/splunk-d2d-oct
>> >> _______________________________________________
>> >> To unsubscribe go to the end of this page:
>> >> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> >> _______________________________________________
>> >> "Databases in Memoriam" -- http://www.prevayler.org
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > All the data continuously generated in your IT infrastructure contains a
>> > definitive record of customers, application performance, security
>> > threats, fraudulent activity and more. Splunk takes this data and makes
>> > sense of it. Business sense. IT sense. Common sense.
>> > http://p.sf.net/sfu/splunk-d2d-oct
>> > _______________________________________________
>> > To unsubscribe go to the end of this page:
>> > http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> > _______________________________________________
>> > "Databases in Memoriam" -- http://www.prevayler.org
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure contains a
>> definitive record of customers, application performance, security
>> threats, fraudulent activity and more. Splunk takes this data and makes
>> sense of it. Business sense. IT sense. Common sense.
>> http://p.sf.net/sfu/splunk-d2d-oct
>> _______________________________________________
>> To unsubscribe go to the end of this page:
>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org