Re: paper on Prevayler

Ralph Johnson wrote:

> Thanks a lot for your comments! I hadn't connected prevalence
> with consensus, but of course they are closely related. My PhD
> work 30 years ago was on distributed consensus but I quit
> paying much attention to that area after the 80s were over. I
> studied Lamport's work closely for awhile, but never figured
> out Paxos.

The fact that so many people have "never figured out Paxos" was
the primary motivation behind Raft. :) You  might enjoy checking
it out. I haven't had a chance to use it yet, but I'm starting up
a study group here at work.

> One of the reasons I didn't connect them was that the prevalent
> systems I have built have never been distributed. The only
> system using prevalence that I have used and that was
> replicated was Croquet (which I plan to include as an example).
> I'm working on a system now that will become replicated, but
> prevalence is just a tiny part of this system, and (for me) not
> the most interesting. One of the big advantages of prevalence
> is that it is so simple. "Simple" means you can ignore it most
> of the time.

Yeah, my own usage of Prevayler has only been for prototype or
early-stage start-up apps, where "simple" is hugely valuable.
Prevayler's appeal in that context is less about how fast it is
at run-time and more about how delightful it is to program with!
However, for serious production usage I really want true
clustering (not just master-slave mirroring), which has made it
hard for me to advocate strongly for Prevayler.

> Distributed consensus tends to focus on fault-tolerance. When
> people talk about "prevalence" they usually mean making things
> fast and simple by avoiding disk. So, though these have a lot
> in common, they tend to emphasize different aspects.

Sure -- not so much avoiding disk altogether, but journaling to
disk in parallel with in-memory processing. Paxos (in practice)
and Raft (in theory and practice) are all about distributed logs,
so they seem a very natural "next step" for Prevayler, simply
replacing the local-disk journal with a replicated one. It's not
_quite_ so easy as that; Prevayler's internals would need some
rearrangement. But conceptually it's very close.

I kept hoping that some day I'd be able to incorporate Paxos into
Prevayler, but there were never any good open-source Java
implementations of Paxos available and I never had the time to
write one myself. Raft is exciting because it's designed to be
implementable, and therefore there are already a bunch of decent
open-source implementations popping up.

> I agree that the need for nondeterminism is as important as the
> limitation on memory. One of my weaknesses when writing is a
> tendency to give priorities, such as "the main limitation",
> when they are not real. On most of the projects that I used
> prevalence, memory wasn't a problem. I ran a server for years
> that always ran on the oldest machine in my lab, and when it
> started to get slow because it was running out of memory, I'd
> move it to a somewhat newer machine with more memory, and throw
> out the old server. When it was obvious that we had more data
> than would fit in memory, we would usually use a real database.

I didn't mean to pick on you for it, but the focus on memory was
always a pet peeve of mine back when Prevayler was getting heated
attention. To me, the two biggest _actual_ practical problems
with Prevayler are (a) the difficulty of determinism and (b) the
lack of true clustering -- both of which are solvable with a bit
of R&D. But _no one_ among Prevayler's detractors ever seemed to
care about those issues; they all focused on "you're stupid if
you think you can fit in memory" or "encapsulating transactions
as objects is silly."

I think history has shown (and your own experience seems to
agree!) that memory is _not_ a problem -- _everyone_ in serious
distributed apps is keeping everything in memory these days --
and the Transaction Pattern fits in very nicely with the growing
emphasis on event-based architectures, asynchronous messaging,
and eventual consistency. Those early architectural principles of
Prevayler have been totally vindicated in the intervening decade.
And that's why I don't think it's fair to describe memory as even
"a limitation," much less "the main limitation"! :)

> > The statement in the draft that "not writing queries in the
> > journal saves a little time when the query is executed, and
> > saves a lot of time when the journal is replayed," seems
> > backwards to me. Executing a query is usually tiny
> > (nanoseconds), whereas writing it to disk is huge
> > (centiseconds). Is your point that _lots_ of queries bloat
> > the journal?
>
> I'm not sure exactly what my point was. When I said "executing
> a query" I meant to include the time to write it to the
> journal. Usually there are a lot more queries than there are
> transactions that change state. I think what I really meant was
> that the time saved would probably be more noticeable when
> replaying the journal than when creating it in the first place,
> but right now I don't remember why I thought that.

Yeah, I expect the ratio of queries to transactions to be
something like 10:1 or 100:1 (though I'm not sure if that's
empirical or just intuitive).

Of course, "noticeable" depends on who's doing the noticing. :)

Unnecessarily journaling queries will definitely be noticed
during recovery, because the journal will be 10x or 100x the size
it needs to be, which will slow down both reading from the file
and deserializing the transactions. Whether that's actually a
problem depends on how often you're taking snapshots. End-users
won't notice unless it keeps the system off-line at a time when
they're actually trying to use it!

When a client first submits a query, whether it's noticeable
(subjectively) depends on how fast the rest of the system is. But
journaling a query always adds significant latency in the
multi-centisecond range (used to be 100-200ms with hard disk
drives; might be 10ms with solid-state drives). Prevayler batches
transactions to achieve higher throughput, but that latency is
unavoidable. Users _expect_ >200ms for _writes_ -- I seem to
remember some usability studies actually showing users getting
_worried_ if clicking a "save" button took much less than that!
But users expect navigating around an app (triggering queries
along the way) to be snappy, ~100ms per click. If you're serving
up a web app, the network latency alone is already eating up that
much, or close to it, so adding another 100ms to journal each
query can definitely mean the difference between "this app feels
pretty snappy" and "this ... app ... is ... so ... slow ...".

Cheers,
Justin

------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org