Re: ACIDity of Prevayler 2.6

Uau!

Considering from past e-mails:

- Atomicity and Consistency have some overlap
- When we say Exceptions, we are usually talking about unexpected events.
Normally either the Commands were written without throwing Exceptions or
there are expected Exceptions that are catched and dealt within the Command.
- The Royal Food Taster provides Rollback with 2 steps:
   - Avoiding a Command that threw an Exception (unexpected event) to be
written to the journal
   - Avoiding partial changes by discarding temporary data used to check
the Commands

We have then two grades (or levels) of Atomicity+Consistency:

A) Without Royal Food Taster: Exceptions (expected and unexpected) happen
and we let the business code deal with them. Advantages: simple and fast.
Disadvantage: being some Exceptions unexpected events, maybe it is not
possible to guarantee that they are deterministic, and not being
deterministic they can really cause trouble in the system based on
Prevaylers definition.

B) With Royal Food Taster: Exceptions (unexpected) cause Rollback.
Advantage: "better" Atomicity+Consistency. Disadvantages: less performance
and more memory needed.

My opinion:

I think the choice should be given to the developers, offering both options.

And I like the idea of trying to do better than Royal Food Taster. I will
give a draft of another idea of Rollback to help with the discussion,
trying to address the 2 steps described above:

Another type of Command called RollbackableCommand with 3 methods:

boolean prepareExecution(); // Prepare to execute, validate pre-conditions,
save data for rollback
void execute(); //Execute the changes, catch and deal with all expected
Exceptions
void rollback(); //Rollback possible changes

And Prevayler would do something like this:

try {
   if (command.prepareExection()) {
      writeCommandToJournal();
      command.execute();
   }
} catch (RuntimeException re) {
   try {
      removeCommandFromJournal();
      command.rollback(); //Maybe it would be easier to just reload whole
system, throw some Exception
   } catch (Exception e) {
      System.exit(-1); //Or another way to reload the whole system
   }
}

Did not think about all the caveats possible. Just beginning a new idea.
Probably for Prevayler 3 :-)

See you,


2012/11/17 Paul Bennett <[email protected]>

> Justin, thanks very much for such clear explanation. Some comments below.
>
> On Nov 16, 2012, at 2:55 AM, Justin T. Sampson wrote:
>
> It's exciting to see such a passionate discussion. :) And also good that
> it's coming around to some more practical questions.
>
> First of all, the ACID properties are ill-defined, incomplete, and
> redundant, so ultimately we shouldn't take them too seriously. Yes, A and C
> have some overlap, as do A and I, C and D, etc. They are a nice set of
> reminders of things to talk about regarding transactions in a persistence
> layer, though. And I still maintain that rollback is just one potential
> mechanism to partially address some of these properties, not a fundamental
> requirement for any of them.
>
>
> Ah, OK - then we are in violent agreement :-). I misunderstood your (and
> Klaus's) previous statements to mean that rollback was always irrelevant,
> and never ever useful. That was what I was arguing against.
>
>
> As for journaling transactions after executing them, that would cause
> queries to have the same latency as transactions, which is a pretty big
> departure from Prevayler's normal performance profile. It would hardly even
> be Prevayler anymore. ;)
>
> Paul wrote:
>
> > Klaus, you haven't answered my question. As far as I can see,
> > without something to maintain consistency in the face of an
> > unexpected exceptions, once that happens, the system is
> > inconsistent and cannot proceed *for all users*. You must
> > either a) shut down and fix the bug or b) remove the txn from
> > the journal.
> >
> > So unlike an ordinary db, which would isolate the exception
> > to a single user, and let others continue to use the
> > non-problemmatic parts of the system, an exception kills the
> > whole thing. That's unacceptable to me.
>
> Why does it kill the whole thing?
>
>
> Yes, OK, that was a little extreme :-) What I was really trying to say was
> the above - that rollback can be useful in dealing with certain cases
> (unpredictable but repeatable exceptions thrown because of coding errors),
> in a certain context. In our system, (which is different from a (normal,
> conventional ?) prevalent system where transactions and queries are written
> as specific classes), it is pretty easy to generically generate the inverse
> of each transaction, which makes implementing rollback on exceptions very
> do-able. The user has the same experience as a 'regular' database system.
> But - as you say above - it is not a complete solution for handling
> inconsistency due to any and all errors. It just reduces the number of
> situations where some kind of outside intervention is needed.
>
>
> The point that Klaus and I are making is really that you have to worry
> about the kind of situation you're describing even if you do have a
> rollback mechanism. I've seen plenty of bugs in relational database
> applications that caused inconsistent data without throwing exceptions. At
> some point a user notices something wonky, or some feature starts breaking
> or throwing exceptions later due to the bad data.
>
>
> Agreed. I was not arguing against that - apologies if I was not clear
>
>
> When such inconsistencies happen, it doesn't typically mean that the
> system is suddenly unusable for everyone. Maybe one entity is inaccessible,
> or some calculations display incorrectly, or one user can't log in. Bugs
> that take down an entire application are pretty rare, in my experience, but
> of course they do occasionally happen -- again, even in systems with
> rollbacks.
>
> I haven't so far addressed the question of what to do in a Prevayler-based
> application when bad data creeps in; I've just clarified that you need to
> be prepared for bad data regardless of rollbacks. In the worst case,
> whatever your persistence layer, there's always the option to restart the
> system from a nightly backup. For smaller fixes, a relational database does
> provide the convenience of a built-in scripting language (some variant of
> SQL) with an interactive interpreter; in a similar vein, some Prevayler
> users have talked about using something like BeanShell or some other
> JVM-based scripting language as a superuser feature in an application,
> allowing an arbitrary script to be submitted as a transaction in a running
> system. The purer approach would be to write a new fix-up transaction in
> Java (possibly even with unit tests!) and either redeploy or hotswap the
> app in order to execute it on the running system.
>
>
> That's very useful, and a few things I hadn't thought of. Thanks!
>
>
> As for the history of dropping the food taster, it happened in this thread
> last November, with not nearly as much excitement as in our present thread:
>
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=CAGs7EcU_X1%2BTE3O-CjCje2gsSoxxjnGTm8UZWb6ZkORE7%3Do77A%40mail.gmail.com&forum_name=prevayler-discussion
>
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
>
> http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>
> -pb
>
>
>
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov

_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org