Re: ACIDity of Prevayler 2.6

Hi

I am currently using this alternate pattern for a system that has many 
reads and few writes, with strong consistency requirements.
Any single-point-in-time snapshot of the journal (at disk level) is correct.


prepareCommandOnJournal(command); // minimize IO during lock scope

writeLock.lock();
try {

  try {

    if (command.prepareExecution()) {

      oldHash = model.hashCode();
      command.execute();

      commitCommandToJournal(command); // might cause some latency
                                       // in concurrent queries
    }
   } catch (Exception re) { // execution or journal IO problem

     try {
       command.rollback(); //Maybe it would be easier to just reload
                           //whole system, throw some Exception

       // optional: check that rollback has really restored the old state
       int newHash = model.hashCode();
       if (newHash != oldHash)
           throw RuntimeException("Rollback was not correct", re);

       // no need to fix the journal

     } catch (Exception fatalEx) {
       System.exit(-1); //Or another way to reload the whole system
     }
   }
}
finaly {

   writeLock.unlock();
}



On 23/11/2012 03:42, Alexandre Nodari wrote:
> Uau!
>
> Considering from past e-mails:
>
> - Atomicity and Consistency have some overlap
> - When we say Exceptions, we are usually talking about unexpected
> events. Normally either the Commands were written without throwing
> Exceptions or there are expected Exceptions that are catched and dealt
> within the Command.
> - The Royal Food Taster provides Rollback with 2 steps:
>     - Avoiding a Command that threw an Exception (unexpected event) to
> be written to the journal
>     - Avoiding partial changes by discarding temporary data used to
> check the Commands
>
> We have then two grades (or levels) of Atomicity+Consistency:
>
> A) Without Royal Food Taster: Exceptions (expected and unexpected)
> happen and we let the business code deal with them. Advantages: simple
> and fast. Disadvantage: being some Exceptions unexpected events, maybe
> it is not possible to guarantee that they are deterministic, and not
> being deterministic they can really cause trouble in the system based on
> Prevaylers definition.
>
> B) With Royal Food Taster: Exceptions (unexpected) cause Rollback.
> Advantage: "better" Atomicity+Consistency. Disadvantages: less
> performance and more memory needed.
>
> My opinion:
>
> I think the choice should be given to the developers, offering both options.
>
> And I like the idea of trying to do better than Royal Food Taster. I
> will give a draft of another idea of Rollback to help with the
> discussion, trying to address the 2 steps described above:
>
> Another type of Command called RollbackableCommand with 3 methods:
>
> boolean prepareExecution(); // Prepare to execute, validate
> pre-conditions, save data for rollback
> void execute(); //Execute the changes, catch and deal with all expected
> Exceptions
> void rollback(); //Rollback possible changes
>
> And Prevayler would do something like this:
>
> try {
>     if (command.prepareExection()) {
>        writeCommandToJournal();
>        command.execute();
>     }
> } catch (RuntimeException re) {
>     try {
>        removeCommandFromJournal();
>        command.rollback(); //Maybe it would be easier to just reload
> whole system, throw some Exception
>     } catch (Exception e) {
>        System.exit(-1); //Or another way to reload the whole system
>     }
> }
>
> Did not think about all the caveats possible. Just beginning a new idea.
> Probably for Prevayler 3 :-)
>
> See you,
>
>
> 2012/11/17 Paul Bennett <[email protected] <mailto:[email protected]>>
>
>     Justin, thanks very much for such clear explanation. Some comments
>     below.
>
>     On Nov 16, 2012, at 2:55 AM, Justin T. Sampson wrote:
>
>>     It's exciting to see such a passionate discussion. :) And also
>>     good that it's coming around to some more practical questions.
>>
>>     First of all, the ACID properties are ill-defined, incomplete, and
>>     redundant, so ultimately we shouldn't take them too seriously.
>>     Yes, A and C have some overlap, as do A and I, C and D, etc. They
>>     are a nice set of reminders of things to talk about regarding
>>     transactions in a persistence layer, though. And I still maintain
>>     that rollback is just one potential mechanism to partially address
>>     some of these properties, not a fundamental requirement for any of
>>     them.
>
>     Ah, OK - then we are in violent agreement :-). I misunderstood your
>     (and Klaus's) previous statements to mean that rollback was always
>     irrelevant, and never ever useful. That was what I was arguing against.
>
>>
>>     As for journaling transactions after executing them, that would
>>     cause queries to have the same latency as transactions, which is a
>>     pretty big departure from Prevayler's normal performance profile.
>>     It would hardly even be Prevayler anymore. ;)
>>
>>     Paul wrote:
>>
>>     > Klaus, you haven't answered my question. As far as I can see,
>>     > without something to maintain consistency in the face of an
>>     > unexpected exceptions, once that happens, the system is
>>     > inconsistent and cannot proceed *for all users*. You must
>>     > either a) shut down and fix the bug or b) remove the txn from
>>     > the journal.
>>     >
>>     > So unlike an ordinary db, which would isolate the exception
>>     > to a single user, and let others continue to use the
>>     > non-problemmatic parts of the system, an exception kills the
>>     > whole thing. That's unacceptable to me.
>>
>>     Why does it kill the whole thing?
>
>     Yes, OK, that was a little extreme :-) What I was really trying to
>     say was the above - that rollback can be useful in dealing with
>     certain cases (unpredictable but repeatable exceptions thrown
>     because of coding errors), in a certain context. In our system,
>     (which is different from a (normal, conventional ?) prevalent system
>     where transactions and queries are written as specific classes), it
>     is pretty easy to generically generate the inverse of each
>     transaction, which makes implementing rollback on exceptions very
>     do-able. The user has the same experience as a 'regular' database
>     system. But - as you say above - it is not a complete solution for
>     handling inconsistency due to any and all errors. It just reduces
>     the number of situations where some kind of outside intervention is
>     needed.
>
>>
>>     The point that Klaus and I are making is really that you have to
>>     worry about the kind of situation you're describing even if you do
>>     have a rollback mechanism. I've seen plenty of bugs in relational
>>     database applications that caused inconsistent data without
>>     throwing exceptions. At some point a user notices something wonky,
>>     or some feature starts breaking or throwing exceptions later due
>>     to the bad data.
>
>     Agreed. I was not arguing against that - apologies if I was not clear
>
>>
>>     When such inconsistencies happen, it doesn't typically mean that
>>     the system is suddenly unusable for everyone. Maybe one entity is
>>     inaccessible, or some calculations display incorrectly, or one
>>     user can't log in. Bugs that take down an entire application are
>>     pretty rare, in my experience, but of course they do occasionally
>>     happen -- again, even in systems with rollbacks.
>>
>>     I haven't so far addressed the question of what to do in a
>>     Prevayler-based application when bad data creeps in; I've just
>>     clarified that you need to be prepared for bad data regardless of
>>     rollbacks. In the worst case, whatever your persistence layer,
>>     there's always the option to restart the system from a nightly
>>     backup. For smaller fixes, a relational database does provide the
>>     convenience of a built-in scripting language (some variant of SQL)
>>     with an interactive interpreter; in a similar vein, some Prevayler
>>     users have talked about using something like BeanShell or some
>>     other JVM-based scripting language as a superuser feature in an
>>     application, allowing an arbitrary script to be submitted as a
>>     transaction in a running system. The purer approach would be to
>>     write a new fix-up transaction in Java (possibly even with unit
>>     tests!) and either redeploy or hotswap the app in order to execute
>>     it on the running system.
>
>     That's very useful, and a few things I hadn't thought of. Thanks!
>
>>
>>     As for the history of dropping the food taster, it happened in
>>     this thread last November, with not nearly as much excitement as
>>     in our present thread:
>>
>>     http://sourceforge.net/mailarchive/forum.php?thread_name=CAGs7EcU_X1%2BTE3O-CjCje2gsSoxxjnGTm8UZWb6ZkORE7%3Do77A%40mail.gmail.com&forum_name=prevayler-discussion
>>
>>     ------------------------------------------------------------------------------
>>     Monitor your physical, virtual and cloud infrastructure from a single
>>     web console. Get in-depth insight into apps, servers, databases,
>>     vmware,
>>     SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>>     Pricing starts from $795 for 25 servers or applications!
>>     http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________
>>     To unsubscribe go to the end of this page:
>>     http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>>     _______________________________________________
>>     "Databases in Memoriam" -- http://www.prevayler.org
>
>     -pb
>
>
>
>     ------------------------------------------------------------------------------
>     Monitor your physical, virtual and cloud infrastructure from a single
>     web console. Get in-depth insight into apps, servers, databases, vmware,
>     SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>     Pricing starts from $795 for 25 servers or applications!
>     http://p.sf.net/sfu/zoho_dev2dev_nov
>     _______________________________________________
>     To unsubscribe go to the end of this page:
>     http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>     _______________________________________________
>     "Databases in Memoriam" -- http://www.prevayler.org
>
>
>
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
>
>
>
> _______________________________________________
> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>


------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org