Re: ACIDity of Prevayler 2.6

It seems very demanding on the prevalent system code to require it to be
rollbackable.

I like the idea of a meaningful prevalentSystem.hashCode() very much. It is
useful for checking schema migrations too.

Klaus


On Fri, Nov 23, 2012 at 7:46 AM, Arnaud Masson <[email protected]> wrote:

> Hi
>
> I am currently using this alternate pattern for a system that has many
> reads and few writes, with strong consistency requirements.
> Any single-point-in-time snapshot of the journal (at disk level) is
> correct.
>
>
> prepareCommandOnJournal(command); // minimize IO during lock scope
>
> writeLock.lock();
> try {
>
>   try {
>
>     if (command.prepareExecution()) {
>
>       oldHash = model.hashCode();
>       command.execute();
>
>       commitCommandToJournal(command); // might cause some latency
>                                        // in concurrent queries
>     }
>    } catch (Exception re) { // execution or journal IO problem
>
>      try {
>        command.rollback(); //Maybe it would be easier to just reload
>                            //whole system, throw some Exception
>
>        // optional: check that rollback has really restored the old state
>        int newHash = model.hashCode();
>        if (newHash != oldHash)
>            throw RuntimeException("Rollback was not correct", re);
>
>        // no need to fix the journal
>
>      } catch (Exception fatalEx) {
>        System.exit(-1); //Or another way to reload the whole system
>      }
>    }
> }
> finaly {
>
>    writeLock.unlock();
> }
>
>
>
> On 23/11/2012 03:42, Alexandre Nodari wrote:
> > Uau!
> >
> > Considering from past e-mails:
> >
> > - Atomicity and Consistency have some overlap
> > - When we say Exceptions, we are usually talking about unexpected
> > events. Normally either the Commands were written without throwing
> > Exceptions or there are expected Exceptions that are catched and dealt
> > within the Command.
> > - The Royal Food Taster provides Rollback with 2 steps:
> >     - Avoiding a Command that threw an Exception (unexpected event) to
> > be written to the journal
> >     - Avoiding partial changes by discarding temporary data used to
> > check the Commands
> >
> > We have then two grades (or levels) of Atomicity+Consistency:
> >
> > A) Without Royal Food Taster: Exceptions (expected and unexpected)
> > happen and we let the business code deal with them. Advantages: simple
> > and fast. Disadvantage: being some Exceptions unexpected events, maybe
> > it is not possible to guarantee that they are deterministic, and not
> > being deterministic they can really cause trouble in the system based on
> > Prevaylers definition.
> >
> > B) With Royal Food Taster: Exceptions (unexpected) cause Rollback.
> > Advantage: "better" Atomicity+Consistency. Disadvantages: less
> > performance and more memory needed.
> >
> > My opinion:
> >
> > I think the choice should be given to the developers, offering both
> options.
> >
> > And I like the idea of trying to do better than Royal Food Taster. I
> > will give a draft of another idea of Rollback to help with the
> > discussion, trying to address the 2 steps described above:
> >
> > Another type of Command called RollbackableCommand with 3 methods:
> >
> > boolean prepareExecution(); // Prepare to execute, validate
> > pre-conditions, save data for rollback
> > void execute(); //Execute the changes, catch and deal with all expected
> > Exceptions
> > void rollback(); //Rollback possible changes
> >
> > And Prevayler would do something like this:
> >
> > try {
> >     if (command.prepareExection()) {
> >        writeCommandToJournal();
> >        command.execute();
> >     }
> > } catch (RuntimeException re) {
> >     try {
> >        removeCommandFromJournal();
> >        command.rollback(); //Maybe it would be easier to just reload
> > whole system, throw some Exception
> >     } catch (Exception e) {
> >        System.exit(-1); //Or another way to reload the whole system
> >     }
> > }
> >
> > Did not think about all the caveats possible. Just beginning a new idea.
> > Probably for Prevayler 3 :-)
> >
> > See you,
> >
> >
> > 2012/11/17 Paul Bennett <[email protected] <mailto:[email protected]>>
> >
> >     Justin, thanks very much for such clear explanation. Some comments
> >     below.
> >
> >     On Nov 16, 2012, at 2:55 AM, Justin T. Sampson wrote:
> >
> >>     It's exciting to see such a passionate discussion. :) And also
> >>     good that it's coming around to some more practical questions.
> >>
> >>     First of all, the ACID properties are ill-defined, incomplete, and
> >>     redundant, so ultimately we shouldn't take them too seriously.
> >>     Yes, A and C have some overlap, as do A and I, C and D, etc. They
> >>     are a nice set of reminders of things to talk about regarding
> >>     transactions in a persistence layer, though. And I still maintain
> >>     that rollback is just one potential mechanism to partially address
> >>     some of these properties, not a fundamental requirement for any of
> >>     them.
> >
> >     Ah, OK - then we are in violent agreement :-). I misunderstood your
> >     (and Klaus's) previous statements to mean that rollback was always
> >     irrelevant, and never ever useful. That was what I was arguing
> against.
> >
> >>
> >>     As for journaling transactions after executing them, that would
> >>     cause queries to have the same latency as transactions, which is a
> >>     pretty big departure from Prevayler's normal performance profile.
> >>     It would hardly even be Prevayler anymore. ;)
> >>
> >>     Paul wrote:
> >>
> >>     > Klaus, you haven't answered my question. As far as I can see,
> >>     > without something to maintain consistency in the face of an
> >>     > unexpected exceptions, once that happens, the system is
> >>     > inconsistent and cannot proceed *for all users*. You must
> >>     > either a) shut down and fix the bug or b) remove the txn from
> >>     > the journal.
> >>     >
> >>     > So unlike an ordinary db, which would isolate the exception
> >>     > to a single user, and let others continue to use the
> >>     > non-problemmatic parts of the system, an exception kills the
> >>     > whole thing. That's unacceptable to me.
> >>
> >>     Why does it kill the whole thing?
> >
> >     Yes, OK, that was a little extreme :-) What I was really trying to
> >     say was the above - that rollback can be useful in dealing with
> >     certain cases (unpredictable but repeatable exceptions thrown
> >     because of coding errors), in a certain context. In our system,
> >     (which is different from a (normal, conventional ?) prevalent system
> >     where transactions and queries are written as specific classes), it
> >     is pretty easy to generically generate the inverse of each
> >     transaction, which makes implementing rollback on exceptions very
> >     do-able. The user has the same experience as a 'regular' database
> >     system. But - as you say above - it is not a complete solution for
> >     handling inconsistency due to any and all errors. It just reduces
> >     the number of situations where some kind of outside intervention is
> >     needed.
> >
> >>
> >>     The point that Klaus and I are making is really that you have to
> >>     worry about the kind of situation you're describing even if you do
> >>     have a rollback mechanism. I've seen plenty of bugs in relational
> >>     database applications that caused inconsistent data without
> >>     throwing exceptions. At some point a user notices something wonky,
> >>     or some feature starts breaking or throwing exceptions later due
> >>     to the bad data.
> >
> >     Agreed. I was not arguing against that - apologies if I was not clear
> >
> >>
> >>     When such inconsistencies happen, it doesn't typically mean that
> >>     the system is suddenly unusable for everyone. Maybe one entity is
> >>     inaccessible, or some calculations display incorrectly, or one
> >>     user can't log in. Bugs that take down an entire application are
> >>     pretty rare, in my experience, but of course they do occasionally
> >>     happen -- again, even in systems with rollbacks.
> >>
> >>     I haven't so far addressed the question of what to do in a
> >>     Prevayler-based application when bad data creeps in; I've just
> >>     clarified that you need to be prepared for bad data regardless of
> >>     rollbacks. In the worst case, whatever your persistence layer,
> >>     there's always the option to restart the system from a nightly
> >>     backup. For smaller fixes, a relational database does provide the
> >>     convenience of a built-in scripting language (some variant of SQL)
> >>     with an interactive interpreter; in a similar vein, some Prevayler
> >>     users have talked about using something like BeanShell or some
> >>     other JVM-based scripting language as a superuser feature in an
> >>     application, allowing an arbitrary script to be submitted as a
> >>     transaction in a running system. The purer approach would be to
> >>     write a new fix-up transaction in Java (possibly even with unit
> >>     tests!) and either redeploy or hotswap the app in order to execute
> >>     it on the running system.
> >
> >     That's very useful, and a few things I hadn't thought of. Thanks!
> >
> >>
> >>     As for the history of dropping the food taster, it happened in
> >>     this thread last November, with not nearly as much excitement as
> >>     in our present thread:
> >>
> >>
> http://sourceforge.net/mailarchive/forum.php?thread_name=CAGs7EcU_X1%2BTE3O-CjCje2gsSoxxjnGTm8UZWb6ZkORE7%3Do77A%40mail.gmail.com&forum_name=prevayler-discussion
> >>
> >>
> ------------------------------------------------------------------------------
> >>     Monitor your physical, virtual and cloud infrastructure from a
> single
> >>     web console. Get in-depth insight into apps, servers, databases,
> >>     vmware,
> >>     SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> >>     Pricing starts from $795 for 25 servers or applications!
> >>
> http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________
> >>     To unsubscribe go to the end of this page:
> >>     http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >>     _______________________________________________
> >>     "Databases in Memoriam" -- http://www.prevayler.org
> >
> >     -pb
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> >     Monitor your physical, virtual and cloud infrastructure from a single
> >     web console. Get in-depth insight into apps, servers, databases,
> vmware,
> >     SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> >     Pricing starts from $795 for 25 servers or applications!
> >     http://p.sf.net/sfu/zoho_dev2dev_nov
> >     _______________________________________________
> >     To unsubscribe go to the end of this page:
> >     http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> >     _______________________________________________
> >     "Databases in Memoriam" -- http://www.prevayler.org
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Monitor your physical, virtual and cloud infrastructure from a single
> > web console. Get in-depth insight into apps, servers, databases, vmware,
> > SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> > Pricing starts from $795 for 25 servers or applications!
> > http://p.sf.net/sfu/zoho_dev2dev_nov
> >
> >
> >
> > _______________________________________________
> > To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> > _______________________________________________
> > "Databases in Memoriam" -- http://www.prevayler.org
> >
>
>
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>



-- 
Valeu, Klaus.

------------------------------------------------------------------------------
Keep yourself connected to Go Parallel: 
INSIGHTS What's next for parallel hardware, programming and related areas?
Interviews and blogs by thought leaders keep you ahead of the curve.
http://goparallel.sourceforge.net

_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org