Re: ACIDity of Prevayler 2.6

This is perhaps slightly side topic, but …   It's been about .3 versions since I've done it but ACIDity was broken in another way when I tried.  

While loading the system with data, a memory error occurred (we had run out of memory).  The system was stopped and restarted.  Truly ACIDic would mean that the system restarts with all the data it had prior to the error and no other data.  The transaction that failed should not supply any information to the system nor should it be re-attempted automatically by the system.

What happens in fact is that the transaction is reattempted automatically and fails again and again unless you make a change.  At the time, the solution was to try to manually delete the serialized transaction before restarting, but the transactions were bundled up together and really pretty much inseparable on disk.  

Like a broken record, I will say again that if the serialized transactions could be managed, the system could do undo style functionality out of the box, at least at desktop scales.

R

On Nov 16, 2012, at 2:06 AM, Arnaud Masson <[email protected]> wrote:

> Many systems have much more queries than transactions,
> so transaction latency wouldn't be so critical.
> 
> Moreover a transaction could be physically pre-serialized (or maybe in 
> parallel thread) on the disk, in some temporary location, before the 
> lock scope.
> After tx successful execution, just a quick "file move/rename" operation 
> would be required to commit it to the journal, not a full serialization.
> 
> 
> 
> On 16/11/2012 08:55, Justin T. Sampson wrote:
>> It's exciting to see such a passionate discussion. :) And also good that
>> it's coming around to some more practical questions.
>> 
>> First of all, the ACID properties are ill-defined, incomplete, and
>> redundant, so ultimately we shouldn't take them too seriously. Yes, A
>> and C have some overlap, as do A and I, C and D, etc. They are a nice
>> set of reminders of things to talk about regarding transactions in a
>> persistence layer, though. And I still maintain that rollback is just
>> one potential mechanism to partially address some of these properties,
>> not a fundamental requirement for any of them.
>> 
>> As for journaling transactions after executing them, that would cause
>> queries to have the same latency as transactions, which is a pretty big
>> departure from Prevayler's normal performance profile. It would hardly
>> even be Prevayler anymore. ;)
>> 
>> Paul wrote:
>> 
>>> Klaus, you haven't answered my question. As far as I can see,
>>> without something to maintain consistency in the face of an
>>> unexpected exceptions, once that happens, the system is
>>> inconsistent and cannot proceed *for all users*. You must
>>> either a) shut down and fix the bug or b) remove the txn from
>>> the journal.
>>> 
>>> So unlike an ordinary db, which would isolate the exception
>>> to a single user, and let others continue to use the
>>> non-problemmatic parts of the system, an exception kills the
>>> whole thing. That's unacceptable to me.
>> 
>> Why does it kill the whole thing?
>> 
>> The point that Klaus and I are making is really that you have to worry
>> about the kind of situation you're describing even if you do have a
>> rollback mechanism. I've seen plenty of bugs in relational database
>> applications that caused inconsistent data without throwing exceptions.
>> At some point a user notices something wonky, or some feature starts
>> breaking or throwing exceptions later due to the bad data.
>> 
>> When such inconsistencies happen, it doesn't typically mean that the
>> system is suddenly unusable for everyone. Maybe one entity is
>> inaccessible, or some calculations display incorrectly, or one user
>> can't log in. Bugs that take down an entire application are pretty rare,
>> in my experience, but of course they do occasionally happen -- again,
>> even in systems with rollbacks.
>> 
>> I haven't so far addressed the question of what to do in a
>> Prevayler-based application when bad data creeps in; I've just clarified
>> that you need to be prepared for bad data regardless of rollbacks. In
>> the worst case, whatever your persistence layer, there's always the
>> option to restart the system from a nightly backup. For smaller fixes, a
>> relational database does provide the convenience of a built-in scripting
>> language (some variant of SQL) with an interactive interpreter; in a
>> similar vein, some Prevayler users have talked about using something
>> like BeanShell or some other JVM-based scripting language as a superuser
>> feature in an application, allowing an arbitrary script to be submitted
>> as a transaction in a running system. The purer approach would be to
>> write a new fix-up transaction in Java (possibly even with unit tests!)
>> and either redeploy or hotswap the app in order to execute it on the
>> running system.
>> 
>> As for the history of dropping the food taster, it happened in this
>> thread last November, with not nearly as much excitement as in our
>> present thread:
>> 
>> http://sourceforge.net/mailarchive/forum.php?thread_name=CAGs7EcU_X1%2BTE3O-CjCje2gsSoxxjnGTm8UZWb6ZkORE7%3Do77A%40mail.gmail.com&forum_name=prevayler-discussion
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Monitor your physical, virtual and cloud infrastructure from a single
>> web console. Get in-depth insight into apps, servers, databases, vmware,
>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>> Pricing starts from $795 for 25 servers or applications!
>> http://p.sf.net/sfu/zoho_dev2dev_nov
>> 
>> 
>> 
>> _______________________________________________
>> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>> 
> 
> 
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org


------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org