Re: large transaction sizes and rollbacks

There are a few interesting things going on here.

On Thu, Mar 3, 2011 at 4:11 AM, John Pearcey <[email protected]> wrote:

> What I have done instead is simply save a date stamp object. To my
> understanding, this will be saved in the transaction log. When
> executeAndQuery runs, the method uses the date object to locate the file ,
> deserialise it and apply the data to the memory model. I have this working
> correctly with no problems. (It's fast too!).
>

This is reasonable, but one risk (that I don't think you've run into yet) is
that all IO is inherently nondeterministic -- what if the file is missing,
for example, or got written with the wrong permissions, etc. Doing any IO
whatsoever inside a Prevayler transaction violates Prevayler's assumption on
deterministic transactions. An IO operation might throw an IOException one
time but work fine another time, which could lead to inconsistent system
state. One option to make this safe would be to include a checksum of the
file as part of the transaction (say, a SHA-1 hash of the original file) and
check it against the file when it's read. If the checksums don't match, OR
if ANY exception is thrown from any IO operations, catch it and throw an
Error instead -- which Prevayler (at least as of 2.3, maybe earlier) will
recognize as a NONdeterministic event and refuse to process any more
transactions or queries until you restart the system (when it will try that
transaction again). That may seem drastic, but it's correct and safe
behavior.

The problem comes if the file has bad data for example (even though it has
> been thoroughly checked, you can never be too sure). executeAndQuery() will
> throw an exception but prevayler does not propagate this and it disappears.
> I have restored the memory model (effectively a manual rollback) but the
> date object is still in the (prevayler) transaction history which means
> that, upon restart, prevayler use this date unnecessarily, since it is bound
> to fail.
>

Right, if the transaction throws an exception during replay then the
exception is ignored, because Prevayler assumes that transactions are
deterministic -- if it throws an exception during replay, it must also have
thrown an exception originally. This is necessary to restore the system to a
consistent state. Did you fix the data model by way of another transaction
after the one with the bad file? If so, then there shouldn't be any real
harm in replaying that file's transaction, it's just some extra time during
startup.

In summary: is there any way to remove a transaction where the application
> to the memory model failed? or
>     is there a way to tell prevayler that the transaction was effectively
> rolled back so that on subsequent startups, it is ignored.
>

The obvious thing is to simply take a snapshot. :) Once you've taken a
snapshot, Prevayler will never read any journal entries from before that
snapshot.

If the transaction you want to eliminate is the very last transaction in the
last journal file, I suppose you could manually truncate the journal file
before that transaction (the boundaries between transactions are plain ASCII
so it should be easy to find). But if the transaction isn't actually
crashing the system then I wouldn't recommend doing something like that.

Cheers,
Justin

------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d

_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org