Re: large transaction sizes and rollbacks

I gave a good deal of thought to the file IO problem and decided that
provided one is careful, it will in practice very safe. In the situation I
have, if a file is read and successfully applied to the model once, it
will do so in exactly the same way on all future occations. The files are
handled by the app and I would not expect them to go missing or change. If
someone could open them and fiddle with them, then the same could be said
about the prevayler transaction log itself. It's possible but unlikely.
But indeed, on future restarts, I would of course want to completely halt
further processing on encountering ANY file IO whatsoever.

I think this was the reason I was confused about why prevayler was
silently ignoring exceptions thrown at that point. I was expecting
prevayler to stop all further processing. What heppens if, for example,
you get an OutOfMemoryException at this point?

changes
> There are a few interesting things going on here.
>
> On Thu, Mar 3, 2011 at 4:11 AM, John Pearcey <[email protected]> wrote:
>
>> What I have done instead is simply save a date stamp object. To my
>> understanding, this will be saved in the transaction log. When
>> executeAndQuery runs, the method uses the date object to locate the file
>> ,
>> deserialise it and apply the data to the memory model. I have this
>> working
>> correctly with no problems. (It's fast too!).
>>
>
> This is reasonable, but one risk (that I don't think you've run into yet)
> is
> that all IO is inherently nondeterministic -- what if the file is missing,
> for example, or got written with the wrong permissions, etc. Doing any IO
> whatsoever inside a Prevayler transaction violates Prevayler's assumption
> on
> deterministic transactions. An IO operation might throw an IOException one
> time but work fine another time, which could lead to inconsistent system
> state. One option to make this safe would be to include a checksum of the
> file as part of the transaction (say, a SHA-1 hash of the original file)
> and
> check it against the file when it's read. If the checksums don't match, OR
> if ANY exception is thrown from any IO operations, catch it and throw an
> Error instead -- which Prevayler (at least as of 2.3, maybe earlier) will
> recognize as a NONdeterministic event and refuse to process any more
> transactions or queries until you restart the system (when it will try
> that
> transaction again). That may seem drastic, but it's correct and safe
> behavior.
>
> The problem comes if the file has bad data for example (even though it has
>> been thoroughly checked, you can never be too sure). executeAndQuery()
>> will
>> throw an exception but prevayler does not propagate this and it
>> disappears.
>> I have restored the memory model (effectively a manual rollback) but the
>> date object is still in the (prevayler) transaction history which means
>> that, upon restart, prevayler use this date unnecessarily, since it is
>> bound
>> to fail.
>>
>
> Right, if the transaction throws an exception during replay then the
> exception is ignored, because Prevayler assumes that transactions are
> deterministic -- if it throws an exception during replay, it must also
> have
> thrown an exception originally. This is necessary to restore the system to
> a
> consistent state. Did you fix the data model by way of another transaction
> after the one with the bad file? If so, then there shouldn't be any real
> harm in replaying that file's transaction, it's just some extra time
> during
> startup.
>
> In summary: is there any way to remove a transaction where the application
>> to the memory model failed? or
>>     is there a way to tell prevayler that the transaction was
>> effectively
>> rolled back so that on subsequent startups, it is ignored.
>>
>
> The obvious thing is to simply take a snapshot. :) Once you've taken a
> snapshot, Prevayler will never read any journal entries from before that
> snapshot.
>
> If the transaction you want to eliminate is the very last transaction in
> the
> last journal file, I suppose you could manually truncate the journal file
> before that transaction (the boundaries between transactions are plain
> ASCII
> so it should be easy to find). But if the transaction isn't actually
> crashing the system then I wouldn't recommend doing something like that.
>
> Cheers,
> Justin
>


------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org