Re: Transaction management in Ozone

Hi Leo,

I think your idea is not bad. The big issue as I see it is not that we might 
loose some data if a system crash occurs in the midst of a transaction but 
that under certain circumstances we can actually corrupt the database and are 
forced to rely on backup to recover. I cant remember who it was that did some 
testing on this and reported the issue initially but it should be in the mail 
archive somewhere. Anyways, your idea would minimize this risk of corruption 
right?

When it comes to the discussion of how to be 100% certain data is written then 
the only way I know is to write our own filesystem or hook into the API of a 
filesystem with the necessary features. But this is outside the scope of the 
Ozone project IMO since it breaks platform independence. 

Best regards,
Per

onsdagen den 7 juli 2004 07.03 skrev Leo Mekenkamp:
> On Tue, 2004-07-06 at 17:23, Jean Morissette wrote:
> > Leo Mekenkamp a écrit :
> > > On Mon, 2004-07-05 at 17:30, Jean Morissette wrote:
> > > [...]
> > >
> > >>Hi Per,
> > >>I am interested about the extreme cases that you mentioned about
> > >>database corruption.  Could you explain to me when and why such a thing
> > >>can happen?  Leo talked previously about ReiserFS4; however I think
> > >> that we need to use JNI calls if we want to use transaction
> > >> functionality offered by Reiser4, no?
> > >
> > > Applications do not have to be aware of the journalling possibilities
> > > of a file system to benefit from them. There is fortunately no use
> > > whatsoever to use JNI.
> > >
> > > Cheers,
> > > Leo
> >
> > Here is a note from http://www.namesys.com/
> > "Full data journaling is considered by many to be a good way to achieve
> > file data integrity across system crashes. However, although file data
> > may appear to be consistent from the kernel point of view, since there
> > is no API exported to the userspace to control transactions, we may
> > end-up in a situation where the application makes 2 write requests (as
> > part of one logical transaction) but only one of these gets journaled
> > before the system crashes. From the application point of view, we may
> > then end up with inconsistent data in the file.
> > Such issues should be addressed with the upcoming ReiserFS v.4 release.
> > Such an API will be exported to userspace and all programs that need
> > transactions will be able to use it."
> >
> > So, if we want to use futur ReiserFS v.4 transaction API, we need too
> > use JNI, no?
>
> Yes, but there is no need to use that transaction API, since ozone has
> its own transaction management system. If an ozone transaction has not
> completed, then it will get rolled back by restoring shadowed data.
>
> > But, the true question is: is-it possible in Java to achieve transaction
> > integrity across system crashes (using shadow-paging like Ozone or
> > Write-Ahead-Logging like the majority of RDBMS)?
>
> Al long as the OS/filesystem handles write actions sequentially without
> changing their order, then I do not see why not.
>
> > I am concerned about
> > Per statement when he says: "some extreme cases such as server power
> > failure which under certain unfortunate circumstances can corrupt the
> > database."  Could you describe the problem please?  Is-there a solution?
>
> The easiest solution is using a UPS. IMHO every (serious) server should
> have power-backup. And of course a stable OS; I for one do not consider
> Windows to be stable enough.
>
> Back to the problem: a transaction in WizardStore commits in 2 phases:
> the pre-commit phase writes all data to disk and the commit phase
> deletes the old data. The problem (IIRC, someone please step in here
> when I am wrong) is when a crash occurs in that last phase; when the
> server comes back up, it will restore all shadow clusters. Since a
> transaction may span several clusters, some clusters shadows may have
> been deleted, others not.
>
> I have been thinking about this, and a possible solution could be
> something like this:
> (pre-commit has written all data to disk)
> - commit phase starts
> - transaction Tn writes all cluster IDs to a file TnF0 (n in Tn being
> the transaction ID)
> - transaction renames TnF0 to TnF1
> - transaction deletes all shadow files
> - transaction deletes TnF1
> - commit phase ends
>
> Instead of restoring all shadow files when recovering from a crash, the
> server would delete all shadow files for which the cluster IDs are in
> one of the TnF1 files, thereby finishing the commit for that transaction
> Tn; TnF0 files will simply be deleted.
>
> Cheers,
> Leo
>
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> digital self defense, top technical experts, no vendor pitches,
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Ozone-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/ozone-users


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com