Re: Transaction management in Ozone

On Tue, 2004-07-06 at 17:23, Jean Morissette wrote:
> Leo Mekenkamp a écrit :
> > On Mon, 2004-07-05 at 17:30, Jean Morissette wrote:
> > [...]
> >>
> >>Hi Per,
> >>I am interested about the extreme cases that you mentioned about 
> >>database corruption.  Could you explain to me when and why such a thing 
> >>can happen?  Leo talked previously about ReiserFS4; however I think that 
> >>we need to use JNI calls if we want to use transaction functionality 
> >>offered by Reiser4, no?
> > 
> > 
> > Applications do not have to be aware of the journalling possibilities of
> > a file system to benefit from them. There is fortunately no use
> > whatsoever to use JNI.
> > 
> > Cheers,
> > Leo
> > 
> 
> Here is a note from http://www.namesys.com/
> "Full data journaling is considered by many to be a good way to achieve 
> file data integrity across system crashes. However, although file data 
> may appear to be consistent from the kernel point of view, since there 
> is no API exported to the userspace to control transactions, we may 
> end-up in a situation where the application makes 2 write requests (as 
> part of one logical transaction) but only one of these gets journaled 
> before the system crashes. From the application point of view, we may 
> then end up with inconsistent data in the file.
> Such issues should be addressed with the upcoming ReiserFS v.4 release. 
> Such an API will be exported to userspace and all programs that need 
> transactions will be able to use it."
> 
> So, if we want to use futur ReiserFS v.4 transaction API, we need too 
> use JNI, no?

Yes, but there is no need to use that transaction API, since ozone has
its own transaction management system. If an ozone transaction has not
completed, then it will get rolled back by restoring shadowed data.

> But, the true question is: is-it possible in Java to achieve transaction 
> integrity across system crashes (using shadow-paging like Ozone or 
> Write-Ahead-Logging like the majority of RDBMS)?

Al long as the OS/filesystem handles write actions sequentially without
changing their order, then I do not see why not.

> I am concerned about 
> Per statement when he says: "some extreme cases such as server power 
> failure which under certain unfortunate circumstances can corrupt the
> database."  Could you describe the problem please?  Is-there a solution?

The easiest solution is using a UPS. IMHO every (serious) server should
have power-backup. And of course a stable OS; I for one do not consider
Windows to be stable enough.

Back to the problem: a transaction in WizardStore commits in 2 phases:
the pre-commit phase writes all data to disk and the commit phase
deletes the old data. The problem (IIRC, someone please step in here
when I am wrong) is when a crash occurs in that last phase; when the
server comes back up, it will restore all shadow clusters. Since a
transaction may span several clusters, some clusters shadows may have
been deleted, others not.

I have been thinking about this, and a possible solution could be
something like this:
(pre-commit has written all data to disk)
- commit phase starts
- transaction Tn writes all cluster IDs to a file TnF0 (n in Tn being
the transaction ID)
- transaction renames TnF0 to TnF1
- transaction deletes all shadow files
- transaction deletes TnF1
- commit phase ends

Instead of restoring all shadow files when recovering from a crash, the
server would delete all shadow files for which the cluster IDs are in
one of the TnF1 files, thereby finishing the commit for that transaction
Tn; TnF0 files will simply be deleted.

Cheers,
Leo




-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com