Re: Transaction management in Ozone

On Wed, 2004-07-07 at 12:05, Arjen van der Meijden wrote:
> > Yes, but there is no need to use that transaction API, since ozone has
> > its own transaction management system. If an ozone transaction has not
> > completed, then it will get rolled back by restoring shadowed data.
> 
> The problem here is that your transaction API has no guarantee that the 
> data is actually written to disk, since Java doesn't allow you to call a 
> "fsync" (afaik, although that is, due to harddisk caches, still no hard 
> guarantee).
> So whenever your transaction-manager thinks it has written something to 
> disc, it may not have been written yet, even the seperate steps of a 
> two-phase commit may not have been written while the TM thinks they have.

http://java.sun.com/j2se/1.4.2/docs/api/java/io/RandomAccessFile.html#RandomAccessFile(java.io.File,%20java.lang.String)
See the rws and rwd explanation.

Also, FileOutputStream.close() calls flush(). Afaik that flush forces a
write down to the bare magnetic particles of a disk; FileOutputStream is
a delegator around the nio FileChannel, and FileChannel also supports 
flush
(http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileChannel.html#force(boolean)), so I assume a force is called when the channel is closed.

As far as the harddisk goes: every storage device should have flush
capabilities, because if it does not have such capabilities one would
never be able to reliably turn the device off. It can be assumed that an
OS can call this flush, and thus the JVM can make indirect use of it.

> > Al long as the OS/filesystem handles write actions sequentially without
> > changing their order, then I do not see why not.
> 
> Afaik there is no guarantee that writes to different files (or even 
> blocks?) are done sequentially... They may be rescheduled to allow 
> better (write) performance, as long as the data that was supposed to be 
> written before a certain action on the same array, is written.

This last thing, together with a flush, is all we need.

> > The easiest solution is using a UPS. IMHO every (serious) server should
> > have power-backup. And of course a stable OS; I for one do not consider
> > Windows to be stable enough.
> 
> Using UPS and a stable OS doesn't free you from crashes. Accepting that 
> crashes can occur, in rare cases, is much better than trying to deny 
> them or point out faulty components (OS, hardware, serveradmins) that 
> where supposed to have been replaced ;)
> We have expensive (in x86-terms) hardware, slackware linux as OS, 
> running in a datacenter with power backup etc etc, but there still are 
> crashes... Should we replace the hardware by even more expensive Sun or 
> IBM Unix-servers? Or should we accept the fact that there may occur 
> crashes so now and then? :)
> 
> My point is that one shouldn't expect _too_ much from the underlying 
> system, but since you can't prevent everything, you may "ignore" a few 
> rare cases.

Could not agree with you more here.

> > The problem (IIRC, someone please step in here
> > when I am wrong) is when a crash occurs in that last phase; when the
> > server comes back up, it will restore all shadow clusters. Since a
> > transaction may span several clusters, some clusters shadows may have
> > been deleted, others not.
> 
> As said, the transactionmanager may be thinking a write is completed on 
> disk, while it is not.

Nope. If this is the case all databases would suffer from a similar
problem. That would mean no database could reliably recover from a
crash.

Cheers,
Leo




-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com