Re: Oracle RAC with ENBD

"Also sprach roland:"
> Is it possible/recommended to use ENBD for shared storage?

Hmmm ...

> Actually, our DBA`s evaluate Oracle RAC on shared storage based on scsi and firewire - but i`m 
> thinking about a scenario with two virtual linux systems (on vmware) using a networked block device.
> There is a paper from someone, who did this with NBD: http://www.fi.muni.cz/~kripac/orac-nbd/

If it is possible with NBD then it is possible with ENBD>

But the word "shared" makes me wonder! You can't in principle share
remote devices because every kernel involved would have a different
cached image of the device. They'd all be unaware of changes made by
the thers if they were to areas hey already had cached.

The only soluton is to not cache anything, something that can be done
by doing direct i/o on the server (the -y switch?).  But how do you turn
off caching on the client kernels?  You can't. All you can do is access
the device with direct i/o there too. So it's an application-dependent
thing.


> but - reading this: http://www.polarisdb.com/rac.htm
> 
> >I looked into several open source software solutions like NBD, ENBD, DRBD, and PVFS. 
> >Unfortunately, each solution either wasn't designed to support multiple node concurrent 
> >access or didn't allow support for parallel concurrent writes. Interestingly, the author 
> >of DRBD indicates that adding this support wouldn't require much effort.
> 
> drives me crazy a little bit.

But you CAN do it provided you promise to use direct i/o (OPEN_DIRECT)
on the clients at the application level and use direct i/o on the server
too (-y ?)

> So - multiple node concurrent access doesn`t even work with NBD ? 
> How can that work, what the author from the first link did?

I don't know. You can tell me!

> Is a NBD suitable for shared storage and can NBD/ENBD used for this?

Only if nether kernel on either side caches. That can be achieved as I
indicated above (I also had a patch for the mount system that converts
every open on a device into a direct i/o open, but this is difficult to
reconcile with most file systems since directi/o requires aligned reads
and writes on page boundaries, and no fs I know takes care to do
that! You really need gfs).


> Could please someone give a comment about this?

Well, it's possible as above. 

> wish you merry christmas!

Send money care of the elves :).

Peter