adding 'replicas' to dchunks (disk chunks)

(cc'd ark-users as it might be of interest)

Folks, I've started adding some code for cheap-and-cheerful
"replicas" for the Sidai "disk chunks".  This is as a first
step toward "tapeless backups" (for which I have a lot of
writing queued up).

Background: a "disk chunk" (dchunk) is a contiguous piece of
disk/filesystem that somehow makes logical sense.  Something
like "Matt's home directory" or "slinger's disk2".  They are
(variously) what gets stuck in automount maps, or amanda
backup lists, or /etc/exports -- in fact, I generate all of
those from the dchunks info for my site.

Now I want to introduce the idea of 'replicas'.  A single
disk chunk can have various "replicas", including one
<master-copy> and zero or more other ones.  The
<master-copy> is *live*, and all the others are in various
states of staleness.  My notion is that (generated) cron
jobs will "refresh" the (non-master) replicas from the
master, at times you specify.  Probably just rsync over
ssh.  (Cheap and cheerful, remember?)

As it stands, this involves wholesale change to the format
of the dchunk xml files.  I doubt any of you care, so... :-)

The dchunk for Matt's home directory now looks like:

   <dchunk name="home-matt" xml-version="2">
   <prototypes>
     <prototype team="." name="home-ALL" />
   </prototypes>

   <master-copy> <param name="who">matt</param> </master-copy>

   <replica1> <param name="who">matt</param> </replica1>
   </dchunk>

All the home directories are mastered on wibble and have
their <replica1> on wobble, as the home-ALL prototype
specifies:

   <dchunk name="home-ALL" xml-version="2" prototype="yes">
   <prototypes>
   	<prototype team="." name="nfs-ALL" />
   </prototypes>

   <master-copy>
     <param name="who" default="no" />
   <table>
     <entry name="automount_map">home</entry>
     <entry name="user_visible_as">/home/@param:who@</entry>
     <entry name="hosting">wibble:/._disc2/home/@param:who@</entry>
     <entry name="part_of">wibble-disc2</entry>
   </table></master-copy>

   <replica1>
     <param name="who" default="no" />
   <table>
     <entry name="is_active">yes</entry>
     <entry name="automount_map">yesterday</entry>
     <entry name="user_visible_as">/yesterday/home-@param:who@</entry>
     <entry name="hosting">wobble:/._disc1/dchunk-replica1/home-@param:who@</entry>
     <entry name="part_of">wobble-disc1</entry>
   </table></replica1>
   </dchunk>

Observe how it says that Matt's *real* home directory is
available as /home/matt; and that replica1 is available
as /yesterday/home-matt.

The who-can-see-what stuff comes from the nfs-ALL prototype
(true for all NFS-mounted gubbins):

   <dchunk name="nfs-ALL" xml-version="2" prototype="yes">
   <prototypes>
       <prototype team="sidai" name="ordinary-nfs" />
       <prototype team="."     name="ALL" />
   </prototypes>

   <master-copy><table>
     <entry name="dev_or_no"> nodev </entry>
     <entry name="hard_or_soft"> hard </entry>
     <entry name="intr_or_no"> intr </entry>
     <entry name="retry"> retry=5 </entry>
     <entry name="root_access"> no </entry>
     <entry name="rw_or_ro"> rw </entry>
     <entry name="suid_or_no"> nosuid </entry>
     <entry name="unit_of_export"> no </entry>
   </table></master-copy>

   <replica1><table>
     <entry name="dev_or_no"> nodev </entry>
     <entry name="hard_or_soft"> hard </entry>
     <entry name="intr_or_no"> intr </entry>
     <entry name="retry"> retry=5 </entry>
     <entry name="root_access"> no </entry>
     <entry name="rw_or_ro"> ro </entry>
     <entry name="suid_or_no"> nosuid </entry>
     <entry name="unit_of_export"> no </entry>
   </table></replica1>
   </dchunk>

The only wrinkle in that is that replica1 stuff is mounted
read-only.  (So you can read /yesterday/home-matt, but you
can't write it.)

Obviously, I'm not finished yet (e.g. no spec for the cron
jobs is included yet).

As you dchunk experts can see, I'm trying to do everything
with plain old ARK tables, rather than a motley collection
of dchunk fields, as it was before.

Comments?

Will


_______________________________________________________________

Don't miss the 2002 Sprint PCS Application Developer's Conference
August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm