Re: news from the Debian Beowulf project?

On Tue, 2007-04-24 at 00:28 +0200, Fabrice Lorrain wrote:
> Adam C Powell IV a écrit :
> > On Fri, 2007-04-20 at 00:41 +0200, Fabrice Lorrain wrote:
> >> Adam C Powell IV a écrit :
> >>> Greetings,
> >>>
> >>> On Sun, 2007-04-15 at 09:35 +0200, karel lang wrote:
> >>>> hello,
> >>>> i'd like to ask, if there are some howto's or materials like that of 
> >>>> newer date about installing and maintaining the Beowulf project on 
> >>>> Debian Linux.
> >>> As far as I know, the most current Debian-specific howtos are on the
> >>> Debian Wiki: start at http://wiki.debian.org/?DebianBeowulf .  There are
> >>> many Debian-specific enhancements such as update-cluster, FAI, dsh, etc.
> >>> which make facilitate cluster management on Debian.  There is of course
> >>> still room for improvement, e.g. generating dhcp server config files
> >>> using update-cluster (bug 156810); ideas and patches are always welcome.
> >>>
> >>> Unfortunately, since the Beowulf Wiki pages were written (mostly by
> >>> yours truly), the diskless package has been orphaned, so that section is
> >>> valid through sarge but not for etch.  Would someone be interested in
> >>> redoing that section for lessdisks -- or in re-introducing diskless?
> >>> (Though the latter will not help with etch.)
> >> Hello,
> >>
> >> I would suggest to have a look at Daniel Baumann & all work on 
> >> debian-live [1] before (re)starting any work on {less}disks{less} packages.
> >>
> >> The infrastructure (debian-live + etch initrd + unionfs) provided to 
> >> build live cd makes it trivial to build diskless environnements. See the 
> >> wiki for the technical informations.
> >>
> >> As debian-live is here to stay, regrouping human effort there seems more 
> >>    effective than splitting it over several overlapping package.
> > 
> > Excellent point Fabrice.  It seems there's also an LTSP howto on the
> > Wiki.  I've marked DiskLess as deprecated.
> > 
> > On the other hand, diskless (and I presume lessdisks and LTSP, though
> > I'm still using sarge diskless packages on etch) has a lot of very nice
> > features for minimizing the hard drive space of a cluster (by not
> > repeating /usr, /bin, /sbin, /var/lib/dpkg etc. over the various client
> > images), nice maintenance tools, etc.  For example, chroot into one
> > place, apt-get update dist-upgrade install remove etc. to update the
> > main chroot, then exit the chroot, and run a simple script which updates
> > all of the images' /etc and /var directories (outside of /var/lib/dpkg).
> 
> <disclamer: I'm not a user of the cited packages, but I've a descent 
> understanding of how they work/>
> 
> I agree with you that minimizing disk storage is important. Chroot + 
> aptitude {update,upgrade} (apt-get should die). is a must have
> ( = simplicity of maintenance).
> 
> Debian-live is using unionfs [1] which allow to mount-merge a ro branch 
> with a rw one. Concretly, it meens you export your 
> /srv/etch-root_chrooted to your cluster nodes and in the initrd you 
> merge it with a rw tmpfs where you can put the few files you need to modify.
> 
> In short, you have a single image for all your client. You can maintaine 
> it with the chroot/aptitude upgrade trick.

This is interesting for /tmp, but because /etc and /var need to have
persistent files, and I'm afraid won't work as well.  For example, it's
helpful on a cluster to have persistent /var/log .

> > Debian Live seems geared toward providing an image for a single client
> > machine for a static image, so I don't know if that's what Beowulf
> > people really want.  Is it possible to update?  How can one copy over
> > multiple images, while preserving their separate configurations?
> 
> My personnal experience, but for workstations (not a cluster 
> environnement) is that there is very few files to manage 
> hardware/environnement differences.
> 
> If you really need to keep separate configuration for each node, you can 
>    maintain them on the server-side (ip or hostname directory tree) and 
> merge them in the tmpfs/unionfs on the client during the initrd 
> initialisation.

Right, then you need to put the merge in the initrd, and have tools to
manage the server-side directories.  That's exactly what diskless and
lessdisks used to do (note no lessdisks uploads since October 2005, and
it doesn't seem to be in etch)-:, and what drbl apparently does (thank
you Jaehyung).  This is the goal of LTSP (Linux Terminal Server Project)
as well, though that doesn't work with Debian at this point.

Debian-Live sounds like a very promising approach, and certainly one
with a broader user base and better prospects for long-term maintenance
than diskless.  But it's not usable for a cluster at this point, and
will take quite a bit of work to get it there.

Thanks,
-Adam
-- 
GPG fingerprint: D54D 1AEE B11C CE9B A02B  C5DD 526F 01E8 564E E4B6

Welcome to the best software in the world today cafe!
http://www.take6.com/albums/greatesthits.html


-- 
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]