Re: Diskless clients?

On Tue, Sep 13, 2005 at 08:02:17AM -0400, Terry Gliedt wrote:
> In a cluster where we use a 'gateway' (single node where tasks are 
> initiated) and a number of 'client nodes' (no users defined, used only 
> to run OM tasks), do the client nodes ever page?

 not if you don't have any swap space :)  Try disabling your swap space and
see if anything bad happens with the jobs you normally (or even abnormally)
run on you cluster.

> Right now we are running with extremely minimal linux installations 
> (Debian sarge) on the client nodes. If these nodes will not page (all 
> machines have 2-4GB), then I wonder about using diskless clients?
> 
> This would allow even less effort to get a node "installed" (we'd use 
> PXE boot to get the kernel up and running) and even better we can 
> disable/remove the disk drives reducing power, heat and the occaissional 
> disk failure (our largest single instance of failure).
> 
> Will this work?  Anyone have any advice on how to do this? I've not done 
> it myself (yet :-)

 I have two clusters running diskless, netbooting a kernel and a CHAOS
ramdisk.  Here's how:
compile yourself a kernel with oM and initrd support.  You need minix
compiled in, too.
 While that's cooking over medium heat :), get CHAOS:
download the CHAOS 1.6 iso image from http://www.purehacking.com/chaos/.
loopback mount the ISO and get chaos-1.6.rdz (and the boot prompt text files
  and bzimage, the stock CHAOS kernel, if you want) out of whichever
  directory it is.
Put them in /tftpboot/chaos-1.6
put this in your /tftpboot/pxelinux.cfg/default:
### cut here
DEFAULT chaos-2427
TIMEOUT 100
DISPLAY chaos-1.6/f1_main.txt
PROMPT 1
F1 chaos-1.6/f1_main.txt
F2 chaos-1.6/f2_about.txt
F3 chaos-1.6/f3_hwreq.txt
F4 chaos-1.6/f4_boot.txt
F5 chaos-1.6/f5_ip.txt
F6 chaos-1.6/f6_copy.txt

LABEL chaos-2427
        KERNEL om/vmlinuz-2.4.27-om20041102-tab
        APPEND panic=10 ramdisk_size=12288 initrd=chaos-1.6/chaos.rdz root=/dev/ram0 rw init=/sbin/init BOOT=1 SHELL=1 OMAUTO=1 IPFILT=0 IPSEC=0 HTTPD=0 SSHD=0

# can't be called memtest86+.bin or syslinux will think it's something else
LABEL memtest86+
        kernel memtest86+
### cut here

 Set up your boot server with DHCP and TFTP servers.  I like dnsmasq, which
does DNS caching too and uses /etc/hosts and /etc/ethers, but the config
file format doesn't provide as many ways to group things into classes as ISC
dhcp3-server, I don't think.  Anyway, this is an exercise for the reader...

downsides:  can't log in, so can't use MPI (other than with the
low-performance trick of telling mpich to run everything on the master node
but with mosrun so they're actually migrated (so communication is over oM
migration communication, not normal TCP)).

 One cluster is dual-Opteron w/ 2GB of RAM in each machine.  The other has
been running for a year with a 2.4.24 oM kernel, and has been very stable.
It's on single-processor P4 desktop machines with 512MB RAM each.  They were
_very_ cheap.  All these machines have hard drives, I just don't make use of
them.  One of these days I'll set up a cluster file system like gpfs...

 I'm looking into FAI to maintain Debian installs on the hard drives of my
dual Opteron cluster, because people want to use MPI, and because then a
batch scheduler can send jobs that don't migrate well to other machines
where they can run their shell script awk/sed/R/bioinformatics app loops to
their hearts content.

 With nfsroot, it might be feasible to do that on diskless machines, esp.
with some tmpfs scratch space.  Or just use NFS to the server...  (It was
just oMFS that didn't like tmpfs, not other parts of oM, IIRC)

-- 
#define X(x,y) x##y
Peter Cordes ;  e-mail: X(peter@cor , des.ca)

"The gods confound the man who first found out how to distinguish the hours!
 Confound him, too, who in this place set up a sundial, to cut and hack
 my day so wretchedly into small pieces!" -- Plautus, 200 BC


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. 
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php