[SSI] openssi/docs/redhat/lyx INSTALL.lyx,NONE,1.6.4.2
Roger Tsang <[email protected]> Mon, 05 Jul 2010 02:12:50 +0000
| Newsgroups | gmane.linux.cluster.ssic.cvs |
|---|---|
| Message-ID | <[email protected]> |
Update of /cvsroot/ssic-linux/openssi/docs/redhat/lyx
In directory sfp-cvsdas-3.v30.ch3.sourceforge.com:/tmp/cvs-serv18788/lyx
Added Files:
Tag: OPENSSI-CENTOS
INSTALL.lyx
Log Message:
Initial import for CentOS
--- NEW FILE: INSTALL.lyx ---
#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass article
\language english
\inputencoding auto
\fontscheme default
\graphics default
\paperfontsize default
\spacing single
\papersize Default
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default
\layout Title
Installing an OpenSSI 2.0.0 Cluster on Red Hat Enterprise Linux 4
\layout Standard
These instructions describe how to install an OpenSSI cluster on Red Hat
Enterprise Linux 4 (
\begin_inset Quotes eld
\end_inset
RHEL4
\begin_inset Quotes erd
\end_inset
) with minimal hardware requirements.
All you need is two or more computers, connected with a private ethernet
switch.
This network is called the "interconnect", and should be private for security
and performance reasons.
Each individual computer in the cluster is called a "node".
\layout Standard
In this basic configuration, the first node's root filesystem is shared
with the rest of the cluster via the interconnect.
This works well for many users.
To learn more about how filesystems are shared over the interconnect, please
see
\series bold
README.cfs
\series default
.
\layout Standard
You can make your filesystems highly-available (
\begin_inset Quotes eld
\end_inset
HA
\begin_inset Quotes erd
\end_inset
) if they are on shared disk hardware that is physically connected to two
or more nodes.
This can be done with Fibre Channel or some other Storage Area Network
(
\begin_inset Quotes eld
\end_inset
SAN
\begin_inset Quotes erd
\end_inset
).
Please see
\series bold
README.hardmounts
\series default
for more information.
\layout Standard
If you do not have shared disk hardware, an alternate solution for HA filesystem
s is provided by the Distributed Replicated Block Device (
\begin_inset Quotes eld
\end_inset
DRBD
\begin_inset Quotes erd
\end_inset
) project.
This solution is provided as a supplemental download from OpenSSI.org.
\layout Standard
Note that any time another document is referenced, you can find it in the
\series bold
docs/
\series default
directory of this release tarball, as well as in your system's
\series bold
/usr/share/doc/openssi/
\series default
directory after you install OpenSSI.
\layout Standard
A good document that explains more about OpenSSI clustering in general is
\series bold
Introduction-to-SSI
\series default
.
It was written by Bruce Walker, who is the project leader for OpenSSI.
\layout Standard
This installation guide is provided in multiple formats for your convenience:
\layout Itemize
ASCII (
\series bold
INSTALL
\series default
)
\layout Itemize
PDF (
\series bold
pdf/INSTALL.pdf
\series default
)
\layout Itemize
HTML (
\series bold
html/INSTALL.html
\series default
)
\layout Itemize
Lyx source (
\series bold
lyx/INSTALL.lyx
\series default
)
\layout Section
Installing Red Hat Enterprise Linux 4
\layout Standard
These instructions assume you are doing a fresh install of OpenSSI.
If you are upgrading from an earlier version of OpenSSI for Red Hat
Enterprise Linux 4, please read
\series bold
docs/README.upgrade
\series default
.
\layout Enumerate
Install RHEL4 on the first node.
There's no need to install a distribution on any node other than the first
one.
\layout Enumerate
Feel free to either let the installer automatically partition your filesystems
or do it yourself using the provided tools.
The ext3 filesystem is preferred over ext2, because of its journalling
capabilities.
\begin_deeper
\layout Standard
\series bold
/boot
\series default
can either be its own partition or a directory on the root filesystem.
Regardless of this choice, these instructions assume that
\series bold
/boot
\series default
is located on the first partition of the first drive (e.g.,
\series bold
/dev/hda1
\series default
or
\series bold
/dev/sda1
\series default
).
\end_deeper
\layout Enumerate
Configure GRUB as the boot loader, rather than LILO.
The OpenSSI project no longer supports LILO.
\begin_deeper
\layout Standard
The Red Hat installer gives you the option to install the boot loader on
the Master Boot Record (
\begin_inset Quotes eld
\end_inset
MBR
\begin_inset Quotes erd
\end_inset
) of a particular disk, or on the boot block of a particular partition.
It is recommended that you install the boot loader on the MBR of your first
internal disk (e.g.,
\series bold
/dev/hda
\series default
or
\series bold
/dev/sda
\series default
).
\end_deeper
\layout Enumerate
Configure the cluster interconnect interface with a static IP address.
The interconnect should be on a private switch for security reasons, so
hopefully this requirement does not cause much trouble, even in a networking
environment with dynamic addresses.
\layout Enumerate
When configuring your firewall, do one of the following:
\begin_deeper
\layout Enumerate
designate as
\begin_inset Quotes eld
\end_inset
trusted
\begin_inset Quotes erd
\end_inset
the interface for the cluster interconnect
\layout Enumerate
disable the firewall
\end_deeper
\layout Section
Installing OpenSSI software
\layout Enumerate
\series bold
\begin_inset LatexCommand \label{enu:Add-driver}
\end_inset
/etc/modprobe.conf
\series default
contains a list of all local network interfaces and their network drivers.
If you plan to add a node that will need a different network driver to
connect to the cluster than what is already listed, add a line for it:
\begin_deeper
\layout LyX-Code
alias eth-extra 8139cp
\layout Standard
Add as many
\series bold
eth-extra
\series default
lines as necessary to specify the various network drivers you'll need.
\end_deeper
\layout Enumerate
Run the
\series bold
./install
\series default
script.
After it installs your packages, it will ask you a few questions about
how you want to configure your cluster and your first node:
\begin_deeper
\layout Enumerate
Enter a node number between 1 and 125.
Every node in the cluster must have a unique node number.
The first node is usually 1, although you might want to choose another
number for a reason such as where the machine is physically located.
\layout Enumerate
Select a Network Interface Card (
\begin_inset Quotes eld
\end_inset
NIC
\begin_inset Quotes erd
\end_inset
) for the cluster interconnect.
It must already be configured with an IP address and netmask before it
will appear in the list.
If the desired card has not been configured, do so in another terminal
then select (R)escan.
\begin_deeper
\layout Standard
The NIC should be connected to a private network for better security and
performance.
It should also be capable of network booting, in case anything ever happens
to the boot partition on the local hard drive.
To be network boot capable, the NIC must have a chipset supported by PXE
or Etherboot.
\end_deeper
\layout Enumerate
Select (P)XE or (E)therboot as the network boot protocol for this node.
PXE is an Intel standard for network booting, and many professional grade
NICs have a PXE implementation pre-installed on them.
You can probably enable PXE with your BIOS configuration tool.
If you do not have a NIC with PXE, you can use the open-source project
Etherboot, which lets you generate a floppy or ROM image for a variety
of different NICs.
\layout Enumerate
OpenSSI includes an integrated version of Linux Virtual Server (
\begin_inset Quotes eld
\end_inset
LVS
\begin_inset Quotes erd
\end_inset
), which lets you to configure a Cluster Virtual IP (
\begin_inset Quotes eld
\end_inset
CVIP
\begin_inset Quotes erd
\end_inset
) address that automatically load balances TCP connections across various
nodes.
This CVIP is highly available and can be configured to move to another
node in the event of a failure.
For more information, please see
\series bold
README.CVIP
\series default
.
\layout Enumerate
Enter a clustername.
It should resolve to your CVIP address, either in DNS or the cluster's
\series bold
/etc/hosts
\series default
file, if you choose to configure a CVIP.
This is required if you want to run NFS server.
For more information, please see
\series bold
README.nfs-server
\series default
.
\begin_deeper
\layout Standard
The current hostname will automatically become the nodename for this node.
\end_deeper
\layout Enumerate
Select whether you want to enable root filesystem failover.
The root must be installed on (or copied to) shared disk hardware, in order
to answer yes to this question.
If you do answer yes, then each time you add a new node, you will be asked
if the node is physically attached to the root filesystem and if it should
be configured as a root failover node (see
\series bold
openssi-config-node
\series default
later).
You can learn more about filesystem failover in
\series bold
README.hardmounts
\series default
.
\layout Enumerate
A simple mechanism for synchronizing time across the cluster will be installed.
Any time a node boots, it will synchronize its system clock with the initnode
(the node where init is running).
You can also run the ssi-timesync command at any time to force all nodes
to synchronize with the initnode.
\begin_deeper
\layout Standard
This timesync mechanism synchronizes nodes to within a second or two of
each other.
If you need a higher degree of synchronization, you can configure Network
Time Protocol (
\begin_inset Quotes eld
\end_inset
NTP
\begin_inset Quotes erd
\end_inset
) across the cluster.
Instructions for how to do this are available in
\series bold
README.ntp
\series default
.
\end_deeper
\layout Enumerate
Automatic process load balancing will be installed as part of OpenSSI.
By default, only programs launched from the
\series bold
bash-ll
\series default
shell will be load balanced.
The
\series bold
bash-ll
\series default
shell is identical to
\series bold
bash
\series default
, except for having load balancing enabled.
\begin_deeper
\layout Standard
To enable load-balancing for a program without launching it from
\series bold
bash-ll
\series default
, add its program name to
\series bold
/etc/sysconfig/loadlevellist
\series default
and run
\series bold
service loadlevel restart
\series default
.
For more information, please see
\series bold
README-mosixll
\series default
.
\end_deeper
\layout Enumerate
If you want to run X Windows, please see
\series bold
README.X-Windows
\series default
.
\end_deeper
\layout Enumerate
After your cluster is configured, you will be prompted to reboot your first
node.
You must reboot to run the OpenSSI code.
\layout Section
Adding new nodes
\layout Enumerate
A new node is added to an OpenSSI cluster using network booting.
This lets you avoid having to install a distribution on more than one node.
To network boot a new node, first select one of its NICs for the cluster
interconnect.
It must have a chipset supported by PXE or Etherboot.
\begin_deeper
\layout Standard
The DHCP server will be automatically configured and started to allow the
new nodes to join the cluster.
\end_deeper
\layout Enumerate
If the selected NIC does
\emph on
not
\emph default
support PXE booting, download an appropriate Etherboot image from the following
URL:
\begin_deeper
\layout LyX-Code
http://rom-o-matic.net/5.2.4/
\layout Standard
Choose the appropriate chipset.
Under
\series bold
Configure
\series default
it is recommended that
\series bold
ASK_BOOT
\series default
be set to
\series bold
0
\series default
.
\series bold
Floppy Bootable ROM Image
\series default
is the easiest format to use.
Just follow the instructions for writing it to a floppy.
\end_deeper
\layout Enumerate
If the node requires a network driver not already in
\series bold
/etc/modprobe.conf
\series default
, follow step
\begin_inset LatexCommand \ref{enu:Add-driver}
\end_inset
of
\series bold
Installing OpenSSI Software
\series default
(Section 2) to add it.
Then rebuild the ramdisk to include the driver and update the network boot
images:
\begin_deeper
\layout LyX-Code
# mkinitrd --cfs -f <initrd-image> <kernel-version>
\layout LyX-Code
# ssi-ksync
\end_deeper
\layout Enumerate
\begin_inset LatexCommand \label{enu:Start-new-node}
\end_inset
Connect the selected NIC to the cluster interconnect, insert an Etherboot
floppy (if needed), and boot the computer.
It should display the hardware address of the NIC it is attempting to boot
with, then hang while it waits for a DHCP server to answer its request.
\layout Enumerate
On the first node (or any node already in the cluster), run
\series bold
openssi-config-node
\series default
.
Select
\begin_inset Quotes eld
\end_inset
Add a new node
\begin_inset Quotes erd
\end_inset
.
It will ask you a few questions about how you want to configure your new
node:
\begin_deeper
\layout Enumerate
Enter a unique node number between 1 and 125.
\layout Enumerate
Select the hardware address of the new node's cluster interconnect NIC.
The list shows all unknown hardware addresses that have recently probed
the cluster's DHCP server.
Choose the one that is displayed on the console of the new node when it
attempts to network boot.
If the desired NIC is not listed, make sure the node has attempted to network
boot on the cluster interconnect, then select (R)escan.
\layout Enumerate
Enter a static IP address for the NIC.
It must be unique and it must be on the same subnet as the cluster interconnect
NICs for the other nodes.
If you want, this program can scan the subnet for IP addresses that seem
to be available.
It does this by pinging every address in the subnet, which can take awhile
for larger subnets.
You can safely skip this feature if you know one or more available IP addresses.
\layout Enumerate
Select (P)XE or (E)therboot as the network boot protocol for this node.
PXE is an Intel standard for network booting, and many professional grade
NICs have a PXE implementation pre-installed on them.
You can probably enable PXE with your BIOS configuration tool.
If you do not have a NIC with PXE, you can use the open-source project
Etherboot, which lets you generate a floppy or ROM image for a variety
of different NICs.
\layout Enumerate
Enter a nodename.
It should be unique in the cluster and it should resolve to one of this
node's IP addresses, so that client NFS can work correctly.
The nodename can resolve to either the IP address you configured above
for the interconnect, or to one of external IP addresses that you might
configure below.
The nodename can resolve to the IP address either in DNS or in the cluster's
/etc/hosts file.
\begin_deeper
\layout Standard
The nodename is stored in
\series bold
/etc/nodename
\series default
, which is a context-dependent symlink (
\begin_inset Quotes eld
\end_inset
CDSL
\begin_inset Quotes erd
\end_inset
).
In this case, the context is node number, which means each node you add
will have it's own view of
\series bold
/etc/nodename
\series default
containing its own hostname.
To learn more about CDSLs, please see the document entitled
\series bold
cdsl
\series default
.
\end_deeper
\layout Enumerate
If you enabled root failover during the first node's installation, you will
be asked if this node should be a root failover node.
This node
\emph on
must
\emph default
have access to the root filesystem on a shared disk in order to answer
yes.
If you answer yes, then this node can boot first as a root node, so you
should configure it with a local boot device.
This is done after this node joins the cluster and is described in step
\begin_inset LatexCommand \ref{enu:Local-boot-device}
\end_inset
.
\layout Enumerate
Save the configuration.
\end_deeper
\layout Enumerate
The program will now do all the work to admit the node into the cluster.
Wait for the new node to join.
A
\begin_inset Quotes eld
\end_inset
nodeup
\begin_inset Quotes erd
\end_inset
message on the first node's console will indicate this.
You can confirm its membership with the cluster command:
\begin_deeper
\layout LyX-Code
# cluster -v
\layout Standard
If the new node is hung searching for the DHCP server, try manually restarting
the dhcpd daemon on the cluster:
\layout LyX-Code
# /sbin/service dhcpd reload
\layout Standard
If the new node is still hung, reboot it.
It should boot properly and join.
\end_deeper
\layout Enumerate
\begin_inset LatexCommand \label{enu:End-new-node}
\end_inset
The following steps tell you how to configure the new node's hardware, including
its swap space, local boot device (optional unless configured for root
failover), and external NICs.
Sorry for the complexity of some of the steps.
The Red Hat installer automates most of it for you during the installation
of the first node, but it's not much help when adding new nodes.
\begin_deeper
\layout Enumerate
Configure the new node with one or more swap devices using
\series bold
fdisk
\series default
(or a similar tool) and
\series bold
mkswap
\series default
:
\begin_deeper
\layout LyX-Code
# onnode <
\emph on
node_number
\emph default
> fdisk /dev/hda
\layout LyX-Code
\emph on
partition disk
\layout LyX-Code
# onnode <
\emph on
node_number
\emph default
> mkswap /dev/hda3
\layout Standard
Add the device name(s) to the file
\series bold
/etc/fstab
\series default
, as documented in
\series bold
README.fstab
\series default
.
\layout Standard
Either reboot the node or manually activate the swap device(s) with the
\series bold
swapon
\series default
command:
\layout LyX-Code
# onnode
\emph on
<node_number>
\emph default
swapon <
\emph on
swap_device
\emph default
>
\end_deeper
\layout Enumerate
\begin_inset LatexCommand \label{enu:Local-boot-device}
\end_inset
If you have enabled root failover you MUST configure a local boot device
on the new node.
Otherwise, configuring a local boot device is optional.
If you are going to configure a local boot device, it is highly recommended
that the boot device have the same name as the first node's boot device.
Remember that we assumed at the beginning of these instructions that the
first node's boot device is located on the first partition of the first
drive (e.g.,
\series bold
/dev/hda1
\series default
or
\series bold
/dev/sda1
\series default
).
\begin_deeper
\layout Standard
Assuming you have already created a suitable partition with
\series bold
fdisk
\series default
, format your boot device with an ordinary Linux filesystem, such as ext3:
\layout LyX-Code
# onnode <
\emph on
node_number
\emph default
> mkfs.ext3 /dev/hda1
\layout Standard
Now run
\series bold
ssi-chnode
\series default
anywhere in the cluster (no need to use
\series bold
onnode
\series default
with this command).
Select the new node, enter its local boot device name, and
\series bold
ssi-chnode
\series default
will copy over the necessary files.
\layout Standard
Finally, you need to manually install a GRUB boot block on the new node:
\layout LyX-Code
# onnode <
\emph on
node_number
\emph default
> grub --device-map=/boot/grub/device.map
\layout LyX-Code
grub> root (hd0,0)
\layout LyX-Code
grub> setup (hd0)
\layout LyX-Code
grub> quit
\end_deeper
\layout Enumerate
You can configure external NICs using the
\series bold
system-config-network
\series default
command:
\begin_deeper
\layout LyX-Code
# onnode <
\emph on
node_number
\emph default
> redhat-config-network
\layout Standard
If you do not have
\series bold
system-config-network
\series default
installed (such as with a non-GUI installation), you can use the older
\series bold
netconfig
\series default
command:
\layout LyX-Code
# onnode
\emph on
<node_number>
\emph default
netconfig --device=ethX
\layout Standard
Do not attempt to configure the NIC you chose for the cluster interconnect;
you may configure any other NIC.
The
\series bold
system-config-network
\series default
command protects you from making this mistake, but the
\series bold
netconfig
\series default
command does not.
\end_deeper
\end_deeper
\layout Enumerate
Repeat steps
\begin_inset LatexCommand \ref{enu:Start-new-node}
\end_inset
-
\begin_inset LatexCommand \ref{enu:End-new-node}
\end_inset
at any time to add other nodes to the cluster.
\layout Enumerate
Enjoy your new OpenSSI cluster!!!
\begin_deeper
\layout Standard
To learn more about OpenSSI, please read
\series bold
Introduction-to-SSI
\series default
.
\layout Standard
One of the first things you can try is running the demo Bruce, Scott and
I have done at recent trade shows.
It illustrates some of the features of OpenSSI clusters.
You can find it here, along with older demos:
\layout LyX-Code
http://OpenSSI.org/#demos
\layout Standard
Recently, a scalable LTSP server has been tested on an OpenSSI cluster.
To learn more about how to set this up, please see
\series bold
README.ltsp
\series default
.
\layout Standard
If you have questions or comments that are not addressed on the website,
do not hesitate to send a message to the user's discussion forum:
\layout LyX-Code
[email protected]
\end_deeper
\layout Author
Maintained by Brian J.
Watson <[email protected]>
\the_end
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first