a few concepts

I have been  trying to setup failover services for a couple of days (on 
and off).
Initially I was using 3 machines, a primary node, a backup node and 1
real (web) server, and then changed to just having having 2, a primary 
node and a server.
In the last  situation, where I had 2 machines ,  the primary always 
took over ("nanny" did not
start either). With the 2 nodes + server, the primary started fine : it 
did not assume the IP
of the  real web server immediately, but soon took over.

What I am trying to do is have a setup where if the web server 
(wilma.example.com/
/192.168.1.4) fails, then the primary node 192.168.1.3 takes over.

What am I missing in my setup? Is using private ip's a a bad idea? Do I 
have to have
a back up node?

(All the machines have public IP's but can't be reached directly as they 
are behind a
firewall, I can't even access those from my machine without going  
through the firewall.
That's why I have resorted to using private IP where I could.)


I have included my logs and configuration :

2 Nodes + a web server
++++++++++++++++++++++/etc/sysconfig/ha/lvs.cf+++++++++++++++++++++++++++++++
serial_no = 77
primary = NODE_1_IP
primary_private = 192.168.1.3
service = fos
backup_active = 1
backup = NODE_2_IP
backup_private = 192.168.1.1
heartbeat = 1
heartbeat_port = 539
keepalive = 18
deadtime = 72
network = direct
reservation_conflict_action = preempt
debug_level = NONE
failover wilma.example.com {
     address = 192.168.1.4 eth0:8
     vip_nmask = 255.255.255.0
     active = 1
     port = 80
     timeout = 60
     send = "GET / HTTP/1.0\r\n\r\n"
     expect = "HTTP"
     start_cmd = "/etc/rc.d/init.d/httpd start"
     stop_cmd = "/etc/rc.d/init.d/httpd stop"
}

======================================================================


The real server:
++++++++++++++++++++++/var/log/httpd/access.log 
--wilma.example.com++++++++++++++
192.168.1.3 - - [24/Jul/2002:10:59:03 +0200] "GET / HTTP/1.0" 200 2947 
"-" "-"
192.168.1.3 - - [24/Jul/2002:10:59:29 +0200] "get i" 400 - "-" "-"
192.168.1.3 - - [24/Jul/2002:10:59:58 +0200] "GET / HTTP/1.0" 200 2947 
"-" "-"
192.168.1.3 - - [24/Jul/2002:11:00:58 +0200] "GET / HTTP/1.0" 200 2947 
"-" "-"

======================================================================

The primary node:
+++++++++++++++++/var/log/messages -- dhcp4.example.com - NODE 
1+++++++++++ ++++++
Jul 24 10:59:28 dhcp4 pulse[3589]: STARTING PULSE AS MASTER
Jul 24 10:59:28 dhcp4 pulse[3589]: Starting Failover Service Monitors
Jul 24 10:59:28 dhcp4 fos[3591]: Stopping local services (if any)
Jul 24 10:59:28 dhcp4 fos[3591]: Shutting down local service 192.168.1.4:80
Jul 24 10:59:28 dhcp4 pulse: pulse startup succeeded
Jul 24 10:59:28 dhcp4 httpd: httpd shutdown failed
Jul 24 10:59:28 dhcp4 fos[3591]: Warning; shutdown of local service 
192.168.1.4:80 returned error 1
Jul 24 10:59:28 dhcp4 fos[3591]: Starting monitor for 192.168.1.4:80 
running as pid 3591
Jul 24 10:59:28 dhcp4 nanny[3619]: Failover service monitor for 
NODE_2_PUBLIC_IP:80 started
Jul 24 10:59:28 dhcp4 nanny[3619]: Remote service 192.168.1.4:80 is 
available
Jul 24 11:00:40 dhcp4 pulse[3589]: partner dead: activating failover 
services
Jul 24 11:00:40 dhcp4 fos[3591]: Shutting down due to signal 15
Jul 24 11:00:40 dhcp4 fos[3591]: Shutting down monitor for 
wilma.example.com 192.168.1.4:80 running as pid 3619
Jul 24 11:00:40 dhcp4 nanny[3619]: Terminating due to signal 15
Jul 24 11:00:40 dhcp4 fos[3591]: will now exit to notify pulse...
Jul 24 11:00:40 dhcp4 fos[3627]: Stopping local services (if any)
Jul 24 11:00:40 dhcp4 fos[3627]: Shutting down local service 192.168.1.4:80
Jul 24 11:00:40 dhcp4 httpd: httpd shutdown failed
Jul 24 11:00:40 dhcp4 fos[3627]: Warning; shutdown of local service 
192.168.1.4:80 returned error 1
Jul 24 11:00:40 dhcp4 fos[3627]: Starting local service 192.168.1.4:80 ...
Jul 24 11:00:41 dhcp4 httpd: httpd startup succeeded
Jul 24 11:00:45 dhcp4 pulse[3626]: gratuitous fos arps finished
=======================================================================

++++++++++++++++++++++/etc/rc.d/rc.lvs+++++++++++++++++++++++++++++++++++++++
#!/bin/sh

iptables -t nat -F -v
echo 1 >/proc/sys/net/ipv4/ip_forward
modprobe iptable_nat
iptables -v -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE
ipvsadm -A -t NODE_1_IP:80 -s rr
ipvsadm -a -t NODE_1_IP:80 -r 192.168.1.1:80 -m
ipvsadm -a -t NODE_1_IP:80 -r 192.168.1.2:80 -m
ipvsadm -a -t NODE_1_IP:80 -r 192.168.1.3:80 -m
ipvsadm -a -t NODE_1_IP:80 -r 192.168.1.4:80 -m
ipvsadm -a -t NODE_1_IP:80 -r 192.168.1.5:80 -m

=======================================================================
And,
++++++++++++++++++dhcp4.example.com- NODE 1+++++++++++++++++++++++++++++++
[root@dhcp4 root]# /sbin/ipvsadm -L
IP Virtual Server version 0.9.7 (size=65536)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  dhcp4.example.com:http rr
  -> 192.168.1.5:http             Masq    1      0          0        
  -> 192.168.1.4:http             Masq    1      0          0        
  -> 192.168.1.3:http             Local   1      0          0        
  -> 192.168.1.2:http             Masq    1      0          0        
  -> 192.168.1.1:http             Masq    1      0          0        

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++