Re: Need help to run UM3

--0__=0ABBFA3FDFF22A718f9e8a93df938690918c0ABBFA3FDFF22A71
Content-type: text/plain; charset=US-ASCII


Horms,
I have the setting in ha.cf and all settings are identical on both
directors.  Here is the log files:

Node1(dmccm1, the primary):
--------------------------------------------------------------------------------------------

heartbeat: 2005/11/01_12:45:29 info: Configuration validated. Starting
heartbeat 1.2.3.cvs.20050404
heartbeat: 2005/11/01_12:45:29 info: heartbeat: version 1.2.3.cvs.20050404
heartbeat: 2005/11/01_12:45:29 info: Heartbeat generation: 40
heartbeat: 2005/11/01_12:45:29 info: UDP Broadcast heartbeat started on
port 694 (694) interface eth1
heartbeat: 2005/11/01_12:45:29 info: ping heartbeat started.
heartbeat: 2005/11/01_12:45:29 info: pid 27569 locked in memory.
heartbeat: 2005/11/01_12:45:29 info: Local status now set to: 'up'
heartbeat: 2005/11/01_12:45:30 info: pid 27572 locked in memory.
heartbeat: 2005/11/01_12:45:30 info: pid 27573 locked in memory.
heartbeat: 2005/11/01_12:45:30 info: pid 27574 locked in memory.
heartbeat: 2005/11/01_12:45:30 info: Link dmccm1:eth1 up.
heartbeat: 2005/11/01_12:45:30 info: pid 27575 locked in memory.
heartbeat: 2005/11/01_12:45:30 info: pid 27576 locked in memory.
heartbeat: 2005/11/01_12:45:30 info: Link x.x.x.1:x.x.x.1 up.
heartbeat: 2005/11/01_12:45:30 info: Status update for node x.x.x.1: status
ping
heartbeat: 2005/11/01_12:45:44 info: Link dmccm2:eth1 up.
heartbeat: 2005/11/01_12:45:44 info: Status update for node dmccm2: status
up
heartbeat: 2005/11/01_12:45:44 info: Local status now set to: 'active'
heartbeat: 2005/11/01_12:45:44 info: Starting child client
"/usr/lib/heartbeat/ipfail" (500,500)
heartbeat: 2005/11/01_12:45:44 info: Starting "/usr/lib/heartbeat/ipfail"
as uid 500  gid 500 (pid 27579)
heartbeat: 2005/11/01_12:45:44 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:46:44 info: Status update for node dmccm2: status
active
heartbeat: 2005/11/01_12:46:44 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:46:54 info: remote resource transition completed.
heartbeat: 2005/11/01_12:46:54 info: remote resource transition completed.
heartbeat: 2005/11/01_12:46:54 info: Initial resource acquisition complete
(T_RESOURCES(us))
heartbeat: 2005/11/01_12:46:54 info: Local Resource acquisition completed.
heartbeat: 2005/11/01_12:46:54 info: Running /etc/ha.d/rc.d/ip-request-resp
ip-request-resp
heartbeat: 2005/11/01_12:46:54 received ip-request-resp
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 OK yes
heartbeat: 2005/11/01_12:46:54 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:46:54 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.25/23/eth0/x.x.x.255 start
heartbeat: 2005/11/01_12:46:54 info: /sbin/ip -f inet addr add x.x.x.25/23
brd x.x.x.255 dev eth0
heartbeat: 2005/11/01_12:46:54 info: /sbin/ip link set eth0 up
heartbeat: 2005/11/01_12:46:54 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p
/var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.25 eth0 x.x.x.25 auto
x.x.x.25 ffffffffffff
heartbeat: 2005/11/01_12:46:55 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:46:56 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master start
heartbeat: 2005/11/01_12:46:56 info: ipvs_syncbackup down
heartbeat: 2005/11/01_12:46:56 info: ipvs_syncmaster up
heartbeat: 2005/11/01_12:46:56 info: ipvs_syncmaster obtained
heartbeat: 2005/11/01_12:46:56 info: Running /etc/ha.d/rc.d/ip-request-resp
ip-request-resp
heartbeat: 2005/11/01_12:46:56 received ip-request-resp
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 OK yes
heartbeat: 2005/11/01_12:46:56 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:46:56 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.51/23/eth0/x.x.x.255 start
heartbeat: 2005/11/01_12:46:56 info: /sbin/ip -f inet addr add x.x.x.51/23
brd x.x.x.255 dev eth0
heartbeat: 2005/11/01_12:46:56 info: /sbin/ip link set eth0 up
heartbeat: 2005/11/01_12:46:56 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p
/var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.51 eth0 x.x.x.51 auto
x.x.x.51 ffffffffffff
heartbeat: 2005/11/01_12:46:57 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:46:57 info: Running /etc/ha.d/rc.d/ip-request-resp
ip-request-resp
heartbeat: 2005/11/01_12:46:57 received ip-request-resp
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 OK yes
heartbeat: 2005/11/01_12:46:57 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:46:57 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.52/23/eth0/x.x.x.255 start
heartbeat: 2005/11/01_12:46:58 info: /sbin/ip -f inet addr add x.x.x.52/23
brd x.x.x.255 dev eth0
heartbeat: 2005/11/01_12:46:58 info: /sbin/ip link set eth0 up
heartbeat: 2005/11/01_12:46:58 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p
/var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.52 eth0 x.x.x.52 auto
x.x.x.52 ffffffffffff
heartbeat: 2005/11/01_12:46:58 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:46:59 info: Running /etc/ha.d/rc.d/ip-request-resp
ip-request-resp
heartbeat: 2005/11/01_12:46:59 received ip-request-resp
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 OK yes
heartbeat: 2005/11/01_12:46:59 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:46:59 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.53/23/eth0/x.x.x.255 start
heartbeat: 2005/11/01_12:46:59 info: /sbin/ip -f inet addr add x.x.x.53/23
brd x.x.x.255 dev eth0
heartbeat: 2005/11/01_12:46:59 info: /sbin/ip link set eth0 up
heartbeat: 2005/11/01_12:46:59 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p
/var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.53 eth0 x.x.x.53 auto
x.x.x.53 ffffffffffff
heartbeat: 2005/11/01_12:47:00 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:08 info: dmccm2 wants to go standby [all]
heartbeat: 2005/11/01_12:47:13 info: standby: acquire [all] resources from
dmccm2
heartbeat: 2005/11/01_12:47:13 info: acquire all HA resources (standby).
heartbeat: 2005/11/01_12:47:13 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:13 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:14 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:15 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:15 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:16 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:17 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:17 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:18 info: all HA resource acquisition completed
(standby).
heartbeat: 2005/11/01_12:47:18 info: Standby resource acquisition done
[all].
heartbeat: 2005/11/01_12:47:18 info: remote resource transition completed.
heartbeat: 2005/11/01_12:47:48 WARN: node x.x.x.1: is dead
heartbeat: 2005/11/01_12:47:48 info: Link x.x.x.1:x.x.x.1 dead.
heartbeat: 2005/11/01_12:47:48 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:47:52 info: dmccm2 wants to go standby [foreign]
heartbeat: 2005/11/01_12:47:57 info: standby: acquire [foreign] resources
from dmccm2
heartbeat: 2005/11/01_12:47:57 info: acquire local HA resources (standby).
heartbeat: 2005/11/01_12:47:57 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:58 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:47:58 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:59 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:48:00 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:48:00 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:48:01 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:48:01 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:48:02 info: local HA resource acquisition
completed (standby).
heartbeat: 2005/11/01_12:48:02 info: Standby resource acquisition done
[foreign].
heartbeat: 2005/11/01_12:48:02 info: remote resource transition completed.
heartbeat: 2005/11/01_12:52:23 info: Link x.x.x.1:x.x.x.1 up.
heartbeat: 2005/11/01_12:52:23 WARN: Late heartbeat: Node x.x.x.1: interval
286650 ms
heartbeat: 2005/11/01_12:52:23 info: Status update for node x.x.x.1: status
ping
heartbeat: 2005/11/01_12:52:27 info: dmccm2 wants to go standby [all]
heartbeat: 2005/11/01_12:52:32 info: standby: acquire [all] resources from
dmccm2
heartbeat: 2005/11/01_12:52:32 info: acquire all HA resources (standby).
heartbeat: 2005/11/01_12:52:32 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:32 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:52:33 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:34 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:52:35 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:35 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:52:36 info: Acquiring resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:37 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
heartbeat: 2005/11/01_12:52:38 info: all HA resource acquisition completed
(standby).
heartbeat: 2005/11/01_12:52:38 info: Standby resource acquisition done
[all].
heartbeat: 2005/11/01_12:52:38 info: remote resource transition completed.
========================================================
Node2( dmccm2, the standby)
----------------------------------------------------------------------------------

heartbeat: 2005/11/01_12:45:35 info: Configuration validated. Starting
heartbeat 1.2.3.cvs.20050404
heartbeat: 2005/11/01_12:45:35 info: heartbeat: version 1.2.3.cvs.20050404
heartbeat: 2005/11/01_12:45:35 info: Heartbeat generation: 5
heartbeat: 2005/11/01_12:45:35 info: UDP Broadcast heartbeat started on
port 694 (694) interface eth1
heartbeat: 2005/11/01_12:45:35 info: ping heartbeat started.
heartbeat: 2005/11/01_12:45:35 info: pid 12499 locked in memory.
heartbeat: 2005/11/01_12:45:35 info: Local status now set to: 'up'
heartbeat: 2005/11/01_12:45:36 info: pid 12502 locked in memory.
heartbeat: 2005/11/01_12:45:36 info: pid 12503 locked in memory.
heartbeat: 2005/11/01_12:45:36 info: pid 12504 locked in memory.
heartbeat: 2005/11/01_12:45:36 info: Link dmccm1:eth1 up.
heartbeat: 2005/11/01_12:45:36 info: Status update for node dmccm1: status
up
heartbeat: 2005/11/01_12:45:36 info: Link dmccm2:eth1 up.
heartbeat: 2005/11/01_12:45:36 info: Status update for node dmccm1: status
active
heartbeat: 2005/11/01_12:45:36 info: pid 12505 locked in memory.
heartbeat: 2005/11/01_12:45:36 info: pid 12506 locked in memory.
heartbeat: 2005/11/01_12:45:36 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:45:36 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:46:35 WARN: node 135.5.1.1: is dead
heartbeat: 2005/11/01_12:46:35 info: Local status now set to: 'active'
heartbeat: 2005/11/01_12:46:35 info: Starting child client
"/usr/lib/heartbeat/ipfail" (500,500)
heartbeat: 2005/11/01_12:46:35 info: Starting "/usr/lib/heartbeat/ipfail"
as uid 500  gid 500 (pid 12515)
heartbeat: 2005/11/01_12:46:35 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2005/11/01_12:46:45 info: local resource transition completed.
heartbeat: 2005/11/01_12:46:45 info: Initial resource acquisition complete
(T_RESOURCES(us))
heartbeat: 2005/11/01_12:46:45 info: remote resource transition completed.
heartbeat: 2005/11/01_12:46:46 info: No local resources
[/usr/lib/heartbeat/ResourceManager listkeys dmccm2] to acquire.
heartbeat: 2005/11/01_12:46:59 info: dmccm2 wants to go standby [all]
heartbeat: 2005/11/01_12:46:59 info: standby: dmccm1 can take our all
resources
heartbeat: 2005/11/01_12:46:59 info: give up all HA resources (standby).
heartbeat: 2005/11/01_12:47:00 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:00 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:00 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:00 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:01 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.25/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:01 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:01 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:01 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:01 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:02 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.51/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:02 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:02 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:02 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:02 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:03 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.52/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:03 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:03 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:03 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:03 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:04 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.53/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:04 info: all HA resource release completed
(standby).
heartbeat: 2005/11/01_12:47:04 info: Local standby process completed [all].
heartbeat: 2005/11/01_12:47:10 WARN: 1 lost packet(s) for [dmccm1] [69:71]
heartbeat: 2005/11/01_12:47:10 info: remote resource transition completed.
heartbeat: 2005/11/01_12:47:10 info: No pkts missing from dmccm1!
heartbeat: 2005/11/01_12:47:10 info: Other node completed standby takeover
of all resources.
heartbeat: 2005/11/01_12:47:44 info: dmccm2 wants to go standby [foreign]
heartbeat: 2005/11/01_12:47:44 info: standby: dmccm1 can take our foreign
resources
heartbeat: 2005/11/01_12:47:44 info: give up foreign HA resources
(standby).
heartbeat: 2005/11/01_12:47:44 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:44 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:44 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:44 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:45 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.25/23/eth0/x.x.x.255 stop

heartbeat: 2005/11/01_12:47:45 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:45 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:45 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:45 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:46 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.51/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:46 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:46 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:46 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:46 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:47 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.52/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:48 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:47:48 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:47:48 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:47:48 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:47:49 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.53/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:47:49 info: foreign HA resource release completed
(standby).
heartbeat: 2005/11/01_12:47:49 info: Local standby process completed
[foreign].
heartbeat: 2005/11/01_12:47:54 WARN: 1 lost packet(s) for [dmccm1] [96:98]
heartbeat: 2005/11/01_12:47:54 info: remote resource transition completed.
heartbeat: 2005/11/01_12:47:54 info: No pkts missing from dmccm1!
heartbeat: 2005/11/01_12:47:54 info: Other node completed standby takeover
of foreign resources.
heartbeat: 2005/11/01_12:52:18 info: dmccm2 wants to go standby [all]
heartbeat: 2005/11/01_12:52:18 info: standby: dmccm1 can take our all
resources
heartbeat: 2005/11/01_12:52:18 info: give up all HA resources (standby).
heartbeat: 2005/11/01_12:52:19 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:19 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:52:19 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:52:19 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:52:20 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.25/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:52:20 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:20 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:52:20 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:52:20 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:52:21 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.51/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:52:21 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:21 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:52:21 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:52:21 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:52:22 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.52/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:52:22 info: Releasing resource group: dmccm1
IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf
LVSSyncDaemonSwap::master
heartbeat: 2005/11/01_12:52:22 info: Running
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
heartbeat: 2005/11/01_12:52:22 info: ipvs_syncmaster released
heartbeat: 2005/11/01_12:52:22 info: Running
/etc/ha.d/resource.d/ldirectord ldirectord.cf stop
heartbeat: 2005/11/01_12:52:23 info: Running /etc/ha.d/resource.d/IPaddr2
x.x.x.53/23/eth0/x.x.x.255 stop
heartbeat: 2005/11/01_12:52:23 info: all HA resource release completed
(standby).
heartbeat: 2005/11/01_12:52:23 info: Local standby process completed [all].
heartbeat: 2005/11/01_12:52:29 WARN: 1 lost packet(s) for [dmccm1]
[238:240]
heartbeat: 2005/11/01_12:52:29 info: remote resource transition completed.
heartbeat: 2005/11/01_12:52:29 info: No pkts missing from dmccm1!
heartbeat: 2005/11/01_12:52:29 info: Other node completed standby takeover
of all resources.
============================================================



Lin Wang
Information Services
SUNY, Downstate Medical Center

[email protected] wrote: -----

      To: [email protected]
      From: Horms <[email protected]>
      Sent by: [email protected]
      Date: 11/01/2005 12:09AM
      Subject: Re: Need help to run UM3

      On Mon, Oct 31, 2005 at 04:11:52PM -0500, Lin Wang wrote:
      >
      > Hi, Horms,
      > Another question:
      > The following setting never worked on my Linux directors,
      > Ping x.x.x.1
      > Respawn hacluster /usr/lib/heartbeat/ipfail
      > The resource services were turned off on the primary when the ping
      was not
      > answered, but the standby did not take over.  At the mean time, the
      fail
      > over to the standby director  worked if I turned off heartbeat on
      the
      > primary.
      > Appreciate your input for where I should check.

      Do you have the following in your config?

      apiauth ipfail uid=hacluster

      Are the configs on both hosts identical?

      Can you send the logs from both machines?

      --
      Horms


      --
      Ultra Monkey - http://www.ultramonkey.org/
      To UNSUBSCRIBE, email to [email protected], with a body:
      unsubscribe ultramonkey-users [email protected]
      where "[email protected]" is YOUR email address.

--0__=0ABBFA3FDFF22A718f9e8a93df938690918c0ABBFA3FDFF22A71
Content-type: text/html; charset=US-ASCII
Content-Disposition: inline

<html><body>
<p><font size="4">Horms,</font><font size="4"><br>
</font><font size="4">I have the setting in ha.cf and all settings are identical on both directors.  Here is the log files:</font><font size="4"><br>
</font><font size="4"><br>
</font><font size="4">Node1(dmccm1, the primary):</font><font size="4"><br>
</font><font size="4">--------------------------------------------------------------------------------------------</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: Configuration validated. Starting heartbeat 1.2.3.cvs.20050404</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: heartbeat: version 1.2.3.cvs.20050404</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: Heartbeat generation: 40</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: UDP Broadcast heartbeat started on port 694 (694) interface eth1</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: ping heartbeat started.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: pid 27569 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:29 info: Local status now set to: 'up'</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: pid 27572 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: pid 27573 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: pid 27574 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: Link dmccm1:eth1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: pid 27575 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: pid 27576 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: Link x.x.x.1:x.x.x.1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:30 info: Status update for node x.x.x.1: status ping</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Link dmccm2:eth1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Status update for node dmccm2: status up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Local status now set to: 'active'</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Starting child client &quot;/usr/lib/heartbeat/ipfail&quot; (500,500)</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Starting &quot;/usr/lib/heartbeat/ipfail&quot; as uid 500  gid 500 (pid 27579)</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:44 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:44 info: Status update for node dmccm2: status active</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:44 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: Initial resource acquisition complete (T_RESOURCES(us))</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: Local Resource acquisition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 received ip-request-resp IPaddr2::x.x.x.25/23/eth0/x.x.x.255 OK yes</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.25/23/eth0/x.x.x.255 start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: /sbin/ip -f inet addr add x.x.x.25/23 brd x.x.x.255 dev eth0</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 info: /sbin/ip link set eth0 up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:54 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.25 eth0 x.x.x.25 auto x.x.x.25 ffffffffffff</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:55 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: ipvs_syncbackup down</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: ipvs_syncmaster up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: ipvs_syncmaster obtained</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 received ip-request-resp IPaddr2::x.x.x.51/23/eth0/x.x.x.255 OK yes</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.51/23/eth0/x.x.x.255 start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: /sbin/ip -f inet addr add x.x.x.51/23 brd x.x.x.255 dev eth0</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 info: /sbin/ip link set eth0 up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:56 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.51 eth0 x.x.x.51 auto x.x.x.51 ffffffffffff</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:57 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:57 info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:57 received ip-request-resp IPaddr2::x.x.x.52/23/eth0/x.x.x.255 OK yes</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:57 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:57 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.52/23/eth0/x.x.x.255 start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:58 info: /sbin/ip -f inet addr add x.x.x.52/23 brd x.x.x.255 dev eth0</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:58 info: /sbin/ip link set eth0 up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:58 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.52 eth0 x.x.x.52 auto x.x.x.52 ffffffffffff</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:58 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 received ip-request-resp IPaddr2::x.x.x.53/23/eth0/x.x.x.255 OK yes</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.53/23/eth0/x.x.x.255 start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: /sbin/ip -f inet addr add x.x.x.53/23 brd x.x.x.255 dev eth0</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: /sbin/ip link set eth0 up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.53 eth0 x.x.x.53 auto x.x.x.53 ffffffffffff</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:00 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:08 info: dmccm2 wants to go standby [all]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:13 info: standby: acquire [all] resources from dmccm2</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:13 info: acquire all HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:13 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:13 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:14 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:15 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:15 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:16 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:17 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:17 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:18 info: all HA resource acquisition completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:18 info: Standby resource acquisition done [all].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:18 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 WARN: node x.x.x.1: is dead</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: Link x.x.x.1:x.x.x.1 dead.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:52 info: dmccm2 wants to go standby [foreign]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:57 info: standby: acquire [foreign] resources from dmccm2</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:57 info: acquire local HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:57 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:58 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:58 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:59 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:00 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:00 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:01 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:01 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:02 info: local HA resource acquisition completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:02 info: Standby resource acquisition done [foreign].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:48:02 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 info: Link x.x.x.1:x.x.x.1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 WARN: Late heartbeat: Node x.x.x.1: interval 286650 ms</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 info: Status update for node x.x.x.1: status ping</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:27 info: dmccm2 wants to go standby [all]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:32 info: standby: acquire [all] resources from dmccm2</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:32 info: acquire all HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:32 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:32 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:33 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:34 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:35 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:35 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:36 info: Acquiring resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:37 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:38 info: all HA resource acquisition completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:38 info: Standby resource acquisition done [all].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:38 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">========================================================</font><font size="4"><br>
</font><font size="4">Node2</font><font size="4">( dmccm2, the standby)<br>
</font><font size="4">----------------------------------------------------------------------------------</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: Configuration validated. Starting heartbeat 1.2.3.cvs.20050404</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: heartbeat: version 1.2.3.cvs.20050404</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: Heartbeat generation: 5</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: UDP Broadcast heartbeat started on port 694 (694) interface eth1</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: ping heartbeat started.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: pid 12499 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:35 info: Local status now set to: 'up'</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: pid 12502 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: pid 12503 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: pid 12504 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Link dmccm1:eth1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Status update for node dmccm1: status up</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Link dmccm2:eth1 up.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Status update for node dmccm1: status active</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: pid 12505 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: pid 12506 locked in memory.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:45:36 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:35 WARN: node 135.5.1.1: is dead</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:35 info: Local status now set to: 'active'</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:35 info: Starting child client &quot;/usr/lib/heartbeat/ipfail&quot; (500,500)</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:35 info: Starting &quot;/usr/lib/heartbeat/ipfail&quot; as uid 500  gid 500 (pid 12515)</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:35 info: Running /etc/ha.d/rc.d/status status</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:45 info: local resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:45 info: Initial resource acquisition complete (T_RESOURCES(us))</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:45 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:46 info: No local resources [/usr/lib/heartbeat/ResourceManager listkeys dmccm2] to acquire.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: dmccm2 wants to go standby [all]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: standby: dmccm1 can take our all resources</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:46:59 info: give up all HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:00 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:00 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:00 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:00 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:01 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.25/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:01 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:01 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:01 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:01 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:02 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.51/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:02 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:02 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:02 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:02 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:03 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.52/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:03 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:03 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:03 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:03 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:04 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.53/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:04 info: all HA resource release completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:04 info: Local standby process completed [all].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:10 WARN: 1 lost packet(s) for [dmccm1] [69:71]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:10 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:10 info: No pkts missing from dmccm1!</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:10 info: Other node completed standby takeover of all resources.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: dmccm2 wants to go standby [foreign]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: standby: dmccm1 can take our foreign resources</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: give up foreign HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:44 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:45 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.25/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:45 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:45 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:45 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:45 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:46 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.51/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:46 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:46 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:46 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:46 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:47 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.52/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:48 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:49 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.53/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:49 info: foreign HA resource release completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:49 info: Local standby process completed [foreign].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:54 WARN: 1 lost packet(s) for [dmccm1] [96:98]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:54 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:54 info: No pkts missing from dmccm1!</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:47:54 info: Other node completed standby takeover of foreign resources.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:18 info: dmccm2 wants to go standby [all]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:18 info: standby: dmccm1 can take our all resources</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:18 info: give up all HA resources (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:19 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.25/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:19 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:19 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:19 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:20 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.25/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:20 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.51/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:20 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:20 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:20 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:21 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.51/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:21 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.52/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:21 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:21 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:21 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:22 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.52/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:22 info: Releasing resource group: dmccm1 IPaddr2::x.x.x.53/23/eth0/x.x.x.255 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:22 info: Running /etc/ha.d/resource.d/LVSSyncDaemonSwap master stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:22 info: ipvs_syncmaster released</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:22 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 info: Running /etc/ha.d/resource.d/IPaddr2 x.x.x.53/23/eth0/x.x.x.255 stop</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 info: all HA resource release completed (standby).</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:23 info: Local standby process completed [all].</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:29 WARN: 1 lost packet(s) for [dmccm1] [238:240]</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:29 info: remote resource transition completed.</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:29 info: No pkts missing from dmccm1!</font><font size="4"><br>
</font><font size="4">heartbeat: 2005/11/01_12:52:29 info: Other node completed standby takeover of all resources.</font><font size="4"><br>
</font><font size="4">============================================================</font><font size="4"><br>
</font><br>
<font size="4"><br>
</font><font size="4"><br>
</font><font size="4">Lin Wang</font><font size="4"><br>
</font><font size="4">Information Services</font><font size="4"><br>
</font><font size="4">SUNY, Downstate Medical Center</font><br>
<br>
<font size="4" color="#990099">[email protected] wrote: -----</font><font size="4" color="#990099"><br>
</font>
<ul>
<ul><font size="4">To: [email protected]</font><font size="4"><br>
</font><font size="4">From: Horms &lt;[email protected]&gt;</font><font size="4"><br>
</font><font size="4">Sent by: [email protected]</font><font size="4"><br>
</font><font size="4">Date: 11/01/2005 12:09AM</font><font size="4"><br>
</font><font size="4">Subject: Re: Need help to run UM3</font><font size="4"><br>
</font><tt><font size="4"><br>
</font></tt><tt><font size="4">On Mon, Oct 31, 2005 at 04:11:52PM -0500, Lin Wang wrote:</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; </font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; Hi, Horms,</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; Another question:</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; The following setting never worked on my Linux directors,</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; Ping x.x.x.1</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; Respawn hacluster /usr/lib/heartbeat/ipfail</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; The resource services were turned off on the primary when the ping was not</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; answered, but the standby did not take over. &nbsp;At the mean time, the fail</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; over to the standby director &nbsp;worked if I turned off heartbeat on the</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; primary.</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">&gt; Appreciate your input for where I should check.</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">Do you have the following in your config?</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">apiauth ipfail uid=hacluster</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">Are the configs on both hosts identical?</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">Can you send the logs from both machines?</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">-- </font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">Horms</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">-- </font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">Ultra Monkey - </font></tt><a href="http://www.ultramonkey.org/"><tt><u><font size="4" color="#0000FF">http://www.ultramonkey.org/ </font></u></tt></a><tt><font size="4"><br>
</font></tt><tt><font size="4">To UNSUBSCRIBE, email to [email protected], with a body:</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">unsubscribe ultramonkey-users [email protected]</font></tt><tt><font size="4"><br>
</font></tt><tt><font size="4">where &quot;[email protected]&quot; is YOUR email address.</font></tt><tt><font size="4"><br>
</font></tt></ul>
</ul>
</body></html>
--0__=0ABBFA3FDFF22A718f9e8a93df938690918c0ABBFA3FDFF22A71--


-- 
Ultra Monkey - http://www.ultramonkey.org/
To UNSUBSCRIBE, email to [email protected], with a body:
unsubscribe ultramonkey-users [email protected]
where "[email protected]" is YOUR email address.