Re: VIP managed by Keepalived disappears...

Here’s a grep of “Keepalived_vrrp” from the server A logs when the VIP disappears. In this case, it actually disappeared sooner during Step 4 of the procedure I described:

Jul 10 12:01:29 sprdrsyas400 Keepalived_vrrp[2311]: Opening file '/etc/keepalived/keepalived.conf'.
Jul 10 12:01:29 sprdrsyas400 Keepalived_vrrp[2311]: Truncating auth_pass to 8 characters
Jul 10 12:01:29 sprdrsyas400 Keepalived_vrrp[2311]: Configuration is using : 64268 Bytes
Jul 10 12:01:29 sprdrsyas400 Keepalived_vrrp[2311]: Using LinkWatch kernel netlink reflector...
Jul 10 12:01:29 sprdrsyas400 Keepalived_vrrp[2311]: VRRP sockpool: [ifindex(4), proto(112), unicast(0), fd(11,12)]
Jul 10 12:01:30 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jul 10 12:01:30 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jul 10 12:01:31 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Entering MASTER STATE
Jul 10 12:01:31 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) setting protocol VIPs.
Jul 10 12:01:31 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35
Jul 10 12:01:31 sprdrsyas400 Keepalived_vrrp[2311]: IPVS: No daemon is running
Jul 10 12:01:31 sprdrsyas400 Keepalived_vrrp[2311]: IPVS: Daemon has already run
Jul 10 12:01:36 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35
Jul 10 12:03:00 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.180.88.29 removed
Jul 10 12:03:00 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.180.88.35 removed
Jul 10 12:03:00 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.181.128.65 removed
Jul 10 12:03:00 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.180.78.29 removed
Jul 10 12:03:03 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.180.88.29 added
Jul 10 12:03:05 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.181.128.65 added
Jul 10 12:03:07 sprdrsyas400 Keepalived_vrrp[2311]: Netlink reflector reports IP 10.180.78.29 added
Jul 10 12:03:09 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jul 10 12:03:09 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35
Jul 10 12:03:09 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jul 10 12:03:09 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35
Jul 10 12:04:25 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jul 10 12:04:25 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35
Jul 10 12:04:25 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jul 10 12:04:25 sprdrsyas400 Keepalived_vrrp[2311]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.180.88.35

At this point, the VIP has disappeared:

$ ip addr list eth0
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:b1:33:39 brd ff:ff:ff:ff:ff:ff
    inet 10.180.88.29/23 brd 10.180.89.255 scope global eth0

And it’s not on server B either….

From: Ranjith R [mailto:[email protected]]
Sent: Wednesday, July 09, 2014 3:45 AM
To: Masuda, Bond
Cc: [email protected]
Subject: Re: [Keepalived-devel] VIP managed by Keepalived disappears...

Hello,

On Wed, Jul 9, 2014 at 2:29 AM, Masuda, Bond <[email protected]<mailto:[email protected]>> wrote:
I have a strange situation that I don’t understand where the VIP that is managed by keepalived disappears. Here’s the setup:

RHEL 6.5 x86_64
Keepalived 1.2.13

server A:

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 51
    priority 250
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass xxxxxx
    }
    virtual_ipaddress {
        10.180.88.35
    }
}

server B:

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass xxxxxx
    }
    virtual_ipaddress {
        10.180.88.35
    }
}

Here’s the scenario:

step 1. : start keepalived on server A and server B. VIP shows up on server A
step 2. : reboot server A. VIP moves to server B
step 3. : server A completes boot cycle. VIP moves back to server A from server B
step 4. : reboot server B. as server B goes offline, VIP remains on server A
step 5. : server B boots up. VIP remains on server A.
step 6. : reboot server B. as server B goes offline, VIP disappears on server A.

Can you attach the log snippets from Server A when the VIP disappears
does this happen everytime when you perform the above steps ?

step 7. : server B boots up. VIP remains lost (neither server A nor B). both servers running keepalived but VIP is lost

At this point, if I restart keepalived on server A, the VIP comes back on server A. If I restart keepalived on server B, nothing happens.

What’s the problem? Is this a bug? How do I diagnose further?


-Bond






















------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Keepalived-devel mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/keepalived-devel

------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft

_______________________________________________
Keepalived-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/keepalived-devel