Graceful failover or how to eliminate failover downtime in keepalived

Pim Rupert <[email protected]>
Newsgroups gmane.linux.keepalived.devel
Message-ID <[email protected]>
Hi,

Whenever there is a failover between two machines with keepalived there is a small window of downtime (about 2 seconds), because the VIP is removed by keepalived first from node A, and added later by keepalived on node B. When a failover is started due to a priority election or scheduled maintenance it seems absurd to tolerate this two seconds of downtime. How do we properly prevent this downtime? 

I know this isn't a problem that is up to Keepalived to fix, but I am keen to learn how you deal with this problem. 

Examples:

1. Priority election (approx. two seconds downtime):

22:54:53 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) forcing a new MASTER election
22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Received higher prio advert
22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Entering BACKUP STATE
22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) removing protocol VIPs.
22:54:53 node2 Keepalived_healthcheckers[2465]: Netlink reflector reports IP 10.0.0.100 removed
# Downtime started
22:54:54 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Transition to MASTER STATE
22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Entering MASTER STATE
22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) setting protocol VIPs.
# Still down!
22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Sending gratuitous ARPs on eth1 for 10.0.0.100
22:54:55 node1 Keepalived_healthcheckers[2476]: Netlink reflector reports IP 10.0.0.100 added
# Finally, we're up again!

2. Scheduled downtime (again, look at all the unnecessary downtime):

07:08:44 node1 Keepalived_vrrp[3045]: VRRP_Script(wsrepready) failed
07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) Entering FAULT STATE
07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) removing protocol VIPs.
07:08:45 node1 Keepalived_healthcheckers[3044]: Netlink reflector reports IP 10.0.0.100 removed
# Downtime started
07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) Now in FAULT state
07:08:46 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Transition to MASTER STATE
07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Entering MASTER STATE
07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) setting protocol VIPs.
07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Sending gratuitous ARPs on eth1 for 10.0.0.100
07:08:47 node2 Keepalived_healthcheckers[2465]: Netlink reflector reports IP 10.0.0.100 added
# Up again! 


Best regards,

Pim



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.