Graceful failover or how to eliminate failover downtime in keepalived
Pim Rupert <[email protected]>
| Newsgroups | gmane.linux.keepalived.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi, Whenever there is a failover between two machines with keepalived there is a small window of downtime (about 2 seconds), because the VIP is removed by keepalived first from node A, and added later by keepalived on node B. When a failover is started due to a priority election or scheduled maintenance it seems absurd to tolerate this two seconds of downtime. How do we properly prevent this downtime? I know this isn't a problem that is up to Keepalived to fix, but I am keen to learn how you deal with this problem. Examples: 1. Priority election (approx. two seconds downtime): 22:54:53 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) forcing a new MASTER election 22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Received higher prio advert 22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Entering BACKUP STATE 22:54:53 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) removing protocol VIPs. 22:54:53 node2 Keepalived_healthcheckers[2465]: Netlink reflector reports IP 10.0.0.100 removed # Downtime started 22:54:54 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Transition to MASTER STATE 22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Entering MASTER STATE 22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) setting protocol VIPs. # Still down! 22:54:55 node1 Keepalived_vrrp[2477]: VRRP_Instance(VI_5) Sending gratuitous ARPs on eth1 for 10.0.0.100 22:54:55 node1 Keepalived_healthcheckers[2476]: Netlink reflector reports IP 10.0.0.100 added # Finally, we're up again! 2. Scheduled downtime (again, look at all the unnecessary downtime): 07:08:44 node1 Keepalived_vrrp[3045]: VRRP_Script(wsrepready) failed 07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) Entering FAULT STATE 07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) removing protocol VIPs. 07:08:45 node1 Keepalived_healthcheckers[3044]: Netlink reflector reports IP 10.0.0.100 removed # Downtime started 07:08:45 node1 Keepalived_vrrp[3045]: VRRP_Instance(VI_5) Now in FAULT state 07:08:46 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Transition to MASTER STATE 07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Entering MASTER STATE 07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) setting protocol VIPs. 07:08:47 node2 Keepalived_vrrp[2466]: VRRP_Instance(VI_5) Sending gratuitous ARPs on eth1 for 10.0.0.100 07:08:47 node2 Keepalived_healthcheckers[2465]: Netlink reflector reports IP 10.0.0.100 added # Up again! Best regards, Pim ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e