Keepalived 1.3.5 - many instances

Damien Clabaut <[email protected]> Wed, 2 Aug 2017 13:57:20 -0400
Newsgroups gmane.linux.keepalived.devel
Message-ID <[email protected]>
Hello,

I am currently testing Keepalived 1.3.5, and more specifically how it 
scales to many instances.

I generate ~200 instances (100 IPv4 + 100 IPv6). All exist on 2 physical 
machines, one (serverA) being master on all instances and the other 
(serverB) being slave on  all instances.

Using track_script, I lower the effective priority on serverA to pass 
all the instances to serverB. This works fine.

Then, if I kill keepalived (pkill keepalived) on serverB, serverA does 
not become master again.

Additional symptoms:

- On the physical interface used for all instances, I see ~500 pps when 
keepalived runs on both machines. After killing it on serverB, I see 
only 20 pps in or out of serverA.

- I get these logs on serverA:
Keepalived_vrrp[133987]: VRRP_Instance(489) forcing a new MASTER election
Keepalived_vrrp[133987]: VRRP_Instance(482) forcing a new MASTER election
Keepalived_vrrp[133987]: VRRP_Instance(682) forcing a new MASTER election
Keepalived_vrrp[133987]: VRRP_Instance(683) forcing a new MASTER election
Keepalived_vrrp[133987]: VRRP_Instance(483) forcing a new MASTER election

for every instance, about every 15 seconds.

- There are many "[keepalived] <defunct>" processes running in addition 
to several times the command I actually ran to launch Keepalived:

[root@serverA ~] ps ax | grep keepalived | grep dclabaut
  81036 ?        S      0:00 keepalived -f 
/home/dclabaut/vrrp_switching/conf_vrrp_full
133985 ?        Ss     0:00 keepalived -f 
/home/dclabaut/vrrp_switching/conf_vrrp_full
133986 ?        S      0:00 keepalived -f 
/home/dclabaut/vrrp_switching/conf_vrrp_full
133987 ?        R    127:58 keepalived -f 
/home/dclabaut/vrrp_switching/conf_vrrp_full

[root@serverA ~] ps ax | grep keepalived | grep defunct | wc -l
132

How can I debug this ? Do you have a list of tests to run (strace, etc) ?

Attached files: configuration template

Regards,

-- 
Damien Clabaut
R&D vRouter
ovh.qc.ca

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Keepalived-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/keepalived-devel
template_vrrp_instance (text/plain, 1.2 KB)
! Configuration File for keepalived

global_defs {
}

{% for vlan in vlans %}
vrrp_script offset_instance_4{{ vlan.name }} {

    script "/scripts/get_vrrp_offset.py 4{{ vlan.name }}"
    interval 5
    timeout 5
    weight -100
    fall 2
    rise 2
}

vrrp_script offset_instance_6{{ vlan.name }} {

    script "/scripts/get_vrrp_offset.py 6{{ vlan.name }}"
    interval 5
    timeout 5
    weight -100
    fall 2
    rise 2
}
{% endfor %}

{% for vlan in vlans %}
vrrp_instance 4{{ vlan.name }} {
    state BACKUP
    interface bond1.{{ vlan.name }}
    use_vmac vrrp4.{{ vlan.name }}.1
    virtual_router_id 1
    priority 110
    advert_int 1
    preempt_delay 5
    virtual_ipaddress {
        {{ vlan.ip4addr }}
    }
    mcast_src_ip {{ vlan.ip4addr }}
    unicast_src_ip {{ vlan.ip4addr }}

    track_script {
        offset_instance_4{{ vlan.name }}
    }
}
vrrp_instance 6{{ vlan.name }} {
    version 3
    native_ipv6
    state BACKUP
    interface bond1.{{ vlan.name }}
    use_vmac vrrp6.{{ vlan.name }}.1
    virtual_router_id 1
    priority 110
    advert_int 1
    preempt_delay 5
    virtual_ipaddress {
				fe80::1/64
        {{ vlan.ip6addr }}
    }


    track_script {
        offset_instance_6{{ vlan.name }}
    }
}
{% endfor %}