Re: Question about VIPs dropping from keepalived pair

Thank you very much again, sir.    I will do some reading on your
suggestions and become more familiar with the aspects of keepalived.
Again, I appreciate your time.

-Chris


-Chris

On Thu, Oct 1, 2015 at 7:41 PM, <[email protected]> wrote:

> Well the master documentation source for keepalived is here
> https://github.com/acassen/keepalived/blob/master/doc/keepalived.conf.SYNOPSIS
> Ignore all other howtos and blogs on the Internet because they are almost
> all a decade or more out of date. I know this sounds a little harsh but it
> also helps to red the VRRP RFC's because they explain a lot about the
> intent of these options, that said the are a relatively short read as far
> as RFC's go.
>
> In that doc read about the authentication section of a vrrp instance it
> will explain how the passwords works.
> Keep in mind the passwords are not encrypted they are in clear text
> because they are not meant to secure VRRP they are simply meant to prevent
> ‎accidental conflicts.
> The best practice is to use dedicated crossover cables or vlans for vrrp
> ‎communications. The reason for this is the design in the RFC'S favors
> speed over security and was really only ever meant to be used on isolated
> networks. The way to achieve this is to ust the interface and track
> interface settings aling with specifying the device on the VIP. The vip
> uses iproute2 syntax so take averythig "ip addr add" on the command-line
> and plug that in to the VIP.
>
>
> *From: *Chris
> *Sent: *Thursday, October 1, 2015 22:17
> *To: *[email protected]
> *Cc: *[email protected]
> *Subject: *Re: [Keepalived-devel] Question about VIPs dropping from
> keepalived pair
>
> prmarino1,
>
> Thank you very much for responding.  I really do appreciate it.  I will
> work on setting the priorities to be something of a gap more like of 10
> rather than 100, and then work on setting up a chart to create unique
> virtual router id's instead of them all having the same.  We'll run some
> tests and increase the number between priorities to see if we can cause
> this condition to happen on purpose, and then a series of other tests.
>
> Can you please explain to me, you mentioned having a unique password set
> on each pair also - does this allow the pair of servers to .. securely talk
> to one another with this password so that some other external source
> (another pair of keepalived hosts within the same site) can't affect it?  I
> am curious about the passwords aspect of keepalived and I'll do some
> reading on this issue, but it always helps tremendously when someone
> experienced is able to comment and supplement what documentation iterates.
>
> Take care and thanks again for your time,
>
> Chris
>
>
> -Chris
>
> On Thu, Oct 1, 2015 at 7:02 PM, <[email protected]> wrote:
>
>> On the same router ID yes that can absolutely cause a problem, and you
>> should also have a uniqe password set on each pair too.
>>
>> The interval shouldn't cause a problem and isn't unusual, in fact you are
>> more likely to have issues with smaller intervals than larger ones
>> especially if you use one of the centisecond interval patches.
>>
>> I've run keepalived on openstack before without issue; however without
>> knowing the details of your network I can not rule out network config
>> issues.
>>
>> One thing I noticed is that difference in priorities is way too big. The
>> difference should be less than 50. Most likely this contributes to your
>> issue because the priority number is used in the calculations of timeouts.
>> Also a difference of more than 50 constitutes a hard failover to the one
>> with the highest priority regardless of nopreempt and preempt delay
>> settings. Both of these things are specified in the VRRP rfc's and are not
>> specific to keepalived.
>>
>> *From: *Chris
>> *Sent: *Thursday, October 1, 2015 21:05
>> *To: *[email protected]
>> *Subject: *[Keepalived-devel] Question about VIPs dropping from
>> keepalived pair
>>
>> Hello,
>>
>> I am happy to be a part of this list now, and I apologize ahead of time
>> if this is not the correct list to post my question to about some odd
>> behavior from keepalived configured on our systems.
>>
>> *Specs*:  Ubuntu 14.04 LTS, instances running in Mirantis OpenStack
>>
>> *Keepalived version:  *Keepalived v1.2.19 (08/25,2015)
>>
>> *The problem: * VIP address randomly gets dropped requiring a restart of
>> keepalived to pick it back up again
>>
>> *Description:  *We had been running our systems out of AWS for the
>> longest times without this happening.   We have recently moved to an
>> OpenStack solution and have a pair of haproxy servers which answer requests
>> for a slew of external facing webapps.  We have haproxy servers for both
>> internal and external services, and this has happened on all of our
>> keepalived pairs, regardless of what services they serve.
>>
>> I have scowered the net trying to find someone with a similar situation
>> as this and have come up empty handed so I thought to ask the all knowing
>> community.
>>
>> A couple of points that I have noticed within our keepalived.conf files:
>>
>>
>>    - We have advert_int set to 10.  I've read that the default value is
>>    1.   I don't know why whoever configured these prior had this value set to
>>    10 and I am wondering what kinds of problems can occur if this value is set
>>    too high?
>>    - I have also noticed that the virtual_router_id is set to the same
>>    integer for all of our pairs of keepalived servers, again, I don't know why
>>    it was configured this way, but I'm curious what kind of problems this can
>>    cause if any with this being the same value.
>>
>>
>> Keepalived.conf configs:
>>
>> *MASTER*
>>
>> root@haproxy-external-02:~# cat /etc/keepalived/keepalived.conf
>> global_defs {
>>   router_id haproxy-external-02
>> }
>> vrrp_instance external_multi_app { #set to cluster unique id
>>   state MASTER #primary peer(s) should be set to MASTER
>>   priority 200 #set to 200 if master
>>   unicast_src_ip 10.152.146.7
>>   unicast_peer {
>>     10.152.146.5
>>   }
>>   #vrrp_unicast_peer 10.152.146.5
>>   #nopreempt #will only work if both are set to BACKUP
>>   interface eth0
>>   #vrrp_unicast_bind 10.152.146.7
>>
>>
>>
>>   *virtual_router_id 42* #unique router id
>>   *advert_int 10*
>>   virtual_ipaddress {
>>     10.152.146.91
>>   }
>> }
>>
>>
>>
>> *BACKUP*
>>
>> root@haproxy-external-01:~# cat /etc/keepalived/keepalived.conf
>> global_defs {
>>   router_id haproxy-external-01
>> }
>> vrrp_instance external_multi_app { #set to cluster unique id
>>   state BACKUP #secondary peer(s) should be set to BACKUP
>>   priority 100 #set to 100 if backup
>>   unicast_src_ip 10.152.146.5
>>   unicast_peer {
>>     10.152.146.7
>>   }
>>   #vrrp_unicast_peer 10.152.146.7
>>   #nopreempt #will only work if both are set to BACKUP
>>   interface eth0
>>   #vrrp_unicast_bind 10.152.146.5
>>
>>
>>
>>   *virtual_router_id 42* #unique router id
>>  * advert_int 10*
>>   virtual_ipaddress {
>>     10.152.146.91
>>   }
>> }
>>
>>
>> *Summary: *I just am not sure if this is an OpenStack related problem,
>> or if keepalived is misconfigured in a way that not having enough
>> advertisements could cause the VIP to just go away.  I do know also, that
>> this symptom occurred when we have had to reboot our switch, the VIPs would
>> drop off and restarting keepalived was required - which is kind of
>> expected, I guess.  But seeing this happen while the switch is up and
>> running is bizarre to me and I am having a hell of a time trying to
>> recreate it, aside from having constant straces running on the PIDs with
>> the hopes that it will happen, I have not been that luck yet.
>>
>>
>> I appreciate your time and I am hoping that perhaps we have just
>> misconfigured the conf file, or if someone else has perhaps come across
>> this situation of VIPs just dropping.  When it happens, of course it is an
>> outage for all services attached to these haproxy servers so its quite bad
>> when this happens.
>>
>> Thank you very much again,
>>
>> Sincerely,
>>
>> Chris
>>
>>
>>
>
>

------------------------------------------------------------------------------

_______________________________________________
Keepalived-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/keepalived-devel