Re: keepalived and DNS

On Tue, Apr 07, 2015 at 01:30:52PM +0200, Anders Henke wrote:
> On 06.04.2015, Ryan O'Hara wrote:
> > I have a problem where keepalived is running in a network namespace
> > that does not have access to DNS. The end result is that we see some
> > flapping between master/backup state whenever a restart occurs. I
> > believe I know why.
> > 
> > On restart, keepalived will allocate new global data, which includes
> > the router ID and default "email_from" value. Both of there are
> > subject to DNS lookups since both will call get_local_name() which
> > in-turn calls getaddrinfo. If keepalived does not have access to DNS,
> > getaddrinfo will block and we see some flapping between master/backup
> > states. At least this is my theory.
> 
> Just out of curiosity, I did block outgoing DNS traffic on my workstation and
> did a few quick tests with getaddrinfo(). My /etc/nsswitch.conf does also prefer files over DNS, but that's a default behaviour for most linux systems.
> 
> Result: as long as my host's nodename and its canonical name are configured correctly in /etc/hosts, getaddrinfo() does correctly translate from nodename (uname.nodename) to canonical name (FQDN).
> 
> $ grep hosts /etc/nsswitch.conf 
> hosts:          files dns mdns4_minimal [NOTFOUND=return] mdns4
> $ iptables -I OUTPUT -p udp --dport 53 -j DROP
> $ iptables -I OUTPUT -p tcp --dport 53 -j DROP
> $ socket_getaddrinfo --canonical --host `uname --nodename`
> Resolved host 'ista', service '0'
>   Canonical name 'ista.s.schlund.de'
> 
> [...]
> 
> ... and /etc/hosts does list
> 
> 172.17.24.42 ista.s.schlund.de ista
> 
> ... so everything is working as expected - including "no access to DNS".
> 
> Maybe this simple approach does solve your issue: configure your nodename 
> in /etc/hosts.
>  
> > My first inclination is to remove getaddrinfo completely. The result
> > would be that router_id would default to <uname> and the default
> > email_from would be root@<uname>. I don't think this would cause any
> > real problems, but wanted to pitch this idea before submitting the
> > patch.
> 
> If you don't have access to DNS, you should still be able to manage /etc/hosts.

Right. I believe this is the current workaround. I'd have to check
with the people that reported this to problem from the OpenStack
community. It would be best for them to comment.

> Regarding default email_from, I'd suppose to make use of an empty sender
> ("MAIL FROM: <>" in SMTP). By doing so, your smarthost/mailrelay won't be spending
> time on delivering out of office replies or bounces to the inaccecssible smtp service on one's loadbalancer (or spewing those messages at a ticket system, an employee, ...).
> 
> However, the From-header, Subject or whatever else in the actual rfc822-message still might/should use the system's canonical name by default.

Then set the notification_email_from address to the correct
value. Problem solved.

All of your points are fine, but it doesn't change the fact that
getaddrinfo is called twice (once for default router_id and once for
default notification_email_from), and these can potentially block,
which causes the flapping. I think it would be best to avoid this. I
also do not thing a canonical name is needed for default router_id. In
addition, default email_from should only be set if smtp notifications
are enabled. You can always explicitly set this value, too. So we
could remove getaddrinfo.

Thanks for the feedback.

Ryan

------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF