GenHash problems

Jan Hugo Prins <[email protected]>
Newsgroups gmane.linux.keepalived.devel
Message-ID <[email protected]> 
Hi,

Today I tried to use version 1.2.12 of keepalived on my test
loadbalancers, but I ran into a strange issue.

In my setup I have a central docroot that is available on all webservers
and in this docroot I have a file named keepalived.html and this is used
to check the MD5HASH to see if the realserver can be used in the
cluster. The MD5HASH for all realservers is the same because they all
have the same keepalived.html.

[root@acclb02 ~]# genhash -s 2a02:b70:0:521:1::21 -p 80 -u /keepalived.html
MD5SUM = e6b253509b573359f16b9e66ecd213ca

[root@acclb02 ~]# genhash -s 2a02:b70:0:521:1::22 -p 80 -u /keepalived.html
MD5SUM = e6b253509b573359f16b9e66ecd213ca

After upgrading today I see the following in my logfile:

Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:443 from VS [2a02:b70:0:521:1:0:80:3]:443
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:80 from VS [2a02:b70:0:521:1:0:80:3]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:80 from VS [2a02:b70:0:521:1:0:80:4]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:80 from VS [2a02:b70:0:521:1:0:80:3]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:3]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:443 from VS [2a02:b70:0:521:1:0:80:3]:443
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:3]:443
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:80 from VS [2a02:b70:0:521:1:0:80:4]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:4]:80
Apr  7 10:00:30 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:80 from VS [2a02:b70:0:521:1:0:80:1]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:443 from VS [2a02:b70:0:521:1:0:80:2]:443
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:80 from VS [2a02:b70:0:521:1:0:80:1]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:1]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::21]:80 url[/keepalived.html], MD5SUM
[ec9cc97322223ff08ae42ee3e5cf81b5].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::21]:80 from VS [2a02:b70:0:521:1:0:80:2]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:80 from VS [2a02:b70:0:521:1:0:80:2]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:2]:80
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: MD5 digest
error to [2a02:b70:0:521:1::22]:80 url[/keepalived.html], MD5SUM
[ab6a4abaac8ce2930043739fdfe01477].
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Removing
service [2a02:b70:0:521:1::22]:443 from VS [2a02:b70:0:521:1:0:80:2]:443
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Lost quorum
1-0=1 > 0 for VS [2a02:b70:0:521:1:0:80:2]:443
Apr  7 10:00:31 acclb02 Keepalived_healthcheckers[1853]: Remote SMTP
server [127.0.0.1]:25 connected.

The IPv4 loadbalancers that are in the same config and share the same
keepalived.html for checking don't have this problem.

The resulting loadbalancer now looks like this:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  95.130.233.23:80 wrr persistent 50
  -> 95.130.233.24:80             Route   1      0          0        
  -> 95.130.233.25:80             Route   1      0          0        
TCP  95.130.233.26:80 wrr persistent 50
  -> 95.130.233.24:80             Route   1      0          0        
  -> 95.130.233.25:80             Route   1      0          0        
TCP  95.130.233.26:443 wrr persistent 50
  -> 95.130.233.24:443            Route   1      0          0        
  -> 95.130.233.25:443            Route   1      0          0        
TCP  95.130.233.27:80 wrr persistent 50
  -> 95.130.233.24:80             Route   1      0          0        
  -> 95.130.233.25:80             Route   1      0          0        
TCP  95.130.233.27:443 wrr persistent 50
  -> 95.130.233.24:443            Route   1      0          0        
  -> 95.130.233.25:443            Route   1      0          0        
TCP  95.130.233.28:80 wrr persistent 50
  -> 95.130.233.24:80             Route   1      0          0        
  -> 95.130.233.25:80             Route   1      0          0        
TCP  [2a02:b70:0:521:1:0:80:1]:80 wrr persistent 50
TCP  [2a02:b70:0:521:1:0:80:2]:80 wrr persistent 50
TCP  [2a02:b70:0:521:1:0:80:2]:443 wrr persistent 50
TCP  [2a02:b70:0:521:1:0:80:3]:80 wrr persistent 50
TCP  [2a02:b70:0:521:1:0:80:3]:443 wrr persistent 50
TCP  [2a02:b70:0:521:1:0:80:4]:80 wrr persistent 50


As you can see I'm missing all my IPv6 loadbalancers.

Some parts of the config:

IPv4:
# Acceptatie Load Balancer
virtual_server 95.130.233.23 80 {
    delay_loop 15
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

# acwf01:
    real_server 95.130.233.24 80 {
        weight 1
        HTTP_GET {
            url {
                        path /keepalived.html
                        digest e6b253509b573359f16b9e66ecd213ca
                        }
            connect_port 80
            connect_timeout 10
            delay_before_retry 1
                }
    }
# acwf02:
    real_server 95.130.233.25 80 {
        weight 1
        HTTP_GET {
            url {
                        path /keepalived.html
                        digest e6b253509b573359f16b9e66ecd213ca
                        }
            connect_port 80
            connect_timeout 10
            delay_before_retry 1
                }
    }

}

IPv6:
virtual_server 2a02:b70:0:521:1:0:80:1 80 {
    delay_loop 15
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

# acwf01:
    real_server 2a02:b70:0:521:1::21 80 {
        weight 1
        HTTP_GET {
            url {
                        path /keepalived.html
                        digest e6b253509b573359f16b9e66ecd213ca
                        }
            connect_port 80
            connect_timeout 10
            delay_before_retry 1
                }
    }
# acwf02:
    real_server 2a02:b70:0:521:1::22 80 {
        weight 1
        HTTP_GET {
            url {
                        path /keepalived.html
                        digest e6b253509b573359f16b9e66ecd213ca
                        }
            connect_port 80
            connect_timeout 10
            delay_before_retry 1
                }
    }
}


Version 1.2.7 that is included with Centos works fine. In the changelogs
I found some changes that might be related:

* Frank Baalbergen fixed genhash. genhash can throw a
  segmentation fault when not providing an argument
* Frank Baalbergen extended genhash code to support IPv6
* Frank Baalbergen extended genhash code to make url default
  value /, same as curl/wget
* Frank Baalbergen extended genhash code to only use default
  url when url is empty


One of the things I notice in the logfiles is that keepalived is able to
create to completly different MD5 hashes for the same content.

Jan Hugo Prins





------------------------------------------------------------------------------
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.