Re: Other strange LB balancing issue

Hello,

Do you think the ARP problem can be a consequence of all my problem ??? 
I didn't applied the hidden patch on my real servers... ?

I'm using direct routing, do you think NAT could be work better ?

Thanks,

Cyrille.





Cyrille Mertes wrote:

Hello,


Actual LB : redhat 8.0, kernel 2.4.18-14, IP Virtual Server version 
1.0.4 (source), piranha-0.7.0-3 (source)
Old configuration : redhat 7.2, kernel 2.4.18, IP Virtual Server version 
1.0.1 (rpm source), piranha-0.6.1-1 (rpm source)

I see a message of Jeremy Kusnetz on 11/18/2002 that looks a lot like a
problem that we have. I don't know if he found a solution. We have been
setuping LVS on different servers and always obtaining the same issue.

Our first unsuccesfull installation was with UltraMonkey, after we 
decided to
try with Piranha to see if our problem would go away, but the problem is 
still
present. We also installed it on different kernels, with RPMs, with the
source, and we always have this problem.
The problem is that the server stops balancing the requests, some times it
stops only on one virtual server and continues balancing correctly on the
others. When it stops balancing it sends all the requests to one of the
real servers (always the same) and stops showing anything in ipvsadm.
It can stop 3 times in one hour, and after it can take 4 hours before 
stopping again.

Here is output of ipvsadm when everything is ok :

IP Virtual Server version 1.0.4 (size=65536)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  216.187.127.124:http wrr
 -> gragra.privatedns.com:http     Route   1      170        398
 -> koon.privatedns.com:http       Route   1      122        212
 -> motti.privatedns.com:http      Route   1      121        233
TCP  216.187.127.122:http wrr
 -> motti.privatedns.com:http      Route   2      277        852
 -> gragra.privatedns.com:http     Route   1      150        433
 -> koon.privatedns.com:http       Route   2      236        837

Here is one output of ipvsadm where you can see that IP .122 stopped 
balancing the requests :


IP Virtual Server version 1.0.4 (size=65536)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  216.187.127.124:http wrr
 -> gragra.privatedns.com:http     Route   1      170        398
 -> koon.privatedns.com:http       Route   1      122        212
 -> motti.privatedns.com:http      Route   1      121        233
TCP  216.187.127.122:http wrr


Here is my pirnha lvs.conf :

serial_no = 344
primary = 216.187.127.77
primary_private = 192.168.6.21
service = lvs
backup_active = 0
backup = 216.187.127.117
backup_private = 192.168.6.10
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 15
network = direct
nat_nmask = 255.255.255.255
reservation_conflict_action = preempt
debug_level = MAX
virtual sebulba_http_122 {
    active = 1
    address = 216.187.127.122 eth0:1
    vip_nmask = 255.255.255.0
    port = 80
    send = "GET / HTTP/1.0\r\n\r\n"
    expect = "HTTP"
    load_monitor = none
    scheduler = wrr
    protocol = tcp
    timeout = 6
    reentry = 15
    quiesce_server = 0
    server motti {
        address = 192.168.6.12
        active = 1
        weight = 2
    }
    server gragra {
        address = 192.168.6.13
        active = 1
        weight = 1
    }
    server koon {
        address = 192.168.6.14
        active = 1
        weight = 2
    }
}
virtual sebulba_http_124 {
    active = 1
    address = 216.187.127.124 eth0:3
    vip_nmask = 255.255.255.0
    port = 80
    send = "GET / HTTP/1.0\r\n\r\n"
    expect = "HTTP"
    load_monitor = none
    scheduler = wrr
    protocol = tcp
    timeout = 6
    reentry = 15
    quiesce_server = 0
    server motti {
        address = 192.168.6.12
        active = 1
        weight = 1
    }
    server gragra {
        address = 192.168.6.13
        active = 1
        weight = 2
    }
    server koon {
        address = 192.168.6.14
        active = 1
        weight = 1
    }
}

Finally, I have a lot of connection timed out with my real servers in 
/var/log/messages :

Nov 29 10:34:43 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:34:43 lb nanny[5987]: shutting down 192.168.6.13:8008 due to 
connection failure
Nov 29 10:34:55 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:36:14 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:36:14 lb nanny[5980]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:36:15 lb nanny[5996]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:36:15 lb nanny[5996]: shutting down 192.168.6.13:8008 due to 
connection failure
Nov 29 10:36:26 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:36:27 lb nanny[5996]: making 192.168.6.13:8008 available
Nov 29 10:37:00 lb nanny[6001]: READ to 192.168.6.14:80 timed out
Nov 29 10:37:00 lb nanny[6001]: shutting down 192.168.6.14:80 due to 
connection failure
Nov 29 10:37:00 lb nanny[5993]: READ to 192.168.6.14:80 timed out
Nov 29 10:37:00 lb nanny[5993]: shutting down 192.168.6.14:80 due to 
connection failure
Nov 29 10:37:00 lb nanny[5992]: READ to 192.168.6.13:80 timed out
Nov 29 10:37:00 lb nanny[5992]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:37:02 lb nanny[5979]: READ to 192.168.6.12:80 timed out
Nov 29 10:37:02 lb nanny[5979]: shutting down 192.168.6.12:80 due to 
connection failure
Nov 29 10:37:12 lb nanny[6001]: making 192.168.6.14:80 available
Nov 29 10:37:12 lb nanny[5993]: making 192.168.6.14:80 available
Nov 29 10:37:12 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:37:14 lb nanny[5979]: making 192.168.6.12:80 available
Nov 29 10:38:09 lb nanny[5996]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:38:09 lb nanny[5996]: shutting down 192.168.6.13:8008 due to 
connection failure
Nov 29 10:38:10 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:38:10 lb nanny[5992]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:38:10 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:38:10 lb nanny[5987]: shutting down 192.168.6.13:8008 due to 
connection failure
Nov 29 10:38:21 lb nanny[5996]: making 192.168.6.13:8008 available
Nov 29 10:38:22 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:38:22 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:39:09 lb nanny[6000]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:39:09 lb nanny[6000]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:39:21 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:39:46 lb nanny[5992]: READ to 192.168.6.13:80 timed out
Nov 29 10:39:46 lb nanny[5992]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:39:55 lb nanny[5980]: READ returned error 104:Connection reset 
by peer
Nov 29 10:39:55 lb nanny[5980]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:39:55 lb nanny[6000]: READ returned error 104:Connection reset 
by peer
Nov 29 10:39:55 lb nanny[6000]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:41:40 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:41:43 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:41:43 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:44:40 lb nanny[6000]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:44:40 lb nanny[6000]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:44:52 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:45:45 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:45:45 lb nanny[5980]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:45:57 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:47:00 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:47:00 lb nanny[5992]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:47:12 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:50:35 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:50:35 lb nanny[5987]: shutting down 192.168.6.13:8008 due to 
connection failure
Nov 29 10:50:47 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:55:03 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:55:03 lb nanny[5992]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:55:03 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:55:03 lb nanny[5980]: shutting down 192.168.6.13:80 due to 
connection failure
Nov 29 10:55:15 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:55:15 lb nanny[5980]: making 192.168.6.13:80 available

Thanks for your help.

Cyrille.



_______________________________________________
LinuxVirtualServer.org mailing list - [email protected]
Send requests to [email protected]
or go to http://www.in-addr.de/mailman/listinfo/lvs-users