Bleed-over between VIPs? Connection routing issue?

I'm having a strange issue that I've been unable to figure out the last 
few days.
My setup consists of a load balancer and 6 real servers in the High 
Capacity High Availability and Load Balancing configuration.  I've been 
running in this configuration for several years now with mostly flawless 
service.

All of the real servers are configured exactly the same with one small 
exception.  To facilitate testing, in the load balancer I've set 2 of 
the web servers to be served by a different VIP than the 4 "production" 
web servers.  All of the web servers are setup with both of the VIPs 
however.  Normally, I set the apache conf to be identical on all the 
servers regardless of whether they will be hosting the test domains or 
not but this time I happened to not enable them on the production 
servers which allowed me to catch this problem easier.

Every now and then (maybe 1 in 10-50 clicks) I will get the default page 
for one of the random production web servers.

At first I thought this was an ARPing issue but after checking 
everything thoroughly, the real servers are not ARPing for the VIPs.

After much log-watching, I've found that it usually (but not always) 
coincides with removing or quiescing real servers or when adding the 
fall back (127.0.0.1 on the load balancer).  It seems that connections 
are being briefly routed incorrectly in the instant between when a 
server is removed and at other various seemingly random times.

I've turned quiescent on and off and persistence off and on as an 
attempted work-around but the problem still continues.  Oddly enough, I 
don't see this happening with my other VIPs (i.e. the production domains 
are not inadvertently being routed to the test servers).

Has anyone seen this before?  I'm trying to figure out how this could 
possibly happen.

Thanks!

Kelly

-- 
--------------------------------------------
-- Kelly Corbin
-- Network Administrator
--
-- http://www.theiqgroup.com
--
-- The IQ Group, Inc.
-- 6740 Antioch Suite 260
-- Merriam, KS 66204
-- (913)-722-6700 x105
-- Fax (913)722-7264
--------------------------------------------


-- 
Ultra Monkey - http://www.ultramonkey.org/
To UNSUBSCRIBE, email to [email protected], with a body:
unsubscribe ultramonkey-users [email protected]
where "[email protected]" is YOUR email address.