A n00b's Load Balancer Questions

Hi all, I'm hoping for a little direction here from you you folks. I'm

definitely a n00b when it comes to load balancers, though I've always 
loved the concept. Now I think it's time for us to make the plunge, and 
I've got everyone's buy-in here to run with it. Now it's time to produce.

Here's the situation. Our company is a small-ish 24/7 call center in one 
site (we do mirror data to a DR site, but I don't expect to include any 
of that in the LB calculus). Our primary applications are the 
call-center app running on 40-50 Apple Macs (!), an internal webserver 
for additional apps, and another that runs our customer portal. Our 
phone switch and ACD runs on Asterisk, and of course there are several 
'utility' apps running on other machines. Our primary concern is 
high-availability of these applications, not scalability. Interruptions 
of any of these immediately impacts customers. Over time, we have 
employed various methods to keep these apps going if there's a problem 
with a back-end server. What I'd *really* like to do is to employ a load 
balancer to make the essential services that these apps rely upon 
continuously available. If we can do that, it means (1) simplified 
application development, (2) more overall reliability with simpler 
maintenance, and (3) I would finally have a way to transparently update 
and upgrade underlying servers/services.

So, I'd like to outline my wish list as I see it here. What I hope you 
can do is help me understand which of these things is well-handled in 
the LB world, and then help me focus on what products can deliver those 
solutions. I don't expect this to be simple, and I know there are many 
differing views and experiences among you. If you're willing to engage 
me a little on this, I will be eager and grateful to learn from you. 
Here goes:

First, I assume these basic services are handled easily by most LB 
solutions:
- DNS
- NTP (not super critical, since nothing immediately stops if this goes 
off-line)
- SMTP

Then, the expected HTTP/HTTPS.
- As implied above, there's no big performance requirement here. For our 
(external) customer portal, all of that traffic is served over a 3Mbps link.
- SSL/TLS acceleration would be nice, and that seems fairly common even 
on the low-end. 100 CPS would be plenty.
- Users of internal web apps are sensitive to latency, just because the 
desktops are running real-time. They need to stay snappy.
- Of course, these are web *applications*, so persistent connections are 
important, though it seems most everything supports methods for that now.
- I'd like to do some reasonably sophisticated health checks. Basically 
query the web server a few different ways and see that I'm getting 
answers that verify the web service is up, that it has database 
connectivity, etc.

I'm thinking those things above are nothing special. Here are the items 
that are a little more in question:

- SIP telephony. As mentioned above, we use Asterisk servers running 
SIP, IP soft phones on the (Mac) desktops, Cisco AS5400 gateways, and 
Polycom IP phones. What I'm trying to provide for here is a failover 
from the primary Asterisk server to a standby system. There's a few 
obvious ways to do this, but simply I envision a case where the LB would 
be in the loop during call setup, re-directing a (new) call to the 
active system. The Asterisk server and endpoint are then in the call,
and the LB has no more part in it. If the Asterisk server dies, any 
calls connected through it are dropped (that's acceptable), but new 
calls are then routed by the LB to the standby box. In a "graceful 
shutdown" mode, the LB would be notified to send all new calls to the 
standby system. We can then down the primary system after clearing its 
calls.

- Directory and Authentication. We're running a Win 2008 AD mainly for 
user management across the back-end servers. This is kind of like DNS. 
There are methods for availability and redundancy, but the Mac clients 
(particularly) don't always behave nicely when the primary fails. 
"Timeout and try the secondary" just causes uncomfortable hiccups in 
operations.

- SMB/CIFS. We do have a file store on a plain old Win 2003/2008 file 
server. It's accessed by the Macs, of course, and Linux and Windows 
clients too; and it always needs to be there. Keep in mind that the real 
"load balancer" purpose is to merely serve clients from a standby system 
in case of a primary failure here. (I'm not expecting that the LB would 
have any role in synchronizing or monitoring sync of the real servers.) 
Basically, can any LBs proxy a file server?

- SQL Server (2005). I've seen a number of references to LB/ADC fronting 
databases, though usually with Oracle for some reason. Our current plan 
is to run SQL's Mirroring to provide a H/A db server pair. The only 
catch is that this requires specific MS-native drivers that support the
mirroring. There are no native drivers for some client systems (mainly, 
but not exclusively, the Macs), so we end up writing a (Java/JDBC) 
middleware layer to bridge the gap. So, I'm wondering if any LB can 
proxy SQL by talking ODBC or some such to our various client systems, 
while talking "native" back to SQL Server. There would only be one 
"real" SQL server in this case, even though it's really a virtual system.

I think that covers the "wish list" for application delivery. Sound 
nuts? I hope not too bad. Here are some other thoughts about our 
situation that may also influence the solution:

- All of these functions are met performance-wise by two or three real 
servers behind the LB. But, I count up to 8 different functions served 
by the LB, with possibly 3 - 4 web server groups, for perhaps a dozen 
different "virtual" LB applications. Some of these real server groups 
operate on different subnets.

- One possibility, if all were simple functions, would be to simply get 
several old Alteon 180 boxes. People seem to have a lot of regard for 
that line. If our more complex needs (wishes) can't be accomplished, 
then this may be a practical route. This brings up the idea of several 
cheap boxes vs. one higher-end box (two really, since we would obviously 
need an H/A pair of load balancers). We often deploy servers with very 
limited functionality - not so "green", I know, but it reduces the
impact of errors/failures (yes, while increasing the chances of having 
one). I would be concerned about a single high-end box (pair) becoming a 
single point of failure for multiple services. If we have one box, then 
it needs to partition well so we don't screw up one thing and kill 
several at the same time.

- Ease of administration is important (of course!). I saw a note here 
some weeks ago that an F5 LLB requires a knowledgeable consultant and a 
week to setup. That doesn't sound very appealing. And yes, I expect some 
of the more complex functions, if they're doable, to require a more 
complex machine; and I also don't want to learn more than one. So, we'll 
just have to evaluate the trade-off between complexity and the value it 
delivers.

- Application development. We do have the advantage of developing most 
applications that we use in-house, so we can be flexible when it comes 
to interfaces with the LB.

- Cost. Obviously it depends to an extent on the value delivered. We're 
a small company, so we're cost-conscious (still have 10 year-old servers 
running some jobs). It also means that I don't have fixed budgets or 
bureaucratic processes to deal with either. My understanding is that 
some of the entry-level "high-line" boxes are now in the $15-20K range. 
We could do that if delivers all we're looking for. If, however, a 
couple $2K boxes do the trick with some sacrifices (or not?!?), we can
work with that too.

- Finally, I have seen references to newer system incorporating 
security/firewall/ips functions. How robust are these? This box will 
necessarily be in the loop for most externally accessible systems, so I 
wonder to what extent it can function as a secure router, in effect? 
That would theoretically let me move some firewall dollars towards the 
LB project? Sound plausible, or am I asking too much?

If you've made it this far, then thank you for reading!! I'm really 
hoping that some of you experienced folks can point me in the right 
direction or let me know if I'm just dreaming when it comes to some of 
this. Thank you sincerely for your time and insight.

Gary Foster


_______________________________________________
lb-l mailing list
[email protected]
http://vegan.net/mailman/listinfo/lb-l
Searchable Archive: http://vegan.net/lb/archive
http://lbdigest.com Load Balancing Digest
http://lbwiki.com Load Balancing Wiki