inhibit_on_timeout

Hello!

Posting for discussion, not fully-baked...

inhibit_on_failure is required for some services, such as Postfix, which
have upper connection limits. If not used, too many connections to one
real server causes Postfix to stop printing the banner, which makes the
health-check fail, which makes keepalived remove the server, which makes
all existing connections hang, which makes Postfix stay "overloaded"
until it starts to time out sockets ~10 minutes later. This can result in
a domino-effect complete outage, especially when leaving for vacation. ;)

However, if persistence is enabled, inhibit_on_failure doesn't do what is
expected in an actual outage, since without changing any other option,
persistence will still stick to a down server as long as connections keep
trying within persistence_timeout. One can set expire_quiescent_template,
but then the Postfix overload case doesn't work again, because it just
immediately tears down connections that come in from the outside rather
than letting them complete.

It seems that really what is needed is for most healthchecks to have a
tristate result: up, down, timeout, and the timeout case is what should
set weight 0, since the server is likely overloaded. This might even
naturally work without needing anything else, though this depends on the
exact behaviour of a down server (fails immediately versus hanging).

Thoughts?

Yes, it can be (and is being) done with MISC_CHECK, but that has even
more subtleties, as I posted about previously. I'm more interested in the
case that everybody assumes is normal (all of the built-in checkers).

Simon-

------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience.  Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk