Workaround suggestion for retry_interval for hosts - comments?

Hi there,

In netsaint (and Nagios I think) there is no retry_interval setting for 
hosts. So if a node goes down, the first time a service fails, the 
host_check_command is run max_attempts times without any delay between 
runs, and the node is considered down. If the node actually goes down just 
before one of the periodic service checks, this notificaiton of "host gone 
down" could happen with a very short delay indeed, regardless of settings 
for all the services.

Just like several other previous posters on this mailing list, we have a 
need to have some sort of delay & retry scheme for hosts, as well as 
services. A router can be busy, "temporarily down is ok", etc.

As I understand, these are the options:

1) Don't use a host_check_command, and use check_host_alive as a service. 
Then the service will be retried and we get the desired effect. Except now 
in the cgis it will look like the node is up, even when in fact the 
service is known to be hard-down.
2) Use a host_check_command similar to "check_host_alive ; sleep 20" and 
then put in a large max_attempts for the node, giving you about 30 seconds 
of delay pr. attempt. This is quite ugly.
3) My suggestion:

As in 1), setup e.g. check_host_alive as a service. Now create a plugin, 
that finds the last hard state of this service, and returns 2 if 
$hard_state eq 'CRITICAL' and 0 otherwise. Then we get the best of both 
worlds. The host command follows the service with retries and all, and the 
host state "looks right" but with the desired delay& retry scheme. Here we 
happen to be so fortunate that we use MySQL for storing state information 
anyway, so in fact this is not so difficult to do.

Any comments on the stability of such a solution? It seems fine with me, 
and my little mock-up test of it seems to work...  Is there any particular 
reason not to have a retry_interval setting for hosts, just as for 
services? It could default to 0...

Sincerely,

Peter Mørch

P.S: This is just as much a post so that future seekers can find a 
workaround if they are looking for one... If anyone finds it  interesting 
enough, I'll see if I can't remove all the site specifics from my little 
script and post it here - but this is the idea, anyway.


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf