Using Holt-Winters prediction for checking time series (eg interface bps, gauge data: number of faults/bugs/degrees/..)

Dear Ladies and Gentlemen,

I am writing to report preliminary experience with the Holt-Winters (HW)
prediction built into the (beta) devlopment branch of rrdtool (1.1.0).

The HW technique is a means of predicting the next value of time series 
data including 

.. COUNTER data, like interface bps

.. GAUGE data, like number of successes, temperature, number of bugs, 
share price ...

Such data can be collected by a separate collector and stored in an RRD 
that supports the HWPREDICT consolidation function (CF).

A Netsaint/Nagios check can then read the FAILURES CF ( the list of
observed measurements that lie outside +- 2 time the HW predicted
deviation from the HW value, when there are __more__ than a threshold
number of outliers) and react accordingly.

It is so easy.

There is a Perl binding to RRD (the Perl module called RRDs::fetch)  
supplied with rrdtool (there are also TCL and Ruby bindings) that enable
the CF values to be read easily by a service check.

The check can then simply report a fault if there are non zero elements 
in the FAILURES CF (perhaps reporting a link to the rrdcgi that would 
graph the observed values and the error bands for the the time series).

There needs to be some understanding of the HW technique to get the most 
out of this method but all in all, there is nothing to do apart from

.. set up the RRD to contain the time series to monitor. The RRS must be 
created by a version of rrdtool that supports HWPREDICT and have the 
HWPREDICT parameters of

 - alpha, beta, gamma adaptation values (these are used to change the 
models corresponding parameters to attribute more or less weight to 
historical data, seasonal variation etc. They greatly influence the 
degree of 'memory' - of spikes - the models preictions have).

 - the number of rows - observations - of the RRD

 - the number of rows constituting a season (eg for a many counters, 
this would be a days worth of observations [the season is the pattern 
of low increase/variability overnight followed by much greater 
change/variance during the day])

.. arrange for a collector program (eg MRTG) to update the RRD

.. have your check read the RRD as above.

The most tricky thing is having the rrdtool fetch occur after an update;  
with Netsaint, check schedule time can vary considerably.

This seems to be worth looking at.

Yours sincerely.

-- 
------------------------------------------------------------------------
Stanley Hopcroft
------------------------------------------------------------------------

'...No man is an island, entire of itself; every man is a piece of the
continent, a part of the main. If a clod be washed away by the sea,
Europe is the less, as well as if a promontory were, as well as if a
manor of thy friend's or of thine own were. Any man's death diminishes
me, because I am involved in mankind; and therefore never send to know
for whom the bell tolls; it tolls for thee...'

from Meditation 17, J Donne.


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf