Re: too many false alarms

As a follow-up(-on) to Ron's response, I think that the resource being
accessed also plays a role in determining the "acceptable" false alarm rate
and the perceived usefulness of an anomaly detection system. I think it
would be impossible, and unrealistic, to state an all encompassing metric
for your entire organization. For example, a 1-in-10 false alarm rate might
be sufficient for an internal wiki that contains no sensitive information
and most alerts could probably be ignored (or at least investigated with a
lower criticality). That metric, however, is completely unsuitable for a
database server that contains card holder information.

On 24/01/2008, Ron Gula <[email protected]> wrote:
>
> I think that is a very unfair question because you can interpret "false
> alarms" a few different ways.
>
> On one hand, if you have an alert system that says to raise an alert
> when a certain condition is met and it raises the alert in error, this
> is a false alarm.
>
> On the other hand, many times the algorithm to detect a condition works
> great, but it's a poor algorithm and its alerts are referred to as a
> false alarm. For example, alerting on "port scanning" when there are
> more than 5 network connections in 5 seconds might seem great except
> when you realize that some applications do this normally.
>
> For SIMs, I see a lot of people who like to add in context in terms of
> what is "normal" for a host based on its past behavior, or on what sort
> of system it is. This type of correlation can also be in error and cause
> false alarms as well. I've seen some VA/IDS correlation systems that are
> purely OS based and will downgrade some legitimate attacks and elevate
> ones that should not happen.
>
> And lastly, there is a big difference in considering the types and
> quality of alerts you get on your console or pager, as compared to what
> you might be able to see later on when you know a certain IP address is
> suspect or bad. In an incident, the "false positive" alerts you might
> not normally want to be alerted on become very valuable.
>
> So the bottom line, I would not tolerate any errors in the detection
> mechanisms, but avoid calling them false positives.
>
> Ron Gula
> Tenable Network Security
>
>
>
>
> Jon Stearley wrote:
> > what false alarm rate do you tolerate for your current monitoring
> > system?  is 1 false alarm in 4 ok?  1 in 10?  1 in 100?
> >
> > a related question is: what false alarm rate must anomaly detection
> > systems achieve to be useful?
> >
> > i know this is person/site/situation/etc specific, and welcome any
> > ballpark figures or experiences.  thanks.
> >
> > -jon stearley
> >
> >
> > _______________________________________________
> > LogAnalysis mailing list
> > [email protected]
> > http://www.loganalysis.org/mailman/listinfo/loganalysis
> >
> _______________________________________________
> LogAnalysis mailing list
> [email protected]
> http://www.loganalysis.org/mailman/listinfo/loganalysis
>



-- 
Andrew Hay
blog: http://www.andrewhay.ca
email: andrewsmhay || at || gmail.com
LinkedIn Profile: http://www.linkedin.com/in/andrewhay

_______________________________________________
LogAnalysis mailing list
[email protected]
http://www.loganalysis.org/mailman/listinfo/loganalysis