Re: Intrusion Detection Evaluation Datasets

Paul Palmer wrote:

> Isn't this essentially one of the things that NSS does as part of their 
> product reviews?

Nope, it isn't.

NSS's evaluation is essentially targeted at misuse detectors (which are
the overwhelming majority of commercial products), and does not really
take into account an appropriate definition of false positives or
detection rates, because they don't make really sense for misuse detectors.

> Granted, getting your product reviewed is not free, but I think you would 
> find that it is probably cheaper to have NSS perform the evaluation than 
> investing in the infrastructure to do it yourself.

NSS evaluation technique are not suitable for the evaluation of anomaly
detectors, in particular for academic purposes.

Stefano