Re: open source artificial ignorance-like systems

[email protected] wrote:
>Logcheck is based upon a log checking program called frequentcheck.sh 
>featured in the Gauntlet(tm) firewall package by Trusted Information Systems Inc.

What goes around...

When I wrote version 1 of frequentcheck.sh it was a "search for the
usual strings" log script. Version 2 was my first experiment with
artificial ignorance. :) Craig liked the idea and asked me if he could
use the script for his own purposes, but since it was licensed code
I suggested that he make his own version. Subsequently, I rewrote
my own version of "retail" (which in frequentcheck.sh was called
logtail....)

Another interesting exploration for log filtering is to use
a bayesian classifier. I did some experiments with that a few
years ago using "bogofilter" to generate a probability that
a message "looked blacklist-y" or "looked whitelist-y"
otherwise it would ask the user to classify the log line.
You then control the probability windows i.e:
- if it's 70% whitelist-y bring it to an admin's attention
- if not and it's 70% blacklist-y discard it
- if not bring it to an admin's attention and ask them to classify
        it as white or black
That, right there, is the adaptive log analysis workflow.
My logbayes experiments are on:
http://www.ranum.com/security/computer_security/code/index.html
and the .tar file has a PDF of a bit about how it works.
There are problems with the idea. Mostly, it's because the
bayesian classifiers out there like bogofilter are too
message-oriented (they're made for Email) - my logbayes
experiment used the INSANELY kludgy method of
writing a line to a file and calling bogofilter on it. Ugh.
Someone needs to write a decent line-by-line classifier
and workflow based on the concept. My experiment
gave decent and intruiging results. The problem I had was
figuring out if it was working right, because I was running
it against log codexes that were so large that manual
review for correct performance was not practical. It
did, however, find the needle in the particular haystack
I was searching from, and I got my customer's $$ so
the story ended happily. :)

mjr.