Re: multilog filtering issues

On Thu, 4 Dec 2008, richard lucassen wrote:

> Hello list,
>
> Why is this NOT giving output to /tmp/testlog/current:
>
> echo -e "./aaa.bbb.ccc: success" | multilog -* +*success -*aaa.bbb.ccc*
> /tmp/testlog/
>
> while this is giving output to /tmp/testlog/current:
>
> echo -e "./aaa.bbb.ccc: success" | multilog -* +*success
> -*aa.bbb.ccc* /tmp/testlog/
>
> According to the docs http://cr.yp.to/daemontools/multilog.html :
>
> <quote>
> pattern is a string of stars and non-stars. It matches any concatenation
> of strings matched by all the stars and non-stars in the same order. A
> non-star matches itself. A star before the end of pattern matches any
> string that does not include the next character in pattern. A star at
> the end of pattern matches any string.
> </quote>
>
> For me, this is a quite cryptic description. I know that djb's doc is
> very accurate, so I must just miss something crucial here. Maybe it's a
> lack of knowledge of the English language. Can someone shine a light on
> this matter for me?
>
> Target: I want publicfile to only log "success" lines, except queries
> for "certain.file", which occur every minute, something like this:
>
> multilog t '-*' '+*success' '-*certain.file*' ./main


I think the bit that's getting you is "A star before the end of pattern 
matches any string that does not include the next character in pattern."

So, in the negative match:
-*aa.bbb.ccc*
the first star will only match things that don't include an 'a'.

So, these will not be filtered:

./aaa.bbb.ccc: success           # './a' contains 'a'
./ajjjjaa.bbb.ccc: success       # './ajjjj' contains 'a'
./xxxxajjjjjaa.bbb.ccc: success  # './xxxxajjjjj' contains 'a'

But, these will:

./jjjaa.bbb.ccc: success         # './jjj' doesn't contain 'a'
aa.bbb.ccc: success              # '' doesn't contain 'a'


Combine it with the prior rule ('+*success'), and you have even more 
bizarre omissions:

./some.example: success   # './some.example: ' contains 's', so isn't matched by the '+' rule


This is a slightly odd variation on regular expressions, that I suspect 
makes the code much, much simpler. (No backtracking.) But, obviously has 
unintended consequences for people expecting something like POSIX globs.



My guess is that (besides the code/performance benefit) the idea is to be 
as specific as possible with your patterns. And, usually, it's best to put 
'*'s before punctuation or spaces.

e.g. (picking a random line from /var/log/messages), to match:
Dec  4 18:21:41 acer su[14135]: Successful su for root by root

you might use:

'+*:*:* * su[*]: Successful su *'

rather than:

'+*Successful*'

(which wouldn't catch: 'Sep  4 18:21:41 ...' for example -- though this 
is probably a better argument for TAI64N than for pattern-specificity)

(Reply to the list, please. I set Reply-To to avoid splashback from 
auto-responders. [Can someone manually remove the ZORG memorial?])

Best,
Ben